The new Government has inherited an open consultation on allowing data brokers to be ever more creepy, to buy and sell more data with less regulation. It’s coated in the language of helping individuals, but the focus is giving data brokers more permission to sell data about you, and giving you less say in the process.
The consultation assumes data must be copied to be useful, but on-device AI makes copying unnecessary, so DSIT was envisaging a world that’s already ending.
The consultation isn’t about consent – all the positive use cases discussed are things that are already legal today; it’s about allowing data flows in all aspects of your life that you wouldn’t say yes to if they had to ask you. DSIT internalised that when the law says you should be able to see data about you, they know Government, the NHS, and the private sector just ignore that because it’s convenient to keep data subjects as literal subjects of decisions made by others.
Open Banking works because it’s a well-defined interface between many different systems, where there is strong regulation and directly measurable gain and loss (it’s money). The first movers are always those who’ll most benefit with least friction. Open Banking was used by landlords to creep on their tenants, because positive new business models take years. If there was a positive example of where the model would work, it would headline the consultation. Instead it’s akin to “smart data” which generally degrades to copying data behind your back for someone else’s financial gain.
Data policy in DSIT was broken by tech advances
The consultation was conceived a long time ago, before AI got good in the hands of everyone. The former parts of DSIT are being chopped up and spread around. The consultation has outlived the department that envisaged it. There has been much wailing about the change, as it could go either way. In medConfidential’s dealings with DSIT, officials seemed to assume they were the smartest guys in government, so why shouldn’t they do whatever they choose? Because they thought they could, often they thought they simply would – capability was confused with licence.
The consultation is poor, even before we consider that the tools to empower individuals are moving faster than the consultation could imagine. Copying data is often no longer a necessity and should increasingly become a last resort, but DSIT’s consultation (and the NHS app redesign philosophy) assumes that the tech of 2017 is the same as tech of 2027. That’ll interact badly with real life.
What if we look at the goal another way? Much of the data held is accessible somewhere by the person it is about. It may be deliberately difficult to read, but the AIs don’t get bored and can do the drudgery.
You can already make some choices to use data. Your device tomorrow will give you more choices than today.
The future is here, but not yet distributed
The “remarkable new feature” in iOS27 is a Siri that can do things. In a few weeks, any mac user can say “Siri, download the data from the table on this webpage, and add it to mydata.csv on my Desktop” followed by “create and run a shortcut to do that every day”, and it works (more complex cases are “write an applescript to…”). Equivalent instructions work for an iPhone/iPad. One can also task Siri to compare it with the details in your calendar or tone of your emails, to see if it can see a pattern. All on device, no data copied anywhere unless you decide it should be (mostly; different problem).
Those with UK “settled status” – where the right to live and work in the UK is entirely dependent upon the Home Office computers being correct today – get no paperwork. It relies on the website working and people are punished more if you don’t notice the State’s systems have failed. Those who are most concerned could ask Siri to log in twice a week to check their magic code still appears, and warn them if not. That’s currently a website, but if it moves to “app only” with the Digital ID scheme, you’d need to run it on your phone which will supposedly make access harder, but Siri doesn’t care about the hoops – if they’re passable by humans they’re passable by Siri too. (Note to Government Digital Service and NHS App teams: this is not a challenge. We’ll have a more technical piece on the architectural choices/mistakes being made because they give Department of Health in England teams a quieter life at the expense of patients and users).
No data intermediaries are needed – if it’s useful enough that people can do it for themselves, if it’s used, then there will be innovation around it. DSIT has seen that it could mandate in one place and has decided it should be able to mandate data be sold in other places. If the cart really can go before the horse, where’s the obvious example?
The AI running for you on your devices can pull together all the documents in your NHS app, the test results, and then analyse them for you on your device. You can spend as long as you like asking what those actual figures might mean – the ability to talk to a non-judging machine to interrogate the data is a new option, it doesn’t replace anything (it can’t prescribe for a start).
As a patient you’ll gain the ability to generate better informed questions when you do see your doctor. You can have confidence that a process working exclusively for you has looked at your paperwork answering the questions you might want to ask, like “is there a test that was recommended that wasn’t done?”, or “is there a test result that was out of normal that was never followed up on?”, or make unseen connections like “my breath/heart rate monitor tells me I feel terrible after being in the office for two consecutive days” or “I sleep terribly when I email this client late at night” or “on days that pollen is very high I feel terrible at home and fine in venues that have filtered air”. None of those will be possible with Palantir’s FDP. The Palantir-managed “Canonical Data Model” could provide data to inform individual on-device processing if patients are allowed a copy of their data.
Whether the DH/E will empower patients in the NHS App is unclear – the legacy institutions and silos will see removing friction as enabling complete visibility into the mistakes that are made and their cover ups. Unempowered and underinformed patients are less effort to manage, and anything that gives them benefits will be buried in the NHS England treacle (which is one reason NHS England is being abolished). However, improvements will come from the persistence of people who know it will change their care and lives. There are patient centred opportunities that don’t involve copying all data into NHS/Palantir AI Platform where any data written to your Single Palantir Record will be retained and sold. DH/E has many meetings with people who want to buy data, it doesn’t have meetings with those who don’t, so decisions are always about more data and more data sales – meaning they confuse unique magic for commodity analysis.
What DH/E may offer (or refuse to offer) will matter less when Siri can parse what you see on your device, and knows the other details about your environment (it knows the weather in your house, it knows your calendar, it knows where you were and what you did yesterday and the days before, it can measure your voice if you talk to it). Individually empowering approaches were entirely ignored by DSIT in favour of access where there’ll be business traction. DCMS can change direction.
Who will be writing the prompts?
AI and other communities are legitimately concerned that the first use of cheap AI will be creeping on people for profit. After all, the first use of expensive AI was creeping on women for orgasms (and the first use of Facebook was…)
Device by device, there’s no way to creep at scale, only work one app at a time with people who get some benefit from their device working for them.
Who is writing the prompts to show what is possible, and what works today? Whose interest will those prompts be written in? Who will share what knowledge of what works? Who’s going to do things for citizens and patients, not for data brokers – it sure wasn’t DSIT.
Our consultation response will appear below, but this is a consultation on a project that should die with the closure of DSIT. DCMS can make different choices.
—
Enclosures to come: our consultation response
==
Find out what happens next: Sign up for our newsletter (we don’t email often) or get small frequent updates via Substack — free to follow, and we are grateful to all those who can donate to help more of this work.