Author Archives: medcon

medConfidential Bulletin, 9th March 2018

It has been a while since we last sent a newsletter. Our apologies for that – we have been kept busy on a number of fronts, but rather than spam you with speculations we believe it’s better to communicate when there are significant developments.


New national opt-out for medical records

An announcement has been delayed for some months and there’s still some time until action is taken, but to quote NHS Digital last week:

The Secretary of State has agreed that the national data opt-out will be introduced alongside the new data protection legislation on 25 May 2018. It has also been agreed to present the national data opt-out as a single question to cover both research and planning. Type 2 opt-outs (which currently prevent identifiable data from leaving NHS Digital) will be converted to the new national data opt-out when it is introduced in May. Patients with type 2 opt-out will be contacted directly about this change.

There are still a number of important questions to be answered, but we’re working on those for you. For example, at this point, the Government has not yet confirmed that every data release that would be covered by the Type 2 opt-out will be covered by the new opt-out.

medConfidential has yet to see the final wording of the question, but this announcement is clear confirmation that if you opted out in 2014 (or subsequently), you will be sent a letter about what happened. We also haven’t yet seen the wording of the letter, as we and the other members of CDAG (the Advisory Group) would previously have done, but apparently we are to be consulted on that too. When we have the ability to cite formal statements on the new process, we will update our website – this is likely to be in May.

So, if you have already opted out, the NHS will write to you about the new opt-out model. Whether anyone will tell other people remains unclear. We do hope the Secretary of State won’t snatch defeat from the jaws of a victory which could improve patient confidentiality and everyone’s confidence in how the NHS uses data.


This week: Data Protection Bill

The Data Protection Bill was delayed by political squabbling, but must pass by early May, and is now on a very tight timescale.

medConfidential’s concerns with the Bill relate to something called the “Framework for Data Processing by Government” which, in effect, creates a ‘Data Controller in Chief’ who can ignore the Information Commissioner, and the fact that the Government wishes to deny your ability to access information on how your records are used, if that might be used by someone else at another time in a way which may “prejudice… effective immigration control”.

Thanks to a great deal of work by many concerned groups and organisations, the Government no longer considers this framework above the law, just above enforcement of the law. The Rule of Law requires that justice both be done, and be seen to be done – requiring transparency that Governments and companies often prefer to avoid.


What you can do

Many parts of England have local elections in May. The ongoing stealth reorganisation of the NHS in England (into 44 “Sustainability and Transformation Partnerships” and “Integrated Care Systems”) will give your local council more responsibility for data re-use in your area. No details will be given until after the elections – of course! – but if anything does emerge before that, we’ll let you know.

The health and care issues that most burden the NHS differ from place to place, sometimes quite widely. So when local politicians ask for your vote in the next few weeks, you might ask them what their council would do about the biggest issues in your area.

You can see the top three issues most impacting health in your local authority, and those nearby, on this map:

(Created thanks to current data from Public Health England, and with the help of tools provided by Democracy Club whose volunteers collate and share information on elections across the UK.)


What’s next?

medConfidential keeps working even when we’re not sending newsletters; we won’t spam you if there’s nothing important to say. As you can see from this Bulletin, we are approaching another critical time for patient confidentiality that we hope can be negotiated with far greater success than in 2014! If you appreciate our ongoing efforts, we accept donations. Thank you for your support.


Phil Booth & Sam Smith
9th March 2018

AI and demonstrations of political power

Last September, a company which helps institutions understand data started a new project. What their client wanted, was to tell whether one category of videos could be distinguished from some others. The project was successful on a test dataset, and they produced a demo. The very happy client forwarded this to their boss, who sent it to their boss, and so on, until then the Home Secretary went on TV to say that the Home Office had better technology than Google for blocking ISIS videos. At no point, was there a need to test or explain whether the demo worked beyond the test data. That seems to be the standard for AI – is data processing a place where the rule of law doesn’t matter?

The Department of Health also launched guidance on “decision making” AIs for use in the NHS. Innovations have always come to healthcare and spread based on understanding – there is no need to throw all past lessons out because a PR budget gets spent. Separately, the “Malicious AI report” is worth reading – but already feels dated as the risks are both real and timely, and political imperatives are rarely spun as malicious.

Given the option for a quick headline and a cheap political point, politicians will choose to score it. With digital systems of any kind, there is a temptation to take a shortcut and claim victory.  The Home Office claimed to have an AI which did what it wanted – by ignoring any real world requirements that made things harder. This is not the greatest of precedents for public bodies using AI tools to make decisions, especially on groups who do not command easy political support.

Explainability” is just putting in the extra time to test models and understand how they work – rather than selling the first thing that seems to meet the goal. That faster approach may have short term financial benefits, but it can be more widely toxic as an outcome generally best avoided. The Home Office can make this claim for this AI, as it as the first Department to do so; the next claim will be treated with the greater scepticism that it deserves. We’ve put the Home Office statements into the Government’s ethics framework – which again shows the failures of that framework.

‘Trustworthy technology’ needs to address systemic harms. The first mover advantage on AI will go to those with the least moral concerns about real world effectiveness, until there is a clear reputational harm for continuing to work on systems which are known to be damaging. This is why the ‘most successful’ public sector AI project in the UK is from the Home Office – harms to others are something they have never bothered to avoid.

What started out as a legitimate technology project – can AI help identify ISIS videos? –  demonstrating potential, was spun as something else. Had explainability been a prerequisite of that project as being considered a success as it was claimed, (rather than simply a stage of a trial). Where an entity refuses to follow such processes, as in the drug discovery arena, reputable actors should simply refuse to deal with them as that should be one of the requirements of being seen as a reputable actor.  The Partnership on AI was supposed to consider how to address such issues – but companies outside the partnership aren’t bound by their rules… But many of the staff of those outside would not wish to be barred from working there due to other associations (there, of course, must be a way to demonstrate lessons have been learnt)…

The AI guidance from the NHS contains a checklist, written by “industry” and the Wellcome Trust, which is so vague it barely addresses previous problems, let alone handling future questions. There are no considerations of the principles of ‘trustworthy technology’ by developers, nor any references to equivalent protocols for decision making AIs as we have for determining doctors are trained or new medicines are safe. Claiming you have a phase 0 success is one thing (whether a drug or AI), claiming you have a phase 3 success is quite another – and so it should be with machine learning tools that need to be explained.

Many of the greatest failures of the Home Office are due to technical ineptitude. While their policy can not correctly distinguish an arse from an elbow, technology has moved on sufficiently to do it for them, letting Marsham Street ignore the details while delivering the opposite of human flourishing.

Does HMG wish to be permanently excluded from buying from working with partnership members because it chased a cheap headline?  Does the partnership have the willingness to ensure members deliver “responsible” AI? It is the public headlines and narrative that matters, and the biggest headline about AI in Government is that of the Home Office wanting to choose content purely based on a single suprious claim. Government acts as a single customer to their suppliers; and the reverse must be true for AI & ethics.


Data Protection and Datasets of National Significance

Second reading of the Data Protection Bill is in a week – and Government has still not explained the effects of their proposals to centralise data policy in the periphery of Whitehall. As DCMS struggle with a politically led centre for AI data and ethics, announcements like the one from the Home Office will grow. Not because they have solved any problems, but they have done something which redefines the problem as sufficient for the box to get ticked, political claims to be made, and someone else to pick up the pieces. The Home Office does not care about DCMS politics or policy, but which way will Google DeepMind be lobbying DCMS on this?

Lord Mitchell amended the Data Protection Bill to require public bodies estimate the worth of their “datasets of national significance”. Lord Darzi is thinking along similar lines about a new deal with patients. While both good and worthy initiatives that are deserving of time, there is a risk other policies will make them irrelevant.

Lord Mitchell’s amendment mandates an assessment that should be written down, but under current rules, what NHS England or any public body will be forced to write by HM Treasury is that giving data to private companies that employ UK staff, will create new tax revenues from those staff (since company profits go offshore). One NHS trust working with Google might create a nice deal for themselves from some data – but the rest of the NHS will still have to pay a much higher rate.

What will happen when the public understand that this is how their data gets used, and where the money goes?

Even if the Government take the clause out of the Data Protection Bill, whether UK data should be flowing to tax havens is likely an increasingly important question for public debate.  This question is not going away – NHS Digital already do some checks that they’re not dealing with an empty shell company (PHE’s only meaningful step is to check that the fees are in their bank account). Does Government wish to ignore an issue that will resonate with the public on data, or leave in place the small and sensible steps Lord Mitchell added to the Bill?



The Data Protection Bill reaches the Commons

Updated: 16 April: the Bill has been renumbered again. All clauses 185-188 are now numbered 183-186. No other meaningful changes…

Updated 11 March: short briefing for commons committee stage

The Data Protection Bill has reached the Commons. We have 3 briefings on the Bill and an annex on the proposal to make DCMS the lead department for data processing by Government:

(We were expecting 2nd reading this Tuesday/Wednesday, but it’s possible the Whitehall bickering over the DCMS data grab has delayed it; if DCMS has put the politics of empire building ahead of the legislative schedule is a really good indicator that they shouldn’t take over the GDS data function…). Those two links (which were published after the briefing was first circulated), confirm that what is the Cabinet Office’s ‘data science ethics framework’ may get rewritten by DCMS to become the ‘Framework for Data Processing by Government’. For that task, even the iteration that has been discussed is entirely unfit for purpose.

GDPR and Transparency in Government

The EU’s Article29 Working Party held a consultation on their transparency guidance, and with an efficiency that probably infuriates Boris Johnson, ignores late submissions.

For the UK’s NHS, the GDPR is generally just a restatement of the existing ethical good practice that medical bodies should have been following anyway – but it does provide an opportunity (and necessity) to review past decisions and bring them up to scratch (and blame the EU for having to do it).

The main new provision for the NHS, and the topic of A29WP’s recent transparency consultation, are the provisions about what transparency and provision of information to the data subject means. Even that isn’t that new – but it is something that Government has paid lip service to for some time (remember the junk mail leaflets?). That leaves a simple question:

What should transparency look like in practice?

For the NHS, there must be an electronic report on how data was used. NHS Digital keeps track, and with a digital login to the NHS (via patient online), the patient can see where data went, why, and what the benefits of those projects turned out to be, and if they wish to read the published papers (and simpler explanations) that resulted from those uses.

The rest of UK Government lags behind the NHS and is far more murky. Clearly stated in the “Better Use of Data” section of the Technology Code of Practice is a requirement that “the service should clearly communicate how data will be used”, which is akin to the GDPR. Unusually for a GDS recommendation, there is no exemplar given – here is ours.

The best way for an ongoing transactional service to communicate how data will be used next month, is to show how it was used last month.  For any data derived from a digital service behind a login (e.g. any eIDAS compliant system, such as Gov.UK Verify), on any subsequent login, a full accounting of how data on that data subject was accessed, copied, analysed or disseminated, should be available to that data subject.

The best way to know how your data will be used next month is to see how it was used last month. Processes will change over time, but not that rapidly.

This information must also be accurate. It is unclear what the consequence of providing misleading information currently is, but there should be some in a post-GDPR world. Mistakes are a prime facie breach of fair processing, and potentially cause serious distress which is a clear breach of current law.

Taking an example of where information could and should be provided, let’s look at Universal Credit: How much burden is placed on the entire system by the fact that how data is used inside UC & DWP is clouded in secrecy and consequent distrust?

The transparency obligations from GDPR do not extend to investigation of fraud or crimes, so it is not universal, but there are many other consequences of the current system which can be mitigated by informing citizens. UC is already a fully digital service, where users login repeatedly, and access and reuse of data by DWP is already (mostly) logged.

UC used to have such a screen visible to claimants – but the DWP civil servants insisted it be turned off as the Minister might like it. Of course the Minister would like it, as it would be an evidence base of facts and accurate information for a citizen on what the Department actually did – the thing for which the Minister gets held publicly accountable.  With an audit trail, visible to those involved, there will be fewer scandals that land on the Secretary of State’s desk when the stated policy was one thing but the actions of the Department were contradictory.

It is only where ministers deliberately mislead the House that GDPR accountability is a negative…

Access to Individual level Data

As part of transparency, it must be clear how promises to citizens are met. While the NHS does audits on recipients of data, companies regularly fail them with negligible consequences

Population scale citizen level datasets include an administrative census such as the cancer registry (everyone with any cancer for the last ~30 years), HES (everyone who has been treated in hospital since the early ’90s), or the National Pupil Database (everyone who has attended a state school since the mid-90s), or other large scale sensitive datasets (the rest of the NHS data estate).

When population scale data (that does not respect dissent) is copied out of the infrastructure of the data controller, it is impossible to ensure that promises to patients are kept. There are no technical measures which can provide assurance that what should have happened, actually did. That assurance is what the ‘limited environment’ of an independently run safe setting provide.

It is already standard process to allow access to detailed (decennial population) Census data in a safe setting where queries can be audited. The transparency and information provisions of GDPR should be read as requiring that where queries on a dataset can not be audited, that state must be available to a data subject since it makes much more likely that the promises of a data controller may be broken – because the controller has no means to know they are kept.

The 2017 Annual Report from the National Data Guardian again calls for “no surprises”. As the GDPR brings more data controllers closer to the standards already required in the NHS, the best way to inform a data subject how their data is likely to be used next month, is to show how it was used last month. From accountability, can come trustworthiness.

As the Whitehall machine grinds on, as the opt out moves to DH from NHS England, and as data moves from CO to DCMS, the forgetting happens: institutions forget what happened, and institutional memory is what they wished happened. was just a communications failure, and not a policy failure; etc. Where they forget, we will have to remind them.

A first look at the Opt Out slide decks

NHS Digital has published some information on the new “National Data Opt Out Programme”. The information for patients is still missing entirely, and everything else is subject to change.

The new model

The new consent model has the opportunity to do good – and it appears far clearer than the old. But if the loopholes and failures of the past continue to be included in the new model, it will fail in terms of public confidence, just like

We continue to be promised that every patient who has opted out will get a letter describing the new arrangements, although patients who didn’t opt out may be left in the dark about how their data will be used.

As Dame Fiona Caldicott was asked to create, the new opt out will be a simple question:

“You have the right to opt out of your personal confidential information being used for [these other] purposes beyond your direct care” (Deck A)

While we feared there may be new non-statutory ‘special purposes’ that override the opt out created, there seem to be none. While NHS England, and Public Health England wanted their desire to use data to be more important than any patient’s ability to dissent – the slides currently say that the Department of Health disagrees with them. This is good.

Data ‘anonymised in line with the ICO’s anonymisation Code of Practice’ will be excluded from the opt out, but we’ll come onto that later. The embedding of the list of exclusions in the Deck A list (as above) is probably just going to cause problems for all sides in future years – the language has to work irrespective of how the NHS reorganises next week, and we hope the words in square brackets above are not in the language that patients see.

It was the interactions and nuances between what the public were told, and what public bodies wanted to actually do in practice, which led to the collapses and repeated failure to restart, and which led to its ultimate demise. DH hopes that avoiding sharing details too early will mean no one spots problems until it is too late. That is a theory unlikely to survive contact with the real world – which it has not yet attempted.

A trustworthiness question from medConfidential

What should the response be when a public authority is found to have misled the public about how it used data? What happens when the release register is agreed to be wrong, in a way which is demonstrably misleading? What should a trustworthy organisation do?

The remainder of this blog post is a slightly technical look at various specifics. When the patient information is published, we will write a patient perspective on where things are.

NHS England has started by issuing a tender for suppliers to ACOs and STPs. That doesn’t seem like the ideal place to start.

The loophole that remains

Data that is “anonymised in line with the ICO code” remains outside the dissent from ‘information being used for purposes beyond direct care’.

This is purely a political choice by the Department – while they may feel they have no requirement to cover it, the opt out is a gift of the Secretary of State, and he may decide what it covers, and it is only a minor implementation change to go from one to the other.

Previously, the formulation for the opt out was ‘dissent from data leaving the HSCIC” (or GP) – the focus was on data as it left an institution. And in that, it was possible to argue that it was no longer personal data when it was moving (emphasis on possible).

Now, the formulation is about “use” of data for “purposes beyond direct care”, and is undoubtedly clearer in many ways, but it looks at ‘use’. But someone is going to have to justify to the public how ‘anonymisation’ is a direct care purpose. Less than a week after DH told the Health Select Committee that it believes patients have no privacy right over non-clinical data held by NHS Digital, this looks to be a fundamental flaw, resulting from a fundamental change.

Moving to the formulation to be about use also reduces the scope for the purposes of the “promotion of health”. DH has likely missed (again) the opportunity to explain to patients whether or not “promotion of health” can include promoting McDonald’s salads or alternate tobacco products.

There are clearly cases where 100% of data is needed; but the vast majority of research projects would come to exactly the same answer if they used data for which dissent had been honoured. The Confidentiality Advisory Group was placed on a statutory footing and was deliberately given the ability to give advice to NHS Digital on these questions – but the Government has never got around to even starting that process.

No data recipient is currently asked to justify why receiving 98% of patient data is insufficient – data recipients are not asked whether they wish to receive data on people who dissent from their data being used for purposes beyond direct care. Many academic research projects outsource some of their ethical obligations to the data provider – who in this case, will be ignoring them. Again, the obligations change because of a new focus on ‘use’, and it is unlikely that NHS Digital (and PHE) not asking that question will simply allow ethical researchers to ignore it.

The Department of Health ignoring a problem just means everyone else has to deal with it. That has also not gone well elsewhere (for everyone else, DH may disagree).

The Disease Registries (including cancer)

According to the slide decks, the new opt out will not apply to the cancer registry or other data held by PHE (although after last week, it is expected that this should change – and that has consequences for the system).

If the new opt out model does cover PHE datasets like the cancer registry, this will need to be clearly explained in the letter that goes to patients.

Either way, the ‘cancer charities’ should explain (in a fair way) both the benefits of the new consent model, and the choices that it offers cancer patients (and also what those choices will not protect their community against).

Unlike some other health conditions, those who fundraise for the cancer charities often end up being encouraged to publish sensitive personal data about their diagnosis (such as the type, and date), when fundraising for charities. Where those fields are released unprotected, (whether opt outs are disregarded or not), that may change the institutional conceit around inadequate protection. (But this probably won’t happen until after something goes very badly for a someone to whom they owe a duty of care…).

If the text in the slide decks remains accurate, and there are two divergent dissent systems, it is unlikely that the cancer registry could ever integrate into the NHS and remain a backwater of unlinked and idiosyncratic data sources. That is not a good outcome for those who wish to see a cure for cancer.

We have no information that isn’t public or above – so it may be we’ve misunderstood things. This has happened before. Feel free to get in touch if your reading is different to ours ..

Records from cancer patients across England given to a study at a company paid by a tobacco firm

Update (16/1): PHE consider tobacco companies to be doing ‘medical research’

The Daily Telegraph reports today (15/1/2018) that Public Health England gave the medical histories of many cancer patients to a company for a “trends in … lung cancer” study – a company that works for a tobacco company, while that tobacco company was taking the Department of Health to court over “plain packaging”.

PHE believe NHS rules should not apply to them – this includes other opt outs or rules on data handling.

We expect more details to emerge in coming days, as the entries in the PHE release register are scrutinised, and many more companies that don’t appear on the register come to light. (if you’d like to help/watch, there’s a communal google doc for the PHE register here, plus a second one for all NHS data releases)

Press quote: Sam Smith, a coordinator at medConfidential said:

“This is a system that relies on public trust; a system so flawed that it’s not yet clear whether PHE broke any rules.

“The release of this data relies upon loopholes in the Data Protection Act – and unless Public Health England satisfied each technicality they’re arguing about, then they may have breached 3 different laws by giving sensitive personal data on the treatment of cancer patients to a US company that works for a tobacco firm.

“But even if tighter NHS rules had been followed, the Department of Health still argue any patient opt outs wouldn’t have applied in this case. What are patients supposed to do?

“It is vital for public trust that uses of patient level data for purposes beyond direct are covered by the new opt out.

medConfidential supports an NHS cancer registry that follows all NHS data rules, and where the new opt out model applies to all patient level data for purposes beyond direct care. Every data flow in the NHS should be consensual, safe, and transparent, and that includes a well run cancer registry inside the NHS.

If you may be affected, what can you do?

If you are a patient with cancer do not make any treatment changes based on this news; but if you have any questions, talk to your nurses or support structures.

If you may be affected (ie you have had a cancer diagnosis), a request to the cancer registry to opt out will result in your data being deleted, which they admit will potentially harm your direct care in the future. We can not recommend you do this, but it is the only current option. Given the situation, we suggest you write to your MP, and ask them to ask the Department of Health what they are doing to fix this. 

You may wish to:

  • Say why/how this affects you or your family – in as much/little detail as you wish.
  • Ask your MP to, on your behalf, ask the Department of Health why your only choices are to have your cancer diagnosis given to companies working for tobacco firms, or to have your treatment history be deleted affecting the future care of you or your family. No part of that choice is appropriate.
  • In short, why does data taken from the NHS not have to follow NHS rules?

PHE’s past failures are what they are – there are limited things that can be done to fix them. However, they can be brought in line with the NHS, and catch up to all the improvements that the HSCIC/NHSDigital has made since 2014 (although this failure could still have happened to them, and opt outs would still not have been respected)

Please also join our mailing list for additional information as this evolves over the next few weeks:

Data Protection Bill: “Framework for data processing by Government”

Update February 2018: The Bill has now moved to the Commons, and clauses have been edited and become clauses 185-188. Updates are on the Commons page. This page covers the Bill as it was in the Lords.

Updated early-January 2018 – briefing, including the AI and ethics unit.

Update December 2018: Wider updated briefing for report stage

Imagine a data processing framework for social networks, where facebook get told, don’t worry about all those laws, the framework will take care of what you need to follow; the ICO, Judges, and courts, and human rights can’t touch you; you don’t need to worry anything pesky like following the law, or checking that election ads aren’t paid for in Rubles – since that’s just too hard for you to do, you don’t have to do it.

The first statutory “Framework for data processing” (in Government), snuck into the Data Protection Bill (clauses 175-178, page 99), legalises government using any data for anything it wishes (such Home Office typos or punitive DWP processing). None of the other rules apply besides what Ministers write into the framework, and they can change it at whim.

The framework is only 23 sub-clauses, but 10 of them remove rights, scrutiny, consultation or oversight. It seems this Government has lessons for Henry VIII on using power to show contempt for both citizens and Parliament.

Of course, Government rarely does data processing these days and instead outsources most of it. So this is not just data processing, but a framework about data controllers and the merging data for data processing.  We have spoken of these risks before (and pages 12-13).

If “Data Trusts” replicate the model of tax havens, then this is the framework that lets any sector, starting with Government, be exempted from the law. We have seen the effects of tax loopholes, this creates data loopholes. The next “frameworks” will apply to AI or health data.

Clauses 175-178 and Schedule 2 paragraph 4 must be removed from the Data Protection Bill.

20 years since Caldicott 1

As the House of Lords Select Committee on AI looks at health data, it is only 2 weeks shy of the 20th anniversary the 1997 ‘Caldicott Review’. In retrospect, it understood the world that was coming then, and the review still holds up well for the future that is still coming now. It said:

“Increasing adherence to the principles will reassure patients and those treating them that confidentiality is safeguarded. Such progress should be monitored and appropriately identified, and individuals held to account wherever patient-identifiable data is present in the Service. We believe that the principles outlined here should also be applied to information identifiable to individual patients concerned with their clinical care, and medical research. It is clear that patients expect nothing less.”

20 years on, patients still expect nothing less.

The Review could have said one other thing – that data hygiene should have been treated as a part of clinical hygiene, and thereby integrating information into clinical governance. It would have avoided a great deal of problems over the last few years, but may also have made the the original creation of Caldicott Guardians effectively impossible. It is a step operationalised by Caldicott 3. While it could have led to a very much more Digital NHS today, it is all too easy to forget that hospital hygiene, under the oversight of clinical governance, has had problems from failed incentives around outsourcing, infected by the profit motive.

Today, the demand from profit seeking technical startups is even greater, the desire to skirt the rules intense, and modern startups push a “lobbyist viable product” onto a cash strapped NHS – selling patients’ data as an asset when a parent company the patient may not have heard of inevitably gets bought.

Whatsapp’s sharing of phone numbers to facebook would breach the rules around patient confidentiality; but then lobbyists went to see NHS England, who changed the rules and put the burden on each clinician to choose…

Whatsapp still shares its data to facebook to offer it’s “patients you may know” feature, and shadow profiles, and the other creepiness. Any officially sanctioned messenger could never do this, but NHS England doesn’t care – it has taken a problem off its desk for free, and dumped it onto every clinician in the country.

As NHS Digital takes egregious actions in the name of “burden reduction”, NHS England increases a burden yet further, because it’s reduces the burden on them from lobbyists wanting a change.

Messaging in the NHS remains a problem unsolved at scale – so the lobbyists now swarm offering their solutions to similar but different problems. NHS England, not running any hospitals, caved. Pagers work because they are used only for one thing – when they go ding, the doctor is needed for something at a level of urgency that has been triaged; but also the doctor can ignore it while dealing with something else. Doctor judgement is supreme over the tools – the pager is a busy doctor’s bullshit blocker – ignore that feature at your peril.  End-to-end properly encrypted messaging is not hard, but the easiest of the tasks.

If NHS England commissioned a messaging system, institutionally it would abuse it the same way it does email, and destroy any benefits due to its own worst institutional micromanaging impulses – a problem non-existent in the pager world. But those who would change the system would include changes that distract clinicians, so the status quo continues. Such perverse incentives were well understood by Dame Fiona Caldicott when she was writing 20 years ago.

As a result, Caldicott Reviews 1, 2 and 3, are all still relevant. The history is still relevant for designing consent, and designing out unethical and harmful behaviours. That NHSE abdicates any political responsibility does not mean NHS Digital may do so when designing technical systems.

For example, the “GP at hand” service, a rebadged “Babylon Health” product invested in by DeepMind’s founders, says it has the right to sell the medical records it holds as an asset of the company when babylon get bought (which it must in order to pay back the investors). This is a model that Caldicott 1 and GMC/BMA guidance has previously made clear is not appropriate. But as with the dodgy deal with the Royal Free, it is the NHS institutions left holding the bag as the company takes what it wants. Why did NHS England approve that service with those conditions?

Companies will change their rules for profit, and use their public relations machines to argue the NHS “harms patients” by walking away from a disturbing deal the companies only offer on a take it or leave it basis.

Such a business model may be fine for a profit focussed business with no sense of public purpose or accountability to anything beyond their bottom line, but the AI companies claim a higher standard… but also, they don’t:

Q52 Lord Swinfen: In your view, do investors have a duty to ensure that artificial intelligence is developed in an ethical and responsible way? If so, how should they do this? Should such development be regulated?

Eileen Burbidge: I thought this was an incredibly insightful question when I saw it on the papers for the session. The stark and objective answer, strictly speaking, is that I do not think investors have a duty to ensure ethical and moral behaviour. Most investors sign up to a code of conduct and are authorised persons by the FCA because they have fiduciary responsibilities as a first and foremost point. That is simply the objective current situation.

To be quite clear, our obligation to our investors is to generate as strong a financial difference as possible.

It will be more social pressure and market pressure.

If the companies also refuse any social or market pressure from the NHS, then they have learnt no lessons at all.

It was with that insight that Caldicott 1 laid down strong and sustainable guidance for handling of patient data. Every failure of the last 20 years has come because that guidance was ignored.

Dame Fiona Caldicott was precient in her first report. That first report from 1997 can be applied to AI and genomics – 2 ideas that would have been almost inconceivable when it was written – and we are confident it will apply to whatever comes after AI and genomics.

Let us also hope that there doesn’t need to be a fourth part in this trilogy…

We still haven’t seen the wording of the new consent choice…

We still haven’t seen the proposed new opt out language (or the loopholes hiding in caveats).

It’s due to be discussed publicly this month, and is supposed to deliver on what Dame Fiona was originally asked to do:

“Develop a single question consent model which makes it absolutely clear to patients and users of care when health and care information about them will be used, and in what circumstances they can opt out.” (Annex A)

The question has already been tested in an opinion poll by the Government. This means 12,500 of the public have seen the question; but we have not…

It’ll be interesting to see what those 12,500 people weren’t told about when health and care information about them will be used:

  1. What patient level data doesn’t the opt out cover?
    1. For recipients outside the NHS: is it all patient level data that originated in the NHS? Or just some? If some, what about the rest?
    2. For recipients inside the NHS: All secondary uses, or just some?
      1. Will the separation between direct care and secondary uses be maintained? How is paragraph 125 of the data lake document accounted for?
    3. Where do PHE and other DH ALBs sit – in or out?
  2. What will patients who have opted out already have to do, to receive the maximum protection available under the law when this new choice comes in? (which, by then, will be GDPR/DPA2018).
    1. The opt outs did not cover everything possible under the DPA1998 – will that failure be repeated?

  3. What will patients be told who do want their records used? What are the exceptions? Where patients wish their records used, will they be able to see every way they were used, and what any benefits of those uses were?

  4. Will companies continue to be able to access data on patients who haven’t opted out, in secret, without the patient being able to see who used it for what? (which includes such as commercial re-use licenses or information intermediaries)

Another way to look at it is will all the crosses in our scorecard become ticks?

Commercial re-use licenses (aka “information intermediaries”) are akin to telling an academic department that it can use data for any project by any funder. For good reason, that is unthinkable for academia – it should not be allowed for companies

The research community is building an impressive maginot line of justifications for research – a set of case studies that epitomise the best of british medical science. However, the problem with was not the research uses, but all the other things being hiding behind a fig leaf of research.

Will that happen again?

Data in the rest of Government: AI, and today’s laws for tomorrow’s benefits

AI has finally got Government to take data seriously.

Information is the blood of any bureaucracy – and copying is the circulatory system. “Digital” in its broadest form is just the latest wave of faster photocopiers – decisions keep getting made no matter how fast the machines work. Any good private secretary knows: if you control the paper flow, you steer the decisions.

Just as the Cabinet Office has “spend controls” for technology, there should be flow controls for data. Current data practice in Government is 5 different scandals away from adequacy. As with our work in the NHS, some of those will be public, some of those will be private – the scandal is optional, the improvements are inevitable.

Even where the is a fundamental disagreement about a policy in the non-secret parts of Government, there should be the ability to have a shared factual understanding of how data is used.  But even in the “non-secret” parts of Government, there are legitimate reasons for some projects to have limited information disclosed (fraud detection being an obvious one where some information should be withheld, or generalised). The recent Data Sharing Code of Practice Consultation from the Cabinet Office seems to get that balance right for fraud data.

It would be helpful to have political leadership stand up and say (again) that “Citizens should know how data about them is used, in the same way taxpayers should know how taxpayers’ money is spent.” (quoting Matt Hancock MP – then Minister for the Cabinet Office). But that is only helpful, not necessary, and there are sub-political choices which deliver benefits for the civil service and Departmental priorities absent political leadership.

The Spring 2017 Conservative Manifesto gave a strong and clear vision of how Verify could be at the heart of a Government that was accountable to its citizens (page 3). The question is whether new guidances lets that be implemented, or stymied. The Article 29 Working Party has yet to issue full guidance on the transparency requirements of GDPR – but waiting to do the minimum is not in the spirit of the UK’s desire for leadership in AI, nor goals regarding data.

Government has a range of data sharing powers, and they should all be subject to transparency – otherwise the failings of one will infect public confidence in all.

Fortunately, the range of discussions currently ongoing give the opportunity for the choices of the future to be better than the the past; if that is the desire. The National Statistician’s Data Ethics Committee is a good start, addressing the highest profile and precedent setting issues across Government. However, as with other parts of the Digital Economy Act (Part 5), there should be a Data Review Board for all data sharing decisions that don’t reach NSDEC: it gives a process for which data sharing decisions can be reviewed.

However, if there is an informed citizenry, with citizens able to see and understand how their data has  been used by government, the more complex questions of AI and algorithms become tractable. The status quo will not lead to a collapse in public services, and they will always be able to catch up, the question is only the nature of the political pain that Ministers will suffer because of their civil servants.

A number of Departments believe that “digital transformation” has either failed or is not for them, and they wish to go another way. But the target was always the outcome not the method, and the test is not the pathway, but delivery. How do Departments transform to reflect their current situation? Will they be accountable and to whom?


Bad ideas beyond the AI Review

The recent “AI Review” talks about how “Navigating a complex organisation like the NHS is an unfathomable task for small startups like Your.MD.”. Your.MD being a company which hosts data they collect in the US (ie subject to US law), and outsources coding to eastern Europe (it’s cheaper), and generally cuts every corner that a startup cuts (the corners being things required to protect NHS patients). It should not be too much to ask that anyone wishing to use NHS patient data is capable of hiring someone who can use google to find NHS data rules. Although, as that is a test that DeepMind catastrophically failed, maybe Monty Python was right to hope for intelligence somewhere out in space.


Loopholes (and the Data Protection Bill)

There are some areas where narrow special interests still see themselves as more important than the promises made to patients or citizens, and as more important the principle of no surprises for patients. No bureaucracy can rid itself of the temptation to do what is in the interests of only the bureaucracy. However, it can decide to hold itself to a higher standard of transparency to the people it serves, and let them make the decisions.

With clause 15 it is Government’s demonstrable intent to carve holes into data protection law for its own purposes. To balance such attempts, through the many gateways through which it is possible in the Bill, there must be transparency to a citizen of how their data is copied, even if it entirely lawfully. That allows a separation between whether data is copied, from the rules that cover data copying and access, and an informed democratic debate

AI has finally got institutions to take data seriously. In doing so, it has created a clear distinction between those who understand data from those who do not (the transition from the latter to the former is incentivised as the latter are easier to replace with an AI). As yet, the AI companies don’t yet understand (or wish to understand) the institutions they want data from – which suggests those companies too are easily replaceable (paras 35-49). The AI review also suggests “data trusts” mirror other dodgy kinds and replace the existing principle of safe havens. While some of the large charities can look at that approach as insurance should public confidence in a particular disease registry collapse, and they are entirely wise to do so, a lawful disease registry should command public confidence.

The dash to big data and AI does not mean everything we have learnt about confidentiality, institutions, and public confidence should be thrown away to satisfy startups with less history than a Whitehall cat.

Any external body which seeks to prevent misuse of data will likely fail over time. It is easy for mediocre managers to believe the sales pitch to buy a big system that will “do everything” – to flood a data lake – while earnestly convincing others that this approach will solve whatever problem they think you have. was supported by many sectors, long after the flaws were undeniable, it was only when the public became aware that their tune changed. How will the new bodies learn from that mistake? Do they even think they have to?

The actions of the Home Office have destroyed the integrity of Country of Birth / ethnicity data in the National Pupil Database. At no point was that a discussion – just a directive. It impossible to expect even the most privacy-interested civil servant to defend such a line – even if they remained implacably opposed, their successor eventually would not. There are 3.5 years before the next census. If the first thing the nation’s children know about a census is that it deports their classmates, the fundamental basis for all statistics about the UK will be fatally undermined for a decade. This isn’t counting cranes, it’s extra resources for the areas that think they have high levels of immigration….

Bad ideas never die until they are replaced by better ideas. The misstep in the life sciences strategy illuminates the way that the future may go wrong – there needs to be a way to course correct over time. Just as every use of data in the NHS should be consensual, safe, and transparent; every use of data by Government can be fair, safe, and transparent. That includes uses by any group who cares to assist and be accountable to the individuals whose data they desire.

Is there an interest in a strategic, practical, and available solution? If not, then how many more data scandals will it take, and how high will the associated price be?

There is a better approach, using today’s laws for tomorrow’s benefits.

What is NHS England’s National Data Lake?

A key metric for care, especially complex care, is how patients feel about it. An extremely expensive drug may delay an outcome, but that measures only money and time – not quality of life, which is difficult to quantify. The way the NHS usually does that is via a “Patient Reported Outcome Measure” – how did the treatment make you feel afterwards?  Do the patients who had the treatment, feel it was worth the side effects? As a key metric, we would have expected some movement towards a digital “100% PROMs” (as covered here in point 3). A treatment may be possible, and easy for hospitals, but does it help patient outcomes?

As a result of NHS England’s neglect, its Data Lake will do nothing to improve patient measures (and some issues only appear in PROMS); it’s all accounting measures from NHS England’s accountants and Whitehall micromanagers. Their idea of a consultant is not someone in a white coat, but someone with a calculator. When NHS England talks about improving care, it’s clear its idea of engaging with patients has not improved since was abandoned due to its failure.

The ‘Target Architecture’ document shows NHS England has learnt nothing.  As always suspected (and denied) for, NHS England now admits that it wants “near real time” access to medical records: letting its accountants and expensive management consultants in an office somewhere second-guess your nurse and your doctor, who listen to you.

  • NHS England’s approach is still driven by its desire to do “near real time” monitoring of doctor and nurse performance – there will be no opt out of accountants looking at your records.
    • 125. Sensitive personal and confidential data (which is fully identifiable) will almost certainly be required to achieve interoperability and to facilitate precision medicine and case finding. The NDG Review opt out will not apply.”

  • NHS England clearly considers its micromanaging more important than either Accountable Care or CCG/STP access…  See the middle box in Figure 2 (p14) and para 40 (p10); more on the fundamental problems with the use of ‘secondary datasets’ for operational purposes in this paper.

  • The only reason reason to make this “near real time” (figure 2 – i.e. at least daily) is to force organisations to hand over operational data to those who have no operational role – ‘secret micromanaging of hospitals from afar’

  • According to NHS England, patients will have to opt out again in May next year, given the GDPR, if they do not want their data used. (This suggests medConfidential will have to run an opt out process since NHS England say it will not be otherwise available to you.)
    • It does acknowledge two GDPR opt-outs will have to be respected, but in the process it breaks the existing opt-out choice for patients.

  • “125… The NDG Review opt out will not apply. However the GDPR Right to object and the GDPR right to restrict processing will apply should a data subject wish to exercise that right and certain criteria are met.”
    • Clause 15 of the Data Protection Bill (as laid) gives the Government the ability, by Regulation, to remove those rights. Either way, the Caldicott Opt-Out should be extended to cover the opt-outs possible under GDPR. Since every patient is being written to, that can be made clear to everyone involved, and they can update their choices according to their wishes in the new environment

Fundamentally, paragraph 125 is entirely in conflict with what the Department of Health implied in its response to Caldicott 3. However, it is NHS England which is expected to organise and fund the individual letters to patients, which will have NHS England’s logo on them; possibly not a DH logo. Will DH put its logo on this?

  • The mishandling of the NHS internal market:
    • While section 4.3 in the Annex tries to muddy the waters, stating patient information will be “anonymised or is provided in aggregate views sometimes linked with wider information sources”, in practice these activities are consistently done using identifiable patient data under perpetually-renewed Section 251 ‘support’.
  • Lessons from and Caldicott 3 have simply been ignored:
    • The document is dated 13 July – just one day after the Government’s response to Caldicott 3, on 12 July…
    • NHSE CIO Will Smart said, also on 13 July: “Let’s get away from this distinction between primary and secondary uses of data – it’s just data, let’s start using it”. Source:
  • While nods towards “transparency” and “trust” are scattered throughout the document, with one whole section devoted to the notion of a “diameter of trust”, NHS England provides no indication of how it intends to deliver the Government’s commitments on transparency of access to patients; nothing appears in any of the ‘architecture’ diagrams, or in the text.
    • The model relies heavily on the use of “de-identified” data to avoid answering difficult questions that ignored. Such data would clearly be linkable to the individual – how else “personalised care”? – so, unless NHS England proposes to also ignore the Government’s transparency commitments, patients will have to be told.
    • This also ignores the critical point that de-identification will never be direct care.

  • “… To promote Better Health for all”, para 6 (p3). While prevention must clearly involve ‘behavioural change through information’, a failure to be honest (and transparent) about the two meanings inherent in “promotion” – as in that has contributed to this mess. That they lead off with this phrase may be good – the NHS needs to take a more preventative approach – but it provides cover still for those with more commercial / corrupting agendas.
  • Commercial reuse will continue, and expand. As NHS England has failed to convince anyone else that it should get a new data collection, it will instead expand the dataset collected under HES (which it has powers to do unilaterally).
    • These new datasets will flow into the “Interoperable regional data hubs” (July), aka “regional data ponds” aka “national data lake” (January), and from there, can be drained by anyone who wants to some data.
    • “Information Brokers” (which NHS Digital calls “information intermediaries”) have also expanded beyond NHS, and now include selling access to the cancer registry.

The Department of Health has confirmed that it will write to every patient who has opted out about the new arrangements. NHS England’s long standing refusal to match that for patients who are in the dark risks creating a perverse incentive for those who gave Jeremy Hunt the benefit of the doubt.

One of the changes since 2014 is that there should be a Doctor in charge of Information at NHS England – when the new Chief Clinical Information Officer is appointed, they should not inherit a toxic programme. contained catastrophic failures as a result of being designed by someone who had never spent a day at medical school.

As they plan the data lake (“regional data ponds” or “interoperable regional data hubs”) has the institutional disregard for patients mean that they’ve forgotten every lesson? Using the language of business, forgetting that no one benefits if the NHS just charges money to itself. There are alternatives.

‘data lake’ is not a clinical term, nor a clinical tool. It’s an IT term, for IT people to talk about large technology projects, and to sell the tools for those projects. The benefits to the NHS are entirely tangential from making a small number of techies in suits feel good about what they’re doing without the “burden” of talking to doctors or patients. Paddling in a data lake, they’re able to talk with confidence – despite their talk being irrelevant to solving problems that front line clinicians and their patients face. The data lake is a solution in search of a problem. wanted to link all data and sell it in secret without consent; it’s new data lake links all data and sells it in secret without consent.  What changed in the intervening 4 years?

Those who want to read your medical records have had weekly updates on this document since January; we saw it via a leak in August, despite being lied to repeatedly by a co-author about being shown a copy. It is very clear why. No one from NHS England was willing to put their name on the document (Will Smart’s name was on the January version). Who on NHS England’s Board will be expected to sign off on this mandate to STPs?

Despite the rhetoric from the Department of Health, it’s pretty clear none of the details in NHS England have changed. Even the Wellcome Trust – who saw weekly updates – expect the same problems from 2014 all over again.

We will be here…