Category Archives: News

20 years since Caldicott 1

As the House of Lords Select Committee on AI looks at health data, it is only 2 weeks shy of the 20th anniversary the 1997 ‘Caldicott Review’. In retrospect, it understood the world that was coming then, and the review still holds up well for the future that is still coming now. It said:

“Increasing adherence to the principles will reassure patients and those treating them that confidentiality is safeguarded. Such progress should be monitored and appropriately identified, and individuals held to account wherever patient-identifiable data is present in the Service. We believe that the principles outlined here should also be applied to information identifiable to individual patients concerned with their clinical care, and medical research. It is clear that patients expect nothing less.”

20 years on, patients still expect nothing less.

The Review could have said one other thing – that data hygiene should have been treated as a part of clinical hygiene, and thereby integrating information into clinical governance. It would have avoided a great deal of problems over the last few years, but may also have made the the original creation of Caldicott Guardians effectively impossible. It is a step operationalised by Caldicott 3. While it could have led to a very much more Digital NHS today, it is all too easy to forget that hospital hygiene, under the oversight of clinical governance, has had problems from failed incentives around outsourcing, infected by the profit motive.

Today, the demand from profit seeking technical startups is even greater, the desire to skirt the rules intense, and modern startups push a “lobbyist viable product” onto a cash strapped NHS – selling patients’ data as an asset when a parent company the patient may not have heard of inevitably gets bought.

Whatsapp’s sharing of phone numbers to facebook would breach the rules around patient confidentiality; but then lobbyists went to see NHS England, who changed the rules and put the burden on each clinician to choose…

Whatsapp still shares its data to facebook to offer it’s “patients you may know” feature, and shadow profiles, and the other creepiness. Any officially sanctioned messenger could never do this, but NHS England doesn’t care – it has taken a problem off its desk for free, and dumped it onto every clinician in the country.

As NHS Digital takes egregious actions in the name of “burden reduction”, NHS England increases a burden yet further, because it’s reduces the burden on them from lobbyists wanting a change.

Messaging in the NHS remains a problem unsolved at scale – so the lobbyists now swarm offering their solutions to similar but different problems. NHS England, not running any hospitals, caved. Pagers work because they are used only for one thing – when they go ding, the doctor is needed for something at a level of urgency that has been triaged; but also the doctor can ignore it while dealing with something else. Doctor judgement is supreme over the tools – the pager is a busy doctor’s bullshit blocker – ignore that feature at your peril.  End-to-end properly encrypted messaging is not hard, but the easiest of the tasks.

If NHS England commissioned a messaging system, institutionally it would abuse it the same way it does email, and destroy any benefits due to its own worst institutional micromanaging impulses – a problem non-existent in the pager world. But those who would change the system would include changes that distract clinicians, so the status quo continues. Such perverse incentives were well understood by Dame Fiona Caldicott when she was writing 20 years ago.

As a result, Caldicott Reviews 1, 2 and 3, are all still relevant. The history is still relevant for designing consent, and designing out unethical and harmful behaviours. That NHSE abdicates any political responsibility does not mean NHS Digital may do so when designing technical systems.

For example, the “GP at hand” service, a rebadged “Babylon Health” product invested in by DeepMind’s founders, says it has the right to sell the medical records it holds as an asset of the company when babylon get bought (which it must in order to pay back the investors). This is a model that Caldicott 1 and GMC/BMA guidance has previously made clear is not appropriate. But as with the dodgy deal with the Royal Free, it is the NHS institutions left holding the bag as the company takes what it wants. Why did NHS England approve that service with those conditions?

Companies will change their rules for profit, and use their public relations machines to argue the NHS “harms patients” by walking away from a disturbing deal the companies only offer on a take it or leave it basis.

Such a business model may be fine for a profit focussed business with no sense of public purpose or accountability to anything beyond their bottom line, but the AI companies claim a higher standard… but also, they don’t:

Q52 Lord Swinfen: In your view, do investors have a duty to ensure that artificial intelligence is developed in an ethical and responsible way? If so, how should they do this? Should such development be regulated?

Eileen Burbidge: I thought this was an incredibly insightful question when I saw it on the papers for the session. The stark and objective answer, strictly speaking, is that I do not think investors have a duty to ensure ethical and moral behaviour. Most investors sign up to a code of conduct and are authorised persons by the FCA because they have fiduciary responsibilities as a first and foremost point. That is simply the objective current situation.

To be quite clear, our obligation to our investors is to generate as strong a financial difference as possible.

It will be more social pressure and market pressure.

If the companies also refuse any social or market pressure from the NHS, then they have learnt no lessons at all.

It was with that insight that Caldicott 1 laid down strong and sustainable guidance for handling of patient data. Every failure of the last 20 years has come because that guidance was ignored.

Dame Fiona Caldicott was precient in her first report. That first report from 1997 can be applied to AI and genomics – 2 ideas that would have been almost inconceivable when it was written – and we are confident it will apply to whatever comes after AI and genomics.

Let us also hope that there doesn’t need to be a fourth part in this trilogy…

We still haven’t seen the wording of the new consent choice…

We still haven’t seen the proposed new opt out language (or the loopholes hiding in caveats).

It’s due to be discussed publicly this month, and is supposed to deliver on what Dame Fiona was originally asked to do:

“Develop a single question consent model which makes it absolutely clear to patients and users of care when health and care information about them will be used, and in what circumstances they can opt out.” (Annex A)

The question has already been tested in an opinion poll by the Government. This means 12,500 of the public have seen the question; but we have not…

It’ll be interesting to see what those 12,500 people weren’t told about when health and care information about them will be used:

  1. What patient level data doesn’t the opt out cover?
    1. For recipients outside the NHS: is it all patient level data that originated in the NHS? Or just some? If some, what about the rest?
    2. For recipients inside the NHS: All secondary uses, or just some?
      1. Will the separation between direct care and secondary uses be maintained? How is paragraph 125 of the data lake document accounted for?
    3. Where do PHE and other DH ALBs sit – in or out?
  2. What will patients who have opted out already have to do, to receive the maximum protection available under the law when this new choice comes in? (which, by then, will be GDPR/DPA2018).
    1. The care.data opt outs did not cover everything possible under the DPA1998 – will that failure be repeated?

  3. What will patients be told who do want their records used? What are the exceptions? Where patients wish their records used, will they be able to see every way they were used, and what any benefits of those uses were?

  4. Will companies continue to be able to access data on patients who haven’t opted out, in secret, without the patient being able to see who used it for what? (which includes such as commercial re-use licenses or information intermediaries)

Another way to look at it is will all the crosses in our scorecard become ticks?

Commercial re-use licenses (aka “information intermediaries”) are akin to telling an academic department that it can use data for any project by any funder. For good reason, that is unthinkable for academia – it should not be allowed for companies

The research community is building an impressive maginot line of justifications for research – a set of case studies that epitomise the best of british medical science. However, the problem with care.data was not the research uses, but all the other things being hiding behind a fig leaf of research.

Will that happen again?

Data in the rest of Government: AI, and today’s laws for tomorrow’s benefits

AI has finally got Government to take data seriously.

Information is the blood of any bureaucracy – and copying is the circulatory system. “Digital” in its broadest form is just the latest wave of faster photocopiers – decisions keep getting made no matter how fast the machines work. Any good private secretary knows: if you control the paper flow, you steer the decisions.

Just as the Cabinet Office has “spend controls” for technology, there should be flow controls for data. Current data practice in Government is 5 different scandals away from adequacy. As with our work in the NHS, some of those will be public, some of those will be private – the scandal is optional, the improvements are inevitable.

Even where the is a fundamental disagreement about a policy in the non-secret parts of Government, there should be the ability to have a shared factual understanding of how data is used.  But even in the “non-secret” parts of Government, there are legitimate reasons for some projects to have limited information disclosed (fraud detection being an obvious one where some information should be withheld, or generalised). The recent Data Sharing Code of Practice Consultation from the Cabinet Office seems to get that balance right for fraud data.

It would be helpful to have political leadership stand up and say (again) that “Citizens should know how data about them is used, in the same way taxpayers should know how taxpayers’ money is spent.” (quoting Matt Hancock MP – then Minister for the Cabinet Office). But that is only helpful, not necessary, and there are sub-political choices which deliver benefits for the civil service and Departmental priorities absent political leadership.

The Spring 2017 Conservative Manifesto gave a strong and clear vision of how Verify could be at the heart of a Government that was accountable to its citizens (page 3). The question is whether new guidances lets that be implemented, or stymied. The Article 29 Working Party has yet to issue full guidance on the transparency requirements of GDPR – but waiting to do the minimum is not in the spirit of the UK’s desire for leadership in AI, nor goals regarding data.

Government has a range of data sharing powers, and they should all be subject to transparency – otherwise the failings of one will infect public confidence in all.

Fortunately, the range of discussions currently ongoing give the opportunity for the choices of the future to be better than the the past; if that is the desire. The National Statistician’s Data Ethics Committee is a good start, addressing the highest profile and precedent setting issues across Government. However, as with other parts of the Digital Economy Act (Part 5), there should be a Data Review Board for all data sharing decisions that don’t reach NSDEC: it gives a process for which data sharing decisions can be reviewed.

However, if there is an informed citizenry, with citizens able to see and understand how their data has  been used by government, the more complex questions of AI and algorithms become tractable. The status quo will not lead to a collapse in public services, and they will always be able to catch up, the question is only the nature of the political pain that Ministers will suffer because of their civil servants.

A number of Departments believe that “digital transformation” has either failed or is not for them, and they wish to go another way. But the target was always the outcome not the method, and the test is not the pathway, but delivery. How do Departments transform to reflect their current situation? Will they be accountable and to whom?

 

Bad ideas beyond the AI Review

The recent “AI Review” talks about how “Navigating a complex organisation like the NHS is an unfathomable task for small startups like Your.MD.”. Your.MD being a company which hosts data they collect in the US (ie subject to US law), and outsources coding to eastern Europe (it’s cheaper), and generally cuts every corner that a startup cuts (the corners being things required to protect NHS patients). It should not be too much to ask that anyone wishing to use NHS patient data is capable of hiring someone who can use google to find NHS data rules. Although, as that is a test that DeepMind catastrophically failed, maybe Monty Python was right to hope for intelligence somewhere out in space.

 

Loopholes (and the Data Protection Bill)

There are some areas where narrow special interests still see themselves as more important than the promises made to patients or citizens, and as more important the principle of no surprises for patients. No bureaucracy can rid itself of the temptation to do what is in the interests of only the bureaucracy. However, it can decide to hold itself to a higher standard of transparency to the people it serves, and let them make the decisions.

With clause 15 it is Government’s demonstrable intent to carve holes into data protection law for its own purposes. To balance such attempts, through the many gateways through which it is possible in the Bill, there must be transparency to a citizen of how their data is copied, even if it entirely lawfully. That allows a separation between whether data is copied, from the rules that cover data copying and access, and an informed democratic debate

AI has finally got institutions to take data seriously. In doing so, it has created a clear distinction between those who understand data from those who do not (the transition from the latter to the former is incentivised as the latter are easier to replace with an AI). As yet, the AI companies don’t yet understand (or wish to understand) the institutions they want data from – which suggests those companies too are easily replaceable (paras 35-49). The AI review also suggests “data trusts” mirror other dodgy kinds and replace the existing principle of safe havens. While some of the large charities can look at that approach as insurance should public confidence in a particular disease registry collapse, and they are entirely wise to do so, a lawful disease registry should command public confidence.

The dash to big data and AI does not mean everything we have learnt about confidentiality, institutions, and public confidence should be thrown away to satisfy startups with less history than a Whitehall cat.

Any external body which seeks to prevent misuse of data will likely fail over time. It is easy for mediocre managers to believe the sales pitch to buy a big system that will “do everything” – to flood a data lake – while earnestly convincing others that this approach will solve whatever problem they think you have. Care.data was supported by many sectors, long after the flaws were undeniable, it was only when the public became aware that their tune changed. How will the new bodies learn from that mistake? Do they even think they have to?

The actions of the Home Office have destroyed the integrity of Country of Birth / ethnicity data in the National Pupil Database. At no point was that a discussion – just a directive. It impossible to expect even the most privacy-interested civil servant to defend such a line – even if they remained implacably opposed, their successor eventually would not. There are 3.5 years before the next census. If the first thing the nation’s children know about a census is that it deports their classmates, the fundamental basis for all statistics about the UK will be fatally undermined for a decade. This isn’t counting cranes, it’s extra resources for the areas that think they have high levels of immigration….

Bad ideas never die until they are replaced by better ideas. The misstep in the life sciences strategy illuminates the way that the future may go wrong – there needs to be a way to course correct over time. Just as every use of data in the NHS should be consensual, safe, and transparent; every use of data by Government can be fair, safe, and transparent. That includes uses by any group who cares to assist and be accountable to the individuals whose data they desire.

Is there an interest in a strategic, practical, and available solution? If not, then how many more data scandals will it take, and how high will the associated price be?

There is a better approach, using today’s laws for tomorrow’s benefits.

What is NHS England’s National Data Lake?

A key metric for care, especially complex care, is how patients feel about it. An extremely expensive drug may delay an outcome, but that measures only money and time, not quality of life – which is difficult to quantify. The way the NHS usually does that is via a “Patient Reported Outcome Measure” — how did the treatment make you feel afterwards?  Do the patients who had it, feel it was worth the side effects? As a key metric, we would have expected some movement towards a digital “100% PROMs” (as covered here in point 3). A treatment may be possible, and easy for hospitals, but does it help patient outcomes?

As a result of NHS England’s neglect, the data lake will do nothing to improve patient measures (and some issues only appear in PROMS); it’s all accounting measures from NHS England’s accountants and Whitehall micromanagers. Their idea of a consultant is not someone in a white coat, but someone with a calculator. When NHS England talks about improving care, it’s clear their idea of engaging with patients has not improved since care.data was abandoned due to their failure.

The ‘Target Architecture’ document shows NHS England has learnt nothing.  As always suspected (and denied) for care.data, NHS England now admits that it wants “near real time” access to medical records: letting their accountants and expensive management consultants in an office second guess your nurse and your doctor who listen to you.

  • NHS England’s approach is still driven by its desire to do “near real time” monitoring of doctor and nurse performance – there will be no opt out of accountants looking at your records.
    • 125. Sensitive personal and confidential data (which is fully identifiable) will almost certainly be required to achieve interoperability and to facilitate precision medicine and case finding. The NDG Review opt out will not apply.”

 

  • NHS England clearly considers its micromanaging more important than either Accountable Care or CCG/STP access…  See the middle box in Figure 2 (p14) and para 40 (p10); more on the fundamental problems with the use of ‘secondary datasets’ for operational purposes in this paper.

 

 

  • The only reason reason to make this “near real time” (figure 2 – i.e. at least daily) is to force organisations to hand over operational data to those who have no operational role – ‘secret micromanaging of hospitals from afar’

 

  • According to NHS England, patients will have to opt out again in May next year, given the GDPR if they do not want their data used. (This suggests medConfidential will have to run an opt out process since NHS England say it will not be otherwise available to you)
    • It does acknowledge two GDPR opt-outs will have to be respected, but in the process it breaks the existing opt-out choice for patients.

 

 

  • “125… The NDG Review opt out will not apply. However the GDPR Right to object and the GDPR right to restrict processing will apply should a data subject wish to exercise that right and certain criteria are met.”

 

    • Clause 15 of the Data Protection Bill (as laid) gives the Government the ability to, by Regulation, remove those rights. Either way, the Caldicott Opt Out should be extended to cover the opt outs possible under GDPR. Since every patient is being written to, that can be made clear to everyone involved, and they can update their choices according to their wishes in the new environment

 

Fundamentally, paragraph 125 is entirely in conflict with what the Department of Health implied in their response to Caldicott 3. However, it is NHS England who are expected to organise and fund the individual letters to patients, and it will have their logo on; possibly not a DH logo. Will DH put their logo on this?

  • The mishandling of the NHS internal market:
    • While section 4.3 in the Annex tries to muddy the waters, stating patient information will be “anonymised or is provided in aggregate views sometimes linked with wider information sources”, in practice these activities are consistently done using identifiable patient data under perpetually-renewed Section 251 ‘support’.

 

  • Lessons from care.data and Caldicott 3 have simply been ignored:
    • The document is dated 13 July – just one day after the Government’s response to Caldicott 3, on 12 July…

 

  • While nods towards “transparency” and “trust” are scattered throughout the document, with one whole section devoted to the notion of a “diameter of trust”, NHS England provides no indication of how it intends to deliver the Government’s commitments on transparency of access to patients; nothing appears in any of the ‘architecture’ diagrams, or in the text.
    • The model relies heavily on the use of “de-identified” data to avoid answering difficult questions that care.data ignored. Such data would clearly be linkable to the individual – how else “personalised care”? – so, unless NHS England proposes to also ignore the Government’s transparency commitments, patients will have to be told.
    • This also ignores the critical point that de-identification will never be direct care.

 

  • “… To promote Better Health for all”, para 6 (p3). While prevention must clearly involve ‘behavioural change through information’, a failure to be honest (and transparent) about the two meanings inherent in “promotion” – as in that has contributed to this mess. That they lead off with this phrase may be good – the NHS needs to take a more preventative approach – but it provides cover still for those with more commercial / corrupting agendas.

 

  • Commercial reuse will continue, and expand. As NHS England has failed to convince anyone else that it should get a new data collection, it will instead expand the dataset collected under HES (which it has powers to do unilaterally).
    • These new datasets will flow into the “Interoperable regional data hubs” (July), aka “regional data ponds” aka “national data lake” (January), and from there, can be drained by anyone who wants to some data.
    • “Information Brokers” (which NHS Digital calls “information intermediaries”) have also expanded beyond NHS, and now include selling access to the cancer registry.

The Department of Health has confirmed that it will write to every patient who has opted out about the new arrangements. NHS England’s long standing refusal to match that for patients who are in the dark risks creating a perverse incentive for those who gave Jeremy Hunt the benefit of the doubt.

One of the changes since 2014 is that there should be a Doctor in charge of Information at NHS England – when the new Chief Clinical Information Officer is appointed, they should not inherit a toxic programme. Care.data contained catastrophic failures as a result of being designed by someone who had never spent a day at medical school.

As they plan the data lake (“regional data ponds” or “interoperable regional data hubs”) has the institutional disregard for patients mean that they’ve forgotten every lesson? Using the language of business, forgetting that no one benefits if the NHS just charges money to itself. There are alternatives.

‘data lake’ is not a clinical term, nor a clinical tool. It’s an IT term, for IT people to talk about large technology projects, and to sell the tools for those projects. The benefits to the NHS are entirely tangential from making a small number of techies in suits feel good about what they’re doing without the “burden” of talking to doctors or patients. Paddling in a data lake, they’re able to talk with confidence – despite their talk being irrelevant to solving problems that front line clinicians and their patients face. The data lake is a solution in search of a problem.

Care.data wanted to link all data and sell it in secret without consent; it’s new data lake links all data and sells it in secret without consent.  What changed in the intervening 4 years?

Those who want to read your medical records have had weekly updates on this document since January; we saw it via a leak in August, despite being lied to repeatedly by a co-author about being shown a copy. It is very clear why. No one from NHS England was willing to put their name on the document (Will Smart’s name was on the January version). Who on NHS England’s Board will be expected to sign off on this mandate to STPs?

Despite the rhetoric from the Department of Health, it’s pretty clear none of the details in NHS England have changed. Even the Wellcome Trust – who saw weekly updates – expect the same problems from 2014 all over again.

We will be here…

Early October Update

What’s happening?

By next summer, we will have a new Data Protection law, and a new NHS opt out model.

We should have a good idea by the end of November what the details all look like. The Department of Health are still playing coy – as until everything is final, then nothing is final.

Decisions in recent weeks have moved from a “big bang” launch next March, into a more gradual rolling start, which can deliver when things are ready. This is a great improvement.

Whatever happens, as things continue to change, we’ll update our scorecard of loopholes to keep you informed. It was first published as part of our recent “annual report”, but things will move on as the process rolls on.

 

NHS data: The rolling start has begun

As the rules stand today, any existing opt out will be upheld automatically within the new system. You can go to your GP receptionist, with our existing form, and they will make the change on their system which takes effect. As a patient, how it works – which system is in use – shouldn’t matter to you.

Shortly, the NHS Digital website will appear to give patients the information on how any data is used, and later a service to tell them how your data is used.

At some point next year, hopefully after you can see how your current wishes have been respected, you can express new wishes (as you can now). But the rolling start added by the last Direction to NHS Digital makes this better and simpler: There is no big bang launch, but a steady rollout as things start. If one thing is delayed, the consequences are fewer.

Your consent choice should follow your data, and when/why your wishes were honoured, or not. There are legitimate exceptions, but there are no legitimate secret exemptions.

As progress rolls forward, our scorecard can keep you up to date on where things are.

 

What else might happen next?

Any future Direction from either the Secretary of State or NHS England, must either leave the effects of your existing opt out in place, or explicitly take an action to remove it. Will the Department of Health or the National Data Guardian going to allow the removal of opting out that NHS Digital has already begun?

That would be a dramatic and novel change to public trust in a new system – undermining the point somewhat.

There is potential for a good outcome :

  • Single tick box, online, covering all secondary data uses in and outside the NHS
    • This includes commercial reuse of cancer data by Public Health England. The ICO is investigating our complaint on this topic, which boils down to a simple question: does PHE tell the truth? (evidence says no)
  • Existing care.data opt outs merged into the new one giving a clear path forwards
  • Letters to every patient about the new arrangements.

Any of these would undermine any other good work:

  • Undoing opting out that is already in place
  • Multiple forms being needed
  • Letters not going to every patient who did not opt out
  • Multiple steps (and digital dark patterns – paragraph 2) in the opt out process.

We do not yet know all the details – and we’ll tell you when there’s evidence in practice. But there is progress.

While the NHS is moving towards a rolling start, the road they’re on is akin to an ambulance going down a busy high street with lights flashing – there’s a good idea how long it should take, but if someone does something unwise in the belief that thinking their goal is more important, it might take a little longer while an obstacle is removed. It’s been nearly 4 years since care.data collapsed. If it takes another few months, that’s ok.

But if the NHS data environment is like a normal street, the rest of Government is more like the Wacky Races.

 

What’s next?

In a couple of weeks, we’ll have an update on the National Data Guardian Bill, which is currently queued up in the House of Commons, and the Data Protection Bill, which is currently in the House of Lords.

While our main focus is on medical data, in our free time, we look at the rest of Government – both central and local.

They are themselves doing some thinking about how data is used, and while views are variable, it mostly reflects the initial reactions to care.data in the NHS. That it couldn’t happen there, and why do they need to change anything.

The lesson from the last 4 years, is that doing this properly takes time. We have taught the NHS this once, and will remain here to make every data flow in the NHS consensual, safe, and transparent.

It would be a surprise if the Government chooses to have worse data handling than the NHS. They will have only themselves to blame.

Overview of Current Data Discussions – October 2017

Two weeks after our annual report and rest of government supplement, there are now a number of data consultations on going. We attempt to summarise them all here.

Data Protection Bill

The Data Protection Bill is passing through the House of Lords. Clause 15 if so significant concern, giving Ministers the ability to carve a hole in the Data Protection Act at will – something this Government claimed it wouldn’t do, as it was key safeguard in the Digital Economy Act earlier this year. As written, it is a dramatic change from the data protection status quo, and gives the Government broad powers to exempt itself from the rule of law.

We have a briefing on the Bill for Second Reading in the Lords.

As the NHS moves towards transparency over medical records, the very information provided via transparency must be subject to the same protections against enforced SAR as the records themselves. It’s unclear whether clause 172(1) does this sufficiently.

Implementing the Digital Economy Act: “Better Use of Data”

To plagiarise Baroness O’Neill, whose approach is very relevant here: better than what?

The Cabinet Office are consulting on the Digital Economy Act Codes of Practice. We have a draft response to that consultation, which goes into more detail on a number of issues raised in our rest of government supplement.

As for how that will be used in practice, the Cabinet Office are having meetings about updating their data science ethics framework, and the ODI is seeking views on their proposed data canvas. The canvas is better, but to qualify as science, it can’t just be some greek on a whiteboard, but must include a notion of accountability for outcomes, and falsifiability of hypotheses.

Otherwise, it’s not science, it’s medieval alchemy – with similar results.

Most interestingly, it appears that despite all it’s flaws, the current “data science ethics framework” is in use by Departments, and they do find it useful for stopping projects that are egregiously terrible. So while the framework allows unlawful and unethical projects through, preventing those was not their goal – the hidden goal was to stop the worst projects where every other “safeguard” has demonstrably failed. This is a good thing; it’s just a pity that the previous team denied it existed. The honesty from the post-reset team is welcome – the previous approach included denying to our face that a meeting like this one was taking place, after someone else had already told us the date.

… part 2 is now here

medConfidential on Life Sciences Strategy

The Government has launched its life sciences strategy.

The operative line which underlies all of this from an NHS perspective is:

“This may require some trade-off between trials infrastructure for nursing and for digital,”

Business want such trade offs, but the NHS and patients will likely have something to say about that. Did DH agree to it?

medConfidential coordinator Phil Booth said:

“The missing piece in here is patient consent. While the strategy mentions Dame Fiona’s Review, it doesn’t actually say whether the human tissue they want to buy will be consented or not” (top of page 8)

“Until we see what the NHS itself is planning, there’s nothing in here that wasn’t on the life sciences wishlist 4 years ago from the flawed care.data scheme; and nothing to suggest they’ve learnt any lessons.

“The Government has confirmed that patients who have opted out will be contacted about the new arrangements; but what will those who trusted the NHS to do the right thing be told?

Any Data Lake will fail; there is an alternative

We’ve added some new words to our front page.

Any attempt to solve problems of records following patients along a care pathway that involves putting all those records into a big pile, will either fail – or first breach the Hippocratic Oath, and then fail.

A Data Lake does not satisfy the need for doctors to reassure their patients (e.g. false positive tests), does not satisfy the need for doctors to hold information confidentially from others (e.g. in the case of Gillick competency, or on the request of a patient), or when institutions cannot tell doctors relevant details, e.g. in situations where there is “too much data, but no clear information”.

From the NHS’ national perspective, micromanagers at NHS England will get to reach into any consultation room and read the notes – especially in the most controversial cases. They might be trying to help, and while members of Jeremy Hunt’s Office itself might not reach in (to be fair, they probably wouldn’t), do you believe the culture at NHS England is such that some NHS middle-manager wouldn’t think that is what they were expected to do, urgently, under the pressure of a crisis?

This is also why any ‘blockchain approach’ to health (specifically) will fail. Such technologies don’t satisfy the clinical and moral need to be opaque – deniability is not a user need of your bank statement.

Just as every civil servant recognises aspects of Sir Humphrey in their colleagues, it is the eternal hope of the administrator – however skilled, and especially when more so – that if a complex system worked just as they think it should, everything would be eternally perfect.

Such a belief, whether held by NHS England, DH, or the Cabinet Office is demonstrable folly. If you build a better mousetrap, the system will evolve a better mouse; everything degrades over time.

It was a President of the Royal Statistical Society who talked about “eternal vigilance”. This is why, and it also provides the solution.

As we’ve outlined before, the alternate approach to a leaky Data Lake is to add accountability to the flow of data along a care pathway.

The system already measures how many patients are at each stage, and their physical transfers; it should give the same scrutiny to measuring how many records follow electronically. Where the patient goes, but their data doesn’t, should be as clear to patients as statistics on clinical outcomes – because access to accurate data is necessary for good clinical outcomes.

Interoperability of systems, in a manner that is monitored, is already being delivered by care providers up and down the country. Creating lakes of records is simply an administrator’s distraction from what we already know works for better care.


medConfidential takes donations

medConfidential comment on DCMS Data Protection “Statement of Intent”

DCMS’s intent is clearly to pay more attention to Civil Service silos than citizens’ data.

Sometimes you reveal as much in what you don’t say, as in what you do. Or in what you pointedly ignore…

The ‘Statement of Intent’ document suggests that the confidential information in your medical records deserves no better protection than your local council’s parking list. This is contradicted by both the Conservative Party Manifesto, and the pre-election commitment around Jo Churchill MP’s Bill in the last Parliament to put the National Data Guardian on a statutory footing. So why is DCMS saying no?

DCMS says it intends this to be a “world leading “ Data Protection regime. Even if this weren’t the UK’s implementation of the General Data Protection Regulation, DCMS would know its intent falls short had its Ministers and officials paid any attention to what’s happening outside their own offices.

Three weeks ago, the Government and the NHS committed to telling data subjects when their NHS medical records have been used, and why; and multinationals such as Telefonica have argued clearly and cogently that full transparency to data subjects is the only way forwards with innovation and privacy, without pitchforks.

The Government, however, is doing the minimum legally necessary – and already failing to meet the promises that it was elected on.

Given the Government’s manifesto and the Government’s commitments elsewhere, it is entirely possible for the UK to use digital tools to implement a world class data transparency and protection framework… So why is DCMS saying no?

Everyone’s experience in AI decision-making

Institutions that include everyone understand that great benefit comes from seeing complex issues in many different ways.

The most life-changing, rapid, and one-off decisions people must make are those to do with their health, and the health of their loved ones. Here too, the benefits of diversity are well understood. In medicine, there is a culture of “second opinions” – you can always ask another doctor for their opinion on a choice. This is acknowledged as a great strength of the medical community; indeed, the seeking of diverse (even possibly contradictory) opinions is actively supported by professionals realistic and humble enough to accept that there may not be one single right answer.

So why, as technology progresses, should we choose a lower standard for AIs offering diagnostic assistance to doctors?

Necessary variation in clinical Artifical Intelligence ‘opinion’ will arise only from open competition amongst providers, all respecting the consensual, safe, and transparent use of patients’ data, underpinned by medical ethics.

When you are ill and have a care team today, the decision process available to clinicians deciding your treatment comes not from a single view, but from a comprehensive assessment considering diverse perspectives.

The same should apply when AIs join a care team, which could mean one AI’s analysis spotting something another has assessed as less significant – it should only take one finding to prompt a new consideration. And should we not meet the urgent demand for more doctors, it may be appropriately diverse, ‘always on’, clinical AI assistance tools that could help recast the mix of experience required. (Or perhaps, in a future AI world, patients will be sick of experts…)

Diversity in the medical AI ecosystem will result from the choice of different modelling approaches and the use of different training data, the variation in outcomes (i.e. advice) will come about for similar reasons as today: differing opinions arising from different choices made by different ‘cultures’. No training dataset that systematically excludes some or any community should be acceptable, but different datasets in different models will result in different suggestions – reflecting the humanity of everyone.

The consultation of multiple clinical support systems should be as straightforward as the consultation of any single system in every hospital that meets modern standards for interoperability (FHIR, or the NHS goal of being paperless by 2020). Therefore, when requesting an assessment from an AI clinical support system, it will be just as easy to ask three – unless a monopolistic supplier limits your care to that provided by their models.

Diversity has sound economic reasons too: a mandate for multiple opinions would ensure a healthy, competitive market in AIs for clinical support. Such a mandate wouldn’t raise costs, as it would triple the market size – and it would ensure a continual process of innovation. Over time, as AI improves, there would be minimal risks in moving to newer systems; during the testing phase, four opinions are as easy to consolidate as three.

Also, where patients consent to research, over time, the health outcomes of those patients can become a measure of the different approaches. In that way, if AIs’ outputs are measured on their clinical benefit, “best” can become a clinical outcome – not a marketing claim. Which also delivers on the Government’s commitment that patients should know how their records have been used, and what was learnt from those projects.

In short, a mandate for progress through safe innovation is deliverable today, in line with professional practice and medical ethics, if that is what we want.

 

A National Health Service

Markets around the NHS must themselves be sustainable, and the NHS is in a position – as a research and development institution, and as the data controller in multiple clinical environments – to manage rapid development and testing of AI in a way that a recent flagship project did unlawfully.

It is clear, however, that some institutions within the NHS feel they are required to give up their patients’ data to avoid “falling behind”. All they are demonstrating is their own lack of awareness.

Every AI company is dependent upon masses of data; some may try to ‘free ride’ off the NHS infrastructure, hoping to copy some of the patients’ data that flows through it for profit, without even paying the taxes that fund the NHS. Whatever the case, in every ‘deal’ that is made, the original data controller remains the data controller – and there is no result that cannot be replicated (more cheaply) by another hospital with a similar dataset later, building on shared experience and published results.

Simply believing ‘the smartest guys in the room’ is neither wise, nor the only choice. Novelty can indeed be part of the legitimate research and care process, but the sort of innovations we need cannot involve the secret testing of AIs on humans without their knowledge or consent.

Great risk to the NHS comes only from the perverse incentives of commercial monopoly, grounded in the belief that there should be just a few data silos. (Guess whose?)

Google DeepMind’s Health division might be entirely dependent upon a continued supply of NHS data, but the NHS is not dependent upon Google unless it chooses to be; other AI developers – and search engines – are available. The NHS is not in a position to ensure an effective market in search engines, but just as it already does for health information, it has the authority to do so for clinical assistance; assuming there is the political desire to have a functional and sustainable system.


This will form the basis of Part II of medConfidential’s submission the House of Lords Inquiry on AI.  We’d welcome your thoughts at sam@medconfidential.org / @smithsam