Category Archives: News

medConfidential comment on DCMS Data Protection “Statement of Intent”

DCMS’s intent is clearly to pay more attention to Civil Service silos than citizens’ data.

Sometimes you reveal as much in what you don’t say, as in what you do. Or in what you pointedly ignore…

The ‘Statement of Intent’ document suggests that the confidential information in your medical records deserves no better protection than your local council’s parking list. This is contradicted by both the Conservative Party Manifesto, and the pre-election commitment around Jo Churchill MP’s Bill in the last Parliament to put the National Data Guardian on a statutory footing. So why is DCMS saying no?

DCMS says it intends this to be a “world leading “ Data Protection regime. Even if this weren’t the UK’s implementation of the General Data Protection Regulation, DCMS would know its intent falls short had its Ministers and officials paid any attention to what’s happening outside their own offices.

Three weeks ago, the Government and the NHS committed to telling data subjects when their NHS medical records have been used, and why; and multinationals such as Telefonica have argued clearly and cogently that full transparency to data subjects is the only way forwards with innovation and privacy, without pitchforks.

The Government, however, is doing the minimum legally necessary – and already failing to meet the promises that it was elected on.

Given the Government’s manifesto and the Government’s commitments elsewhere, it is entirely possible for the UK to use digital tools to implement a world class data transparency and protection framework… So why is DCMS saying no?

Everyone’s experience in AI decision-making

Institutions that include everyone understand that great benefit comes from seeing complex issues in many different ways.

The most life-changing, rapid, and one-off decisions people must make are those to do with their health, and the health of their loved ones. Here too, the benefits of diversity are well understood. In medicine, there is a culture of “second opinions” – you can always ask another doctor for their opinion on a choice. This is acknowledged as a great strength of the medical community; indeed, the seeking of diverse (even possibly contradictory) opinions is actively supported by professionals realistic and humble enough to accept that there may not be one single right answer.

So why, as technology progresses, should we choose a lower standard for AIs offering diagnostic assistance to doctors?

Necessary variation in clinical Artifical Intelligence ‘opinion’ will arise only from open competition amongst providers, all respecting the consensual, safe, and transparent use of patients’ data, underpinned by medical ethics.

When you are ill and have a care team today, the decision process available to clinicians deciding your treatment comes not from a single view, but from a comprehensive assessment considering diverse perspectives.

The same should apply when AIs join a care team, which could mean one AI’s analysis spotting something another has assessed as less significant – it should only take one finding to prompt a new consideration. And should we not meet the urgent demand for more doctors, it may be appropriately diverse, ‘always on’, clinical AI assistance tools that could help recast the mix of experience required. (Or perhaps, in a future AI world, patients will be sick of experts…)

Diversity in the medical AI ecosystem will result from the choice of different modelling approaches and the use of different training data, the variation in outcomes (i.e. advice) will come about for similar reasons as today: differing opinions arising from different choices made by different ‘cultures’. No training dataset that systematically excludes some or any community should be acceptable, but different datasets in different models will result in different suggestions – reflecting the humanity of everyone.

The consultation of multiple clinical support systems should be as straightforward as the consultation of any single system in every hospital that meets modern standards for interoperability (FHIR, or the NHS goal of being paperless by 2020). Therefore, when requesting an assessment from an AI clinical support system, it will be just as easy to ask three – unless a monopolistic supplier limits your care to that provided by their models.

Diversity has sound economic reasons too: a mandate for multiple opinions would ensure a healthy, competitive market in AIs for clinical support. Such a mandate wouldn’t raise costs, as it would triple the market size – and it would ensure a continual process of innovation. Over time, as AI improves, there would be minimal risks in moving to newer systems; during the testing phase, four opinions are as easy to consolidate as three.

Also, where patients consent to research, over time, the health outcomes of those patients can become a measure of the different approaches. In that way, if AIs’ outputs are measured on their clinical benefit, “best” can become a clinical outcome – not a marketing claim. Which also delivers on the Government’s commitment that patients should know how their records have been used, and what was learnt from those projects.

In short, a mandate for progress through safe innovation is deliverable today, in line with professional practice and medical ethics, if that is what we want.

 

A National Health Service

Markets around the NHS must themselves be sustainable, and the NHS is in a position – as a research and development institution, and as the data controller in multiple clinical environments – to manage rapid development and testing of AI in a way that a recent flagship project did unlawfully.

It is clear, however, that some institutions within the NHS feel they are required to give up their patients’ data to avoid “falling behind”. All they are demonstrating is their own lack of awareness.

Every AI company is dependent upon masses of data; some may try to ‘free ride’ off the NHS infrastructure, hoping to copy some of the patients’ data that flows through it for profit, without even paying the taxes that fund the NHS. Whatever the case, in every ‘deal’ that is made, the original data controller remains the data controller – and there is no result that cannot be replicated (more cheaply) by another hospital with a similar dataset later, building on shared experience and published results.

Simply believing ‘the smartest guys in the room’ is neither wise, nor the only choice. Novelty can indeed be part of the legitimate research and care process, but the sort of innovations we need cannot involve the secret testing of AIs on humans without their knowledge or consent.

Great risk to the NHS comes only from the perverse incentives of commercial monopoly, grounded in the belief that there should be just a few data silos. (Guess whose?)

Google DeepMind’s Health division might be entirely dependent upon a continued supply of NHS data, but the NHS is not dependent upon Google unless it chooses to be; other AI developers – and search engines – are available. The NHS is not in a position to ensure an effective market in search engines, but just as it already does for health information, it has the authority to do so for clinical assistance; assuming there is the political desire to have a functional and sustainable system.


This will form the basis of Part II of medConfidential’s submission the House of Lords Inquiry on AI.  We’d welcome your thoughts at sam@medconfidential.org / @smithsam

Summer reading: Data for Research and Statistics in 2017

We’ve previously published an overview entitled “Governance of a Digital Economy in the medium term: AI, blockchain, genomics, and beyond” and a detailed answer to three questions we get asked. We now add this fourth on current questions around research and statistics (and NHS England).

Those questions are:

  1. Should the UK sequence the full genome of the entire population? (pg 1) (no)
  2. Can there be innovative, speculative analysis of individual-level sensitive data in a way that is Consensual, Safe, and Transparent? (pg 2) (yes)
  3. Is there a need for “AI exceptionalism” in data handling and administrative data? (pg 3) (no)
  4. Implications for research and statistics on extending Secondary Uses to facilitate third party time-sensitive micromanagement of Direct Care.

Also related, is the medConfidential response to the Code of Practice on Statistics consultation from the UK Statistics Authority.


Care Episode Histories: There will be a new dataset that replaces HES. The question is where that dataset will be copied, who will access it and on what terms, and whether dissent will be honoured for secondary uses.

The Government’s response to Caldicott 3 has made very clear: Patients will know about every access to their records, whether for direct care or secondary uses.

NHS England’s non-clinical staff look at it purely in terms of data protection; what about the medical profession’s obligation to confidentiality?

For PHE/CPRD, there remain copying loopholes that may remain in theory, and it’s unclear whether they wish their activities to be consensual, safe, or transparent.

The NHS has said that it will use digital tools to tell individuals how data about them is used, and have a public register of data sharing – both are necessary for trustworthiness. Whereas the Government still hasn’t committed to a Register of where it copies any data, including your medical information, under the Digital Economy Act; let alone mandating that its many digital services tell you how your data gets used.

Given the GDS/DCMS claims of digital leadership, being this far behind the NHS has got to get embarrassing. Given Government manifesto commitments, and the unknown hopes of a “Digital Charter”, we’ll see if anything is implemented

Newsletter: medConfidential Summer Roundup, 21 July 2017

Before everyone starts their summer, here are a few ‘tied-up loose ends’ that had previously been left dangling.

Your GP records: If your GP uses TPP or EMIS, you can today begin to see how your data has been accessed. Neither TPP nor EMIS yet cover their research databases, but they will have to shortly because of the findings of the Caldicott Review.

 

Caldicott Review: The Government has finally responded to Dame Fiona Caldicott’s 2016 Review. It has committed that you will be able to see how your records have been used, both for direct care and all other uses. This will be phased in “by 2020”, mostly (we hope) in 2018. If you have opted out, you will be written to about any changes before they happen. Our longer response is now up.

 

Google DeepMind broke the law by copying 1.6 million medical records, according to the Information Commissioner – and the company was rebuked by its own Reviewers. medConfidential’s complaint was found to be true; Google’s statements, not so much.

 

Your DNA: The Chief Medical Officer has opened a “national conversation” about the future of genomics. This starts with patients who have unknown cancers or rare diseases, who may see significant benefits from genomics. But it involves two questions, which boil down to: “Can we do this for your care?” and “If we don’t get an answer now, do you want us to keep your details in a research project which might give you an answer sooner? If not, we’ll run the test again in a year or two.”

There should be no difference to the person’s immediate care, and each patient is given a reasonable choice. If this can be done for cancer genomics, it can clearly be done elsewhere. We would have included a link on how to feed back your thoughts on the CMO’s Report and next steps, but there isn’t one.

 

Patient views on Research: The “Understanding Patient Data” project has run some workshops looking at privacy or research. The blatantly faulty premise of this work is exposed by the Information Commissioner, who has stated: “It’s not privacy or innovation – it’s privacy and innovation.” Had UPD included us in any of their planning, we’d have pointed that out.

 

The GP IT provider TPP: The trial of functionality to allow GPs using TPP’s systems to properly execute their responsibilities to patients should conclude shortly, and – assuming no major problems are found – be rolled out to every GP that uses TPP. With other changes eventually being correctly implemented, this should reassure all sides.

As part of this process, TPP’s notoriously litigious founder instructed lawyers to send us a “reputation management” (defamation) letter, which also said that TPP had no desire to respond to medConfidential.

 

Public Health England is still in denial about its data and consent troubles. Its officials consider themselves part of your cancer care team, despite very few patients having any idea who they are, or why this should be. Beyond the institutional desire to ignore and distrust the Caldicott Consent Choice, what will change?

PHE’s problems are far wider than just consent, but it is a good place to start. Yet another Review is due to be published soon. Will the disease registries move under the NHS umbrella, or will PHE continue to refuse reform – and if so, will you know how your data gets used? Transparency is not the same as respect for confidentiality, but it does make ignoring confidentiality only possible by being dishonest.

 

Funding: We are very grateful indeed to the Joseph Rowntree Reform Trust Ltd for awarding us a further year’s grant, covering 80% of our core funding, that will enable us to continue working towards consensual, safe, and transparent data flows in the NHS – and to defend human rights in the face of your data being copied without your knowledge or approval.

 

Brexit: As Brexit Britain draws closer, and having already introduced measures that try to make NHS staff hassle brown people for documentation, the NHS now faces a three-way stand-off – a ‘Brexit Triangle’: does the Department of Health now direct NHS staff to hassle everyone who looks or sounds ‘foreign, or to hassle absolutely everyone, or do we give in and issue everyone with ID cards?

We may not know the outcome – but we do know that, armed with facts, every patient can speak with the authority of their own lived experience of the NHS. Please do keep informing yourselves, and informing others. Maybe you could share this newsletter with them?

 

What’s next? We hope you enjoy your summer. We have quite a lot to do, getting ready for when Civil Servants and Parliament return in September. Our NHS friends are, of course, working all through the summer. We wish them, and you, well

medConfidential Response to the Government’s Caldicott 3 Response

The foundations on which you build anything are critical. The more complex and interdependent the system, the more vital it is to firmly establish its fundamental principles. As we saw with care.data, when eroded, the whole endeavour can collapse.

The Government’s commitment to transparency is therefore significant. The pressing question is, when it will be delivered – we’ve now been told when it should be delivered, but that’s not quite the same thing. This is important because it is transparency measures that provide the basis for informed consent, a theme we’ll return to at the conclusion.

Regarding each patient opt-out, to prevent data leaving GPs’ systems:

“…we will honour these until 2020 to allow the new national opt-out to be implemented, and for full engagement with primary care professionals and the public.”

Whatever happens in the interim, full engagement has to mean a formal public consultation in 2020, based on the facts as they are known to the public at that point. Anything less would be to break the confidence that the public are being asked to give.

 

The implications of consent

The National Data Guardian, the Department of Health, and NHS Digital have all committed to telling patients how their data is used – both for direct care, and for purposes beyond direct care. This is good. But this is a commitment that must be delivered, consistently and without compromise.

If various dark corners want to continue to grab data in secret, the public will be far less forgiving. care.data may have had a pass, because there was no way for individual patients to know how their data has been used. Under this commitment, they will be able to.

It is doubtful that patients will look kindly on being lied to, again – even if attempts to do so are masked by dodgy definitions of the fence line between one bit of DH and another.

As the NHS begins to understand the implications of confidentiality and consent, medConfidential will be here.

 

Will NHS England and PHE follow the consent model?

In a blatant example of self-important special pleading, page 35 of the Response quotes PHE telling DH and the NDG what they must do, at a point where PHE also refuse to be a part of the solution. (We note also a passive-aggressive defence of Windows XP on page 17.) PHE has repeatedly refused to honour opt outs, dissents, or any other form of objection. The Government has proposed no change to this – why not?

The Government’s Response indicates that, rather than resolve the problem of invoice reconciliation – which has been discussed repeatedly – NHS England has stubbornly dug in its heels, and refused to consider it a problem. So accountants are still to take copies of patients’ identifiable records to check companies aren’t ripping the NHS off – despite there being other, safer, better ways to protect the NHS against fraud. Yet again, NHS England is both part of the problem, and an impediment to the solution – its officials refusing to consider change because they don’t want the effort of having to change the way that CCGs operate.

In a stark illustration of attitudes that still prevail, the day after the Government’s Response was published, the Chief Information Officer of NHS England stood up at a conference and said, “Let’s get away from this distinction between primary and secondary uses of data – it’s just data, let’s start using it”. It appears not only did Mr Smart (like his predecessor) ‘skip medical school’ – he also seems to have skipped reading anything written by the National Data Guardian. Not entirely the lesson you’d hope was understood at the Royal Free…

If the online opt-out process from NHS Digital is discredited from the start by not taking account of PHE’s continued data grab of cancer patients’ records without their consent, medConfidential will run an online opt out process that does.

Of course, Dr Rashbass might continue to ignore those requests too – in the mistaken assumption that just because he thinks of every person who has ever had cancer as his patient, those patients have any idea of who he is or why he’s grabbing their medical history. Clearly, some have yet to learn the important lesson that believing you are a good person, doing a good thing – or even being a good person – is not the same as doing the right thing.

Hopefully the McNeil Review will resolve this outstanding issue, whenever it is published and commenced. However, given the lack of critical engagement, there is still a strong risk that choices may turn out to be a ‘cargo cult’ copy of consensual, safe, and transparent – rather than anything effective. A digital form of the worst of homeopathic quackery.

Whatever U-turns and failures lie ahead, medConfidential will be here.

 

Assuming everybody manages to get this right…

In September 2014 we had a meeting with NHS England, in which the question was asked: “What happens after the care.data problems are resolved?” This was the result (which also looked at backdoor data changes) – at a point where there had already been a commitment that care.data would only be available within a safe setting. Will that commitment be honoured for any and every future dataset?

The principles of that post are sound, and still apply. We don’t yet know what promises will be made about the Data Lake today, only to be broken tomorrow. But what was clear from the Expert Reference Group process was that the data collected will include everything over time – sexual health records, mental health records, abuse records, genomics.

A safe setting means legitimate projects can access the data they need by minimising side effects.

If we were writing on “backdoor changes” today, we’d add PHE and the cancer registry – plus Genomics England, and similarly for other sources of data – but the principles we outline for change remain sound.

Caldicott 3 has delivered something for everyone: whether you wish your data to be used or not, you will be able to see how your wishes have been honoured – and, as the Secretary of State has said: if you don’t want your medical records to be used, they won’t be. All this is capable of being delivered with the Caldicott Consent Choice, implemented properly.

If and when this is delivered, or those patients who are content for their data to be used, the question is what the commitment to transparency will cover. At present, the Hospital Episode Statistics are sent to ~400 different places around the country each month, and NHS Digital hopes none of them has a cyber security accident. It’s only a matter of time.

A safe setting moves data use from “should usually follow” the rules, to “demonstrably always followed” the rules.

The proposed ‘Data Lake’ repeats NHS England’s near-sociopathic disregard of the central fact that in health and care you are  dealing with human beings; people who are usually sick, and often worried. Data is not “the new oil”, nor is it water – and there’s no such thing as a ‘Lake’ of it; there is the collected care episode history of every patient in every UK hospital, for approaching 30 years.

If the current HES are replaced with a more detailed, and even more sensitive, ‘Care Episode Histories’ dataset, that dataset should only ever be available in a safe setting, and all projects – whether for direct care or secondary uses – must be logged for the patient to see. With greater detail, comes some security. It is self-evident that NHS Digital cannot know how data is used once it has left its control, and yet it distributes hundreds of copies of huge numbers of individual-level medical histories that are identifiable (pretending the birth dates of your children are a secret from everyone you know – and others besides…).

Patients will look at accountability trails especially when contentious decisions are made.

 

If Will Smart’s expensive consultants wish to consider themselves as providing Direct Care, then they must appear in the (non-local) direct care – i.e. SCR – access logs made available to patients. The principle of “Hello… my name is…” must apply to all direct care – for, just as a doctor should take the time to explain themselves, real transparency means that NHS England’s micromanagers will be expected to do so as well. When they operate on perverse incentives in a crisis, patients will have the information as to how interventions were handled – which will rarely make a crisis less contentious. It’s not hard to see this won’t end well.

Secondary users, by definition, cannot be expected to introduce themselves to patients – so this applies as much to PHE as it does to NHS England. We assume this separation is why NHS Digital will have two lists of data accesses; the split may  appear odd at first glance, but it is likely better for patients.

If Mr Smart still wants to play about with big databases, with scant regard for human suffering or people’s privacy and dignity, then the Home Office is hiring. But he chose to work in the NHS, which has fundamental values.

Those fundamental values include both confidentiality, and using the data of consenting patients to help other people. Replacing the sale dissemination of data with a better dataset in a safe setting has always been part of the solution the NHS needs. It was deficits in thinking and leadership that led to care.data – and it seems the administrators of NHS England may yet have to learn that in return for changing what they take from us, they may have to change what they themselves do.

Accountability removes possible unwelcome contingencies – which in turn will allow more complex research, in an environment of reduced risk and concern.

A consolidated collection of care episode histories, that are treated as such, could be the basis for a stable data infrastructure in the NHS. A Data Lake cannot.

Whatever promises may be made in order to get hold of the data, it is transparency and accountability to properly-engaged and interested patients that will keep the system honest in the long term. And there will always be competing pressures.

Local councils, for example, will keep funding reports that say local councils should have access to any and all medical records they wish. So we repeat: the Government’s commitment to transparency is significant, for it is transparency that provides accountability in even untrusted systems.

As NHS England moves towards a new, transparent data collection – whatever the plan, and whether it chooses to share it or not – medConfidential will be here.

You shouldn’t pay that – a better approach to invoice reconciliation

Yet again, the Government’s response to Caldicott 3 has decided to ignore the problem of accountants getting masses of identifiable patient information in order to pay some invoices.

A CCG receiving an invoice needs to answer four questions:

  1. Is this a patient we pay for?
  2. Was this care provided to this patient?
  3. Have we already paid for that care?
  4. Has someone else already paid for that care?

The current system ignores question number 4.

As a result of question 2, CCGs expect to get copies of all records on all patients – taking on the burden of keeping them safe – just so as to be able to check anything that they may wish to. The inherent dangers in this are clear, and to do it requires a perpetual ‘temporary’ exception that is only lawful if “necessary or expedient”, and it is unclear whether GDPR will end this in 2018.

All 4 questions, for any particular invoice, are quite straightforward to answer. Given an invoice, the category, and some form of patient identifier, does the data show that there are (some form of) medical records for that treatment, and are those records marked as having been paid by a previous invoice?

Each of the 4 questions needs only a yes or no answer – an answer that won’t reveal any of the contents of the medical records to the accountant doing the check.

The CCG’s accounting needs only the data that is on the invoice (question 2). And even that can be minimised, over time, using the pseudonym system that the Government’s response to Caldicott 3 requires NHS Digital to create for internal use.

While ‘the system’ knows who the patients are, accountants handling bills don’t have to. For corner cases – where there is a question or query – NHS England can adjudicate, based on a “necessary” rather than “expedient” existing process. This also means that any systemic failures or fraud perpetrated against a number of CCGs would be immediately visible, and could then be investigated at a national level. Against one CCG might be a mistake; against many looks criminal.

Even HMRC understands that giving its staff access to the (tax) records of their neighbours will end badly in the public view. Yet NHS England believes the current invoice reconciliation system should continue.

In its reading of Caldicott 3, NHS England would rather remain part of the problem than become part of the solution. Its officials’ flawed obsession with a Data Lake means they cannot politically support anything that doesn’t involve more copying of data.

Whether that approach meets the lawful test of expediency, and GDPR, remains to be seen.

medConfidential comment on the Government’s response to the Caldicott 3 Review

medConfidential’s comment on the Written Ministerial Statement responding to the Caldicott 3 Review

While more details will emerge over the next several weeks, and given this is only a response to Dame Fiona Caldicott’s Review (and not any of the work by NHS England which depends upon it), medConfidential is in the first instance cautiously positive.

Original statement: http://www.parliament.uk/business/publications/written-questions-answers-statements/written-statement/Lords/2017-07-12/HLWS41/

In summary, the Statement says a number of things:

  • Patients will be offered a digital service through NHS.uk that will enable them to see how their medical records are used: both for direct care, and secondary uses beyond direct care.
  • Existing opt-outs preventing patients’ data being extracted from GP practices are protected until at least 2020.
  • There will be further consultations on the details of any changes.
  • Patients who have opted out will be written to about the Caldicott consent model when implementation is finalised (but before changes take effect).
  • NHS Improvement will begin to take cyber security into account. CQC now do.

Reflecting the very strong response from front-line clinicians and technical staff to the WannaCry ransomware outbreak, the Statement is very strong on cyber-security. Whether the analogue administrators that caused so much unnecessary hassle during that event have learnt lessons will become clear, next time…

With the newly-digital DCMS about to launch the Data Protection Bill, will the Government actually deliver on its commitment to a Statutory National Data Guardian?

Phil Booth, Coordinator of medConfidential said:

“We welcome the clear commitment that patients will know how their medical records have been used, both for direct care and beyond. This commitment means that patients will have an evidence base to reassure them that their wishes have been honoured.

“Some of the details remain to be worked out, but there is a clear commitment from the Secretary of State. The focus on digital tools shows the benefit to the whole NHS of the work towards NHS.uk. It is now up to NHS Digital and NHS England to deliver.

“The wait for consensual, safe, and transparent data flows in the NHS is hopefully almost over, and then new data projects can move forwards to deliver benefits for patients and vital research. Today’s announcement is about fixing what NHS England had already broken. The perils of a National Data Lake may lie ahead, but we hope lessons have been learnt, so we don’t end up back here in another 4 years.”

Google now tries to blames Doctors and Snapchat for its unlawful behaviour

Responding to Google’s claims that doctors “use” Snapchat to send photos for a second opinion, coordinator of medConfidential Phil Booth said: “Had Google managed to buy Snapchat, they wouldn’t have said anything about this. The Report blames doctors for hygiene, and the hospital for it’s IT systems, everyone but Google. Now they’re blaming doctors for their choice of secure messaging apps to care for patients with whom they have a direct care relationship – something Google clearly fails to understand.”

If the assertions are based on evidence acquired in the Review, that should have been reported to CQC – unless there was a see no wrong, hear no wrong policy in place. Google provided no evidence that Doctors actually do this, just that they could install an app. They could also use any google messaging tool (except no one uses any of them). We fully expect DeepMind will “surprisingly” come out with a messaging app for doctors, which will be no better than email, and so solve none of the widely understood problems that mean fax machines are still useful. 

Doctors are responsible for safely caring for their patients, and it’s up to them which safe and lawful tool to use. The only reason DeepMind care is they have an tool to sell; and they’re still in denial that they way they built it was unlawful.

We’re mostly surprised that Google didn’t use this to kick Facebook; but perhaps they didn’t want to criticise another member of the Partnership on AI…

Original press release here: https://medconfidential.org/2017/medconfidential-initial-comment-on-the-google-deepmind-independent-reviewers-report/

medConfidential initial comment on the Google DeepMind Independent Reviewers’ report

UPDATE 2pm: responding to Google’s claims that doctors use secure messaging to send photos, Phil Booth said: “Had Google managed to buy Snapchat, they wouldn’t have said anything about it. The report blames doctors for hygiene, and the hospital for it’s IT systems. Now they’re blaming doctors for their choice of secure messaging apps to care for patients with whom they have a direct care relationship.”

Doctors care for their patients, and it’s up to them which safe and lawful tool to use. The only reason DeepMind care is they have an tool to sell; and they’re still in denial that they way they built it was unlawful.


The report answers none of the obvious questions that a supposedly independent Review of unlawful data copying should have answered.  

The ICO confirmed on Monday that DeepMind Health’s deal with the Royal Free had broken the Data Protection Act in at least 4 ways [1], and they have been given weeks to fix it. There is now a formal undertaking in place for correction of their project’s ongoing breaches of the Data Protection Act [2]. As of this week, DeepMind remains in clear breach of UK privacy laws. (page 7)

The National Data Guardian’s letter, referred to by the Review, shows clearly that DeepMind were aware of the unlawful nature of their processing last December[3] and the Review suggests they chose to do nothing about it.

In addressing “law, regulation and data governance”, the Reviewers say “We believe that there must be a mechanism that allows effective testing without compromising confidential patient information” (page 9, right column). So many people agree that there are already such processes – DeepMind just didn’t use any of them. It is unclear why the “Independent Reviewers” feel this is anyone but Google’s problem. (Here’s the sandbox for Cerner – which the Royal Free uses.)

If, as Prof John Naughton analogises, the Royal Free’s response to the ICO decision was “like a burglar claiming credit for cooperating with the cops and expressing gratitude for their advice on how to break-and-enter legally”, this report is DeepMind saying “It wasn’t me! Ask my mum…” thinking that’s an alibi.

DeepMind accepts no reponsibility [4], and its Reviewers seem happy with that.  Which, given DeepMind’s broad AI ambitions, should frankly be terrifying…

Responding to the Review, medConfidential Coordinator Phil Booth said:

“If Page 7 (right column) is accurate in its description of record handling at the Royal Free, then CQC must conduct an urgent inspection of data hygiene at the hospital; or was this just “independent” hyperbole to make Google look good?”

“The Reviewer’s way to not criticise DeepMind is to avoid looking at all the things where DeepMind did anything wrong. The Reviewers may think “this is fine”, but anyone outside the Google bunker can see that something has gone catastrophically wrong with this project.”

“Google DeepMind continues to receive excessive amounts of data in breach of four principles of the Data Protection Act, and the Independent Reviewers didn’t think this worth a mention. DeepMind did something solely because they thought it might be a good idea, ignorant of the law, and are now incapable of admitting that this project has unresolvable flaws. The ICO has forced both parties to fix them within weeks having ignored them for approaching 2 years.

“DeepMind Health needs real senior management with a experience of caring for patients, i.e. a Regulated Medical Professional, as a Chief Medical Officer. The second paragraph on the inside front cover (which isn’t even a numbered page in the printed document, but page 2 in the PDF) shows how badly they have failed from the start.”

For further information or for immediate or future interview, please contact Phil Booth, coordinator of medConfidential, on 07974 230 839 or coordinator@medconfidential.org

 

Notes to editors:

  1. Information Commissioner’s Office summary of their finding https://ico.org.uk/about-the-ico/news-and-events/news-and-blogs/2017/07/royal-free-google-deepmind-trial-failed-to-comply-with-data-protection-law/
  2. The ICO requires that the Royal Free and DeepMind take actions within a month of the undertaking issuance – page 7. https://ico.org.uk/media/action-weve-taken/undertakings/2014352/royal-free-undertaking-03072017.pdfMany of these issues were highlighted to DeepMind by MedConfidential last year, and which they have repeatedly and systemically ignored.
  3. Sky News reported in May that the unlawful nature of the DeepMind data processing was first formally brought to the Royal Free & DeepMind’s attention in December 2016 by the National Data Guardian. http://news.sky.com/story/google-received-16-million-nhs-patients-data-on-an-inappropriate-legal-basis-10879142 Paragraph 4 of the letter from the National Data Guardian to the Hospital clearly shows that they were first formally of their legal failings in December.
  4. Details of medConfidential’s complaint are available here:
  5. This complaint has now been vindicated by the investigation, despite an extremely strong PR response from Google. Contemporary quotes from project advocates, which now ring hollow, include: [all emphasis added]a) Mustafa Suleyman, Co-Founder at DeepMind, has said:

    i) “As Googlers, we have the very best privacy and secure infrastructure for managing the most sensitive data in the world. That’s something we’re able to draw upon as we’re such a core part of Google.” [Guardian, 6/5/16]
    ii) “We have, and will always, hold ourselves to the highest possible standards of patient data protection.” [Daily Mail, 4/5/16]
    iii) How this came about all started with Dr Chris Laing, of the Royal Free Hospital: “We went for coffee and ended up chatting for four hours.” [BBC News Online, 19/7/16]
    iv) More recently, in an interview with Mr Suleyman published on 20/3/17: “When pushed on how the public would be assured that its sensitive data was safe, Suleyman replied, “first there is the law”.” [Digital Health, 20/3/17]

    b) George Freeman MP, at the time a Minister in the Department of Health: “NHS patients need to know their data will be secure and not be sold or used inappropriately, which is why we have introduced tough new measures to ensure patient confidentiality.” [Daily Mail, 4/5/16]

    c) Professor Hugh Montgomery, (consultant for Google’s DeepMind project) said, on Radio 4’s PM programme on 4 May 2016:

    i) “So this is standard business as usual. In this case, it was a standard information data sharing agreement with another supplier, which meets all of those levels of governance. In fact, the agreement there, or the standards of management of those data, meets the very very highest levels. It meets something called HSCIC level 3, which most hospitals trusts don’t even reach.” [Recording of audio available, see link below]
    ii) “So firstly, this isn’t research. Research is governed by an entirely separate process that would require anonymisation of data and all sorts. This is data processing.”
    iii) “It’s fair to say again that not only is this data at the very highest standards, and beats every standard, and more in the United Kingdom. But the data is encrypted end-to-end, and they have to, like everyone else in the health service, stick to the law.”
    iv) Recording of audio available at: https://www.dropbox.com/s/cfimojgec24rlrj/
    20160504­deepmind­radio4­pm.mp3?dl=1
    20160504­deepmind­radio4­pm.mp3?dl=1

    d) Will Cavendish, now Strategy Lead for DeepMind Applied, formerly Informatics Accountable Officer at the Department of Health, said (when IAO):

    …“The vital importance of trust, security, and cyber security.” … “To be honest, it used to be that not a week goes by, now it’s not a day goes by, without stories of hacking, data leaks, inadvertent data sharing. This absolutely erodes the trust that underpins the work that we do.” https://www.youtube.com/watch?v=5Ej3PRF1jUw&t=2h15m5s

    e) Dr Julian Huppert, Chair and “on behalf of the Panel of Independent Reviewers for Google DeepMind Health” said in an e-mail to medConfidential on 6/7/16:

    i) “one of our roles is to look in detail at how DeepMind Health uses patient data, and to confirm that it complies with the highest ethical and regulatory standards.”
    ii) “We believe from what we have seen so far that DeepMind has a clear commitment to the Caldicott Principles, and that they have to date been honest in their public and private comments. We also believe they are willing to work constructively with regulators, and remain within the law.

     

  6. DeepMind’s response to the ICO finding has been to blame everyone but themselves. As they begin to regularly refresh part of their Review board, perhaps Shaun Spicer will be available to help.

 

-ends-

[PRESS RELEASE] Google DeepMind deal with the Royal Free Hospital broke the law

The Information Commissioner’s Office has today ruled that the deals which gave Google DeepMind copies of 1.6 million patients’ hospital records are unlawful:

https://ico.org.uk/action-weve-taken/enforcement/royal-free-london-nhs-foundation-trust/

The ICO’s ruling determines that the deals breached four of the Data Protection principles:

https://ico.org.uk/media/action-weve-taken/undertakings/2014353/royal-free-undertaking-cover-letter-03072017.pdf

medConfidential first complained to the National Data Guardian and ICO in June 2016. [1]

In February 2017, the National Data Guardian said that copying of patients’ data to develop the Streams app was on an “inappropriate legal basis”:

http://news.sky.com/story/google-received-16-million-nhs-patients-data-on-an-inappropriate-legal-basis-10879142

Google DeepMind – the AI company developing the app – has given various contradictory quotes about its intent over time, repeatedly asserting that what it was doing was lawful. [2]

Apparently entirely coincidentally, the “Independent Reviewers” of Google DeepMind Health have a report due out, via the Science Media Centre at 00:01 this Wednesday. The timing may be a coincidence – just as it was apparently a complete coincidence that the Royal Free released a press release about how wonderful the project was, without mentioning the word Google once, 72 hours after receiving the letter from the National Data Guardian saying the data use was unlawful. [3]

On seeing the ICO’s ruling, Phil Booth, coordinator of medConfidential said:

“We look forward to Google DeepMind’s Independent Reviewers’ report on Wednesday.”

For further information or for immediate or future interview, please contact Phil Booth, coordinator of medConfidential, on 07974 230 839 or coordinator@medconfidential.org

Notes to editors

1) Details of medConfidential’s complaint are available here:

a) Timeline of events, as of 31/5/16: https://medconfidential.org/wp-content/uploads/
2016/06/medconfidential-deepmind-timeline.pdf

b) Complaint to Regulators: https://medconfidential.org/wp-content/uploads/2016/06/
medconfidential-to-regulators.pdf

c) Shortly after submission, the MHRA found that the project should have been registered with them (and wasn’t): https://techcrunch.com/2016/07/20/
deepminds-first-nhs-health-app-faces-more-regulatory-bumps/

2) This complaint has now been vindicated by the investigation, despite an extremely strong PR response from Google. Contemporary quotes from project advocates, which now ring hollow, include: [all emphasis added]

a) Mustafa Suleyman, Co-Founder at DeepMind, has said:

i) “As Googlers, we have the very best privacy and secure infrastructure for managing the most sensitive data in the world. That’s something we’re able to draw upon as we’re such a core part of Google.” [Guardian, 6/5/16]

ii) “We have, and will always, hold ourselves to the highest possible standards of patient data protection.” [Daily Mail, 4/5/16]

iii) How this came about all started with Dr Chris Laing, of the Royal Free Hospital: “We went for coffee and ended up chatting for four hours.” [BBC News Online, 19/7/16]

iv) More recently, in an interview with Mr Suleyman published on 20/3/17: “When pushed on how the public would be assured that its sensitive data was safe, Suleyman replied, “first there is the law”.” [Digital Health, 20/3/17]

b) George Freeman MP, at the time a Minister in the Department of Health: “NHS patients need to know their data will be secure and not be sold or used inappropriately, which is why we have introduced tough new measures to ensure patient confidentiality.” [Daily Mail, 4/5/16]

c) Professor Hugh Montgomery, (consultant for Google’s DeepMind project) said, on Radio 4’s PM programme on 4 May 2016:

i) “So this is standard business as usual. In this case, it was a standard information data sharing agreement with another supplier, which meets all of those levels of governance. In fact, the agreement there, or the standards of management of those data, meets the very very highest levels. It meets something called HSCIC level 3, which most hospitals trusts don’t even reach.” [Recording of audio available, see link below]

ii) “So firstly, this isn’t research. Research is governed by an entirely separate process that would require anonymisation of data and all sorts. This is data processing.”

iii) “It’s fair to say again that not only is this data at the very highest standards, and beats every standard, and more in the United Kingdom. But the data is encrypted end-to-end, and they have to, like everyone else in the health service, stick to the law.”

iv) Recording of audio available at: https://www.dropbox.com/s/cfimojgec24rlrj/
20160504­deepmind­radio4­pm.mp3?dl=1

d) Will Cavendish, now Strategy Lead for DeepMind Applied, formerly Informatics Accountable Officer at the Department of Health, said (when IAO):

…“The vital importance of trust, security, and cyber security.” … “To be honest, it used to be that not a week goes by, now it’s not a day goes by, without stories of hacking, data leaks, inadvertent data sharing. This absolutely erodes the trust that underpins the work that we do.” https://www.youtube.com/watch?v=5Ej3PRF1jUw&t=2h15m5s

e) Dr Julian Huppert, Chair and “on behalf of the Panel of Independent Reviewers for Google DeepMind Health” said in an e-mail to medConfidential on 6/7/16:

i) “one of our roles is to look in detail at how DeepMind Health uses patient data, and to confirm that it complies with the highest ethical and regulatory standards.”

ii) “We believe from what we have seen so far that DeepMind has a clear commitment to the Caldicott Principles, and that they have to date been honest in their public and private comments. We also believe they are willing to work constructively with regulators, and remain within the law.

3) https://www.royalfree.nhs.uk/news-media/news/new-app-helping-to-improve-patient-care/