Tag Archives: DeepMindRFH

medConfidential – mid June update

We’ll have more on implementation of the hospital data opt-outs when the dust has settled after the referendum.

“Intelligent Transparency”

According to a letter from a Minister, “Intelligent Transparency” is the new goal. We hope that all Department of Health decisions will prove “intelligent” from a patient view, and not just the political priorities of their desk in Whitehall.

Will transparency extend to telling you how your data has been used?  Or is that the sort of intelligence they don’t want you to have?

Tech startups are no magic bullet

We’re waiting for a response from the Regulators about DeepMind’s project at the Royal Free Hospital Trust. Whatever they say, we note that Google has now made public commitments to move towards the transparency expected of them. Regulators are still investigating, and given the contradictory statements, it may take some time.

We look forward to seeing what they will tell the public about their experiments to replace doctors.

What can you do: The Hospital Episode Statistics consultation

The Hospital Episode Statistics cover data from the nation’s hospitals for over 25 years. The HSCIC is looking for everyone’s views on privacy in the data. We’ll have a long response in a few weeks, but you can quickly complete their survey (or just email enquiries@hscic.gov.uk with a subject of “HES PIA consultation”). You don’t need to answer all the questions – you can just say why it matters to you that your privacy and opt out applies to hospital data. 

Update on Google Deepmind’s NHS app – is it “just resting” ?

It appears Google Deepmind has suspended use/“pilots” of their experimental app until they have received regulatory clarification – ie, until it is legal.

This whole incident was harmful to patient trust, it was harmful to the hospital, and it was harmful to Google. All because, it appears, there was a desire to go faster than waiting a few weeks for regulatory and data approvals, and so used a bizarre cut’n’shut agreement.

The controversy has never been about whether the app would help clinicians with their patients. It has been entirely about what happened for people who were not patients of those clinicians. Some of questions from over a week ago remain unanswered. “Collect it all” might have applied to version 1 of the app, but they now have first hand experience of how it makes things harder not easier.

Tech teams often like naming their work. Perhaps the next version will be “Streams 2: This time we read the regulations”…

Google Deepmind could have followed the rules about applications for direct care, and the usage of data for “development work” (ie, “secondary uses”). They just didn’t, for some reason that we will ask their independent reviewers to get to the bottom of.

The app deserves to come back safely, if the humans running the project can follow the rules to get the data and processes that they wish to bring to the NHS. Nothing in our understanding of what they were doing, and the existing rules, should have prevented from doing this “pilot” (apparently), entirely legally, with conventional legal agreements. They simply didn’t do so.

If Google Deepmind choose to walk away from the project, it won’t be because they wanted to help the NHS; it’ll be because they wanted to help only on their terms. For the hospital, and the NHS more widely, it is yet another reminder that some offers of help may come with too high a price.

MedConfidential comment on Friday’s New Scientist revelations about Google Deepmind


Extraordinarily, the New Scientist has quoted Google as having used as part of an unregulated algorithm in the direct care of patients[1].

This follows up on previous news that Google Deepmind had acquired millions of detailed patient histories for unclear purposes[2]. Google Deepmind’s response was to focus that they were keeping the data safely[3], and to ignore questions over what they were doing with it, and whether they should have had it in the first place[4].

MedConfidential has long argued that every patient should be able to know how data about them has been used. If there had been a Ministerial commitment to do that, this mess of unanswered questions would not have happened.[5]

Announced yesterday, it is Government policy to “encourage and support data-driven techniques in policy and service delivery”. Innovation is welcome and vital, but it should be grounded in medical ethics and a clinical relationship, and not ride roughshod over processes in place to protect all involved.[6]

Responding to the latest information, MedConfidential coordinator Phil Booth said:

“Deepmind has spent a fortnight hiding behind the NHS. It’s now clear that this was a unregulated “development” project for deepmind, but a patient care project for the NHS.

“These algorithms evolve: errors get fixed, improvements get made. What approvals did Deepmind have from the medical regulators at the early stages? As the provider of a tool used in direct care, they are responsible for ensuring it meets all safety standards.

“Training doctors to make safe decisions takes years, and requires many exams to be passed. Have Google shown that each version used in direct care met all relevant grades, standards, and regulations?


For immediate or future interview, please email coordinator@medconfidential.org 

Notes to editors:


  1. See https://www.newscientist.com/article/2088056-exclusive-googles-nhs-deal- does-not-have-regulatory-approval/  “We [Deepmind] and our partners at the Royal Free are in touch with MHRA regarding our development work.”


  1. See https://www.newscientist.com/article/2086454-revealed-google-ai-has-access- to-huge-haul-of-nhs-patient-data/ and http://techcrunch.com/2016/05/04/concerns- raised-over-broad-scope-of-deepmind-nhs-health-data-sharing-deal/


  1. Google’s self-defence https://www.theguardian.com/technology/2016/ may/06/deepmind-best-privacy-infrastructure-handling-nhs-data-says-co-founder refers to their self-reported scores in the IG Toolkit https://www.igt.hscic.gov.uk/AssessmentReportCriteria.aspx?tk=424999242358961&lnv=3&cb=e8c1aaf1-c40d-45af-9bb9-adc46c712924&sViewOrgId=49979&sDesc=8JE14 . Those scores have not yet been audited by the HSCIC.


  1. The question of why Google Deepmind had the histories of people who never had a blood test at the relevant hospital, and who may never return to the hospital, remains unanswered.


  1. Much like a bank statement, every patient should be able to see a data usage report, which tells them where data about them has been used, and why, and what the benefits of that usage were. A commitment to investigate implementation was made in late 2014, but remains delayed by the Caldicott Review of Consent. For more, see https://medconfidential.org/2014/what-is-a-data-usage-report/


  1. MHRA rules require medical devices to have appropriate pre-approved procedures in place to confirm they’re working as expected, and to ensure any conceivable failures have mitigations considered in advance. The New Scientist article confirms they do not have those approvals as algorithms in their software develop.

Google Deepmind – part 1


[this piece covers the state of play as on Sunday 8th May. It may be updated or replaced as new facts emerge]

If you are unwell: seek medical attention. These issues should not prevent you getting the care you need. The below discussion only relates to one Trust, the Royal Free in London, for all patient hospital events since sometime in 2010.

Last summer, following medConfidential’s work on care.data, Dame Fiona Caldicott was asked to review consent in the NHS. That report has still not been published, and provides recommendations. Patients should be able to know every way data has been used, as a condition of using that data – contracts shouldn’t allow secrets from patients.

Following a New Scientist article, there’s been a lot of press discussion about google deepmind receiving over 5 years of detailed medical data from the Royal Free NHS Trust in London. This project is steeped in secrecy, hiding details from patients and the public.

Concerns have not been about the patients whose information would be displayed in this app. Concerns are solely the data of the patients whose data could never be displayed in the app, as they have never had any of the blood tests (etc) it displays. That is 5 in every 6 patients. For the other 1 in 6, there is a potential benefit.

When we were first approached, our initial question was “what are they doing with this?” – details were hidden and emerged only through press investigations.

It looked like what Deepmind were doing should have been a research project – but it had not followed any ethics or research processes. It was using a dataset for the “Secondary Uses Service” – which strongly suggested this was a secondary use.

Data can be used for direct care – the care given to you by a doctor or other clinician. It is also used for other purposes, called “secondary uses”. These include purposes such as research, and the design of models for calling people in for screening (including for detection of kidney problems).

The New Scientist published last Friday, and the question remained unanswered until Wednesday. In an appearance on Radio 4, it emerged that the reason they had followed none of the research processes was simple: it wasn’t research.  It was claimed to be for direct care. The Professor speaking goes on to detail the limits that clinical rules and ethics put on who can access data for direct care.

As a result, on Wednesday afternoon, the question changed to Who is the direct care (ie clinical) relationship between?

Deepmind have made a case that they will look after the data – we’ve no reason to question that different point. This is not about losing data, it’s about whether they should have had most of it in the first place. What data should they have, and how should they have got it?

To answer that question, it has to be clear what they are doing. It is not.

More generally, to have confidence, patients should know how data about them has been used. What is Deepmind hiding in this case? And why? Will they give a full accounting of how they’ve used patient data, and what for, and what happened in direct care as a result?

Every data flow in the NHS should be consensual, safe, and transparent.

Why does google think what it does with the medical history of patients can be secretive, invasive, and possibly harmful?

Throughout most of medConfidential’s work, we are able  to say “opting out will not affect the care you receive”, because large amounts of work have been done by all sides to make sure it does not. If you opt out of “secondary uses” of your data released by HSCIC, it does not affect care compared to someone who did not opt out. Due to the lack of process, and the corners cut by google deepmind avoiding all the relevant processes, that may not necessarily be true. We hope the Trust will clarify what their opt out does. If you didn’t want your data handed to google for speculative purposes, what happens if you get injured and show up at the Royal Free’s A&E? How is your care affected? Did they cut that corner too?

Patients should not be punished for deepmind’s cut corners.

Scalpels Save Lives

Our friends in the research world promote that #datasaveslives, and it does, just like scalpels do.

To be completely clear, deepmind have said that their project is “not research”. That’s why they didn’t follow any research processes. There are 1500 projects which followed the proper processes and appear on the “approved data releases” register – the Deepmind project is not one of them.

Data, and good data hygiene, is as much a requirement of a modern hospital as sterile scalpels. Following the right processes to provide sterile instruments is not seen as an “unnecessary burden”, even if accountants may wish to cut costs due to the expense. Scalpels have to be sterile for a very good reason.

Similarly, processes put in place to protect data are around the same level of importance as adequate cleaning. It may seem like an unnecessary burden to some. Just as too little cleaning will cause problems that clearly demonstrate the necessity of what was previously decried as too much. Those who cut corners are rarely the ones who suffer from the decision. There is a fundamental difference between causation and correlation

Deepmind seem to be a powerful new tool.

Were it was an instrument to be used in surgery, it would not be enough for it to be powerful and new, it must also be safe. Otherwise the harm can be significant.

Rather than clean and safe, if seems deepmind is covered in toxic waste.

It’s not that deepmind couldn’t go through the processes to ensure safety. We don’t know why they didn’t.

Deepmind might be a better instrument, or it might be the new nightmare drug. Technology tools aren’t a panacea. Have lessons been learnt after the “epic failure” of “Google flu trends”?

Research, testing, and regulatory oversight is designed to prove that changes are safe. They also correct any unintended harms to patients as the process proceeds.

How much of that happened in this case? 

If Google DeepMind publish attributable and citable comments in response to these questions, we’ll link to them.