Category Archives: News

More DeepMind secrecy – What the lawyers didn’t look at

The Royal Free has been recommended by ‘independent’ lawyers to terminate its ‘Memorandum of Understanding’ with DeepMind (page 68, second bullet from bottom)

If the “research” agreement with DeepMind – the MoU covering “the use of AI to develop better algorithms” – isn’t terminated, the deliberate exclusions from the legal opinion can only be interpreted as an attempt to mislead the public, once again.

What is the legal basis for continuing to copy 8 years of data on every patient in the hospital? While DeepMind claims the “vital interest” of patients, it still keeps the data of over a million past patients whose interests it will never serve, because RFH’s systems cannot provide “live data” (para 26.1) – despite the report saying that is only temporary (para 15.1).

When RFH completes its move to “fully digital”, will the excessive data be deleted?

The biggest question raised by the Information Commissioner and the National Data Guardian appears to be missing – instead, the report excludes a “historical review of issues arising prior to the date of our appointment” (page 9, para 8.4, 5th bullet, and page 17, para 5,bullet 7).

The report claims the ‘vital interests’ (i.e. remaining alive) of patients is justification to protect against an “event [that] might only occur in the future or not occur at all” (page 43, para 23.2). The only ‘vital interest’ protected here is Google’s, and its desire to hoard medical records it was told were unlawfully collected. The vital interests of a hypothetical patient are not vital interests of an actual data subject (and the GDPR tests are demonstrably unmet).

The ICO and NDG asked the Royal Free to justify the collection of 1.6 million patient records, and this legal opinion explicitly provides no answer to that question (page 75, para 5, final bullet).

The lawyers do say (page 23, para 12.1) “…we do not think the concepts underpinning Streams are particularly ground-breaking.” In Streams, DeepMind has built little more than a user-friendly iPhone app – under scrutiny, its repeated claims of innovation are at best misleading.

But Google DeepMind clearly still thinks it is above the law; it tries to defend all of the data it has by pointing at different justifications each time. Is this the ‘ethical’ ‘accountable’ approach we must accept from the company that wants to build dangerous AIs?

-ends-

Background to the long running saga.

GDPR DAY BRIEFING: ‘Your data matters’ and NHS patients’ data, with the ‘new’ National Data Opt-Out

May 25th 2018 will start an awareness-raising programme with the public that their data matters – and what, if anything, changes as a result of patients’ increased rights under GDPR.

With regard to NHS patients’ health data:

  • A new NHS ‘National Data Opt-Out’ commences on GDPR day (May 25th);
  • NHS Digital continues to sell (i.e. disseminates under contract, on payment of a fee) patients’ data, as it has been doing for years;
  • GDPR expands the definition of ‘identifiable data’ (one reason why everyone’s privacy policies are changing);
  • Will NHS Digital ignore patient opt-outs on these newly-identifiable data releases, relying on definitions from the old Data Protection Act?
  • NHS Digital / DH refuse to ask data applicants why 98% of NHS patients’ data isn’t enough for them; while there may be legitimate reasons to override patient opt-outs, pretending new legislation does not apply to data releases (yet again) is not one of them.

Your Data Matters… but NHS Digital sells it anyway

NHS Digital still forgets about patients. Unfortunately, it sees them less as people and more as ‘lines in a database’.

NHS Digital continues to sell a product called ‘Hospital Episode Statistics’ (HES); a dataset that is not actually statistics but that rather consists of individual patients’ lifelong hospital histories, with every medical event dated and linked together by a unique identifier. As of May 24th, two-thirds of NHS Digital’s data disseminations do not respect patients’ right to object (‘opt out’) to their data being used for purposes beyond their direct care.

If you read NHS Digital’s own Data Release Registers, or view them at TheySoldItAnyway.com, [1] you can see for yourself the evidence of where data goes – and where patients’ express wishes are deemed not to matter.

After four years, and further breaches of health data, NHS Digital ignores the choices of the 1.4 million people who opted out and still sells their (and every other hospital patient’s) data for commercial reuse. Those who claim to need 100% of the data for some reason, need merely explain to a competent data release body why 98% of people’s data isn’t enough – an explanation they’re currently not even asked to provide.

GDPR clarifies that the hospital data NHS Digital continues to sell is identifiable data – so claimed exemptions (item 5) to people’s opt outs don’t apply. Especially for those who remember the dates of particular medical events in hospital, such as the birth dates of their own children, or who can read about them online. [2]

‘Could do better’

Last week, the Department for Education called a halt to the sale of the data it collects on schoolchildren [3] for the very reason the NHS continues using to justify its sale of patients’ data.

NHS Digital now has a research environment [4] which allows far higher safety for patients’ data – but the companies that don’t want the NHS to be able to see what they’re doing with the data are special pleading. It is precisely these hidden uses to which patients are most likely to object.

NHS Digital’s customers, for example, still include for-profit companies such as Harvey Walsh, an “information intermediary” that – exactly as it did in and before 2014, and despite having breached the terms of its contract since then – continues to service commercial clients including pharmaceutical companies, which use the information to promote their products to doctors.

The digital service launching for GDPR Day in fact does less than the form that’s been available on medConfidential’s website since late 2013. [5] Our GP form works immediately – if you use the new digital service, your GP won’t know about it for months.

Discussing a damning ‘report’ in the House of Commons, the chair of the Health Select Committee censured NHS Digital for its “dimmest grasp of the principles of underpinning confidentiality”. [6] The Government has agreed to raise the bar for breaching patients’ confidentiality when handing information to the Home Office; will NHS Digital now respect the choices of those patients who wish to keep the information in their medical records confidential too?

The solution to this is straightforward: DH can Direct NHS Digital to respect objections (opt-outs) in all releases of HES that CAG has not approved to have data released without patients’ objections honoured. There may be projects that require 100% of patient data; two-thirds of them do not.

The ICO has not yet updated its (non-statutory) Anonymisation Code of Practice to match GDPR, although its guidance on the GDPR definition of personal data and newer codes on subject access rights show the definitions in GDPR mean NHS Digital’s current practice does not deliver on its promises to patients.

The NHS has ignored legislative changes and harmed research projects before – see note 4 in this post. This effect is one of the main things that prompted the Wellcome Trust to create the Understanding Patient Data initiative.

But it is still (a bit) better than it was…

NHS Digital now sells less of your data than it used to; it only sends out hundreds of copies of the nation’s hospital records – ‘pseudonymised’, but containing information that GDPR recognises makes it identifiable, and therefore still personal data.

You now have the ability to make a choice for you and (after a fashion) your family that will work, in due course [7] – but NHS Digital needs to listen to Dr Wollaston, “take its responsibilities seriously, understand the ethical underpinnings and stand up for patients”, respect that patients’ data matters, and fully honour everyone’s choices.

Questions for interviewees:

  • What does the NHS’ online-only opt-out service not do on day one, that the GP-led process did last week?
  • How many steps does it take for a family to express their collective choice on how their data is used?
  • When this new digital dissent process was signed off under the Government’s Digital Service Standard, did Ministers spare a thought for their families at all?
  • Will patients’ opt-outs be honoured in the dissemination of HES under GDPR?
    • If not, will those patients who already opted out be told why not?
  • A mother with 2 children is over 99% likely to be identifiable from their children’s birth dates alone; given the enhanced GDPR ‘right of access’ to any recipient data to which opt-outs have not been applied, will NHS Digital keep selling what GDPR defines as the identifiable patient data of those who have opted out?
    • What is the burden on medical research of this choice by NHS Digital, made to placate its commercial customers?

If you or any patient would like to see how their data is used, and where their choices are being ignored, please visit TheySoldItAnyway.com

Notes for Editors

1) NHS Digital publishes its data release register as a spreadsheet but it fails to link to, e.g. its own audit reports – so medConfidential has created a more readable version that does.

2) All the information required to identify Ed Sheeran’s entire hospital history in HES appears in this BBC News article, published online on 19 May 2018: http://www.bbc.co.uk/news/uk-england-suffolk-44155784

3) ‘Sharing of school pupils’ data put on hold’, BBC News, 15 May 2018: http://www.bbc.co.uk/news/technology-44109978

4)  A ‘safe setting’, as medConfidential and others recommended in evidence to Parliament back in 2014: https://digital.nhs.uk/services/research-advisory-group/rag-news-and-case-studies/remote-data-access-environment-to-speed-up-access-to-data

5) We have updated our form to reflect the name change. See https://medconfidential.org/how-to-opt-out/

6)  https://www.theyworkforyou.com/debates/?id=2018-05-09a.746.6#g771.0

7) The National Data Opt-out should be respected by bodies across the health and care system “by 2020”.

medConfidential campaigns for confidentiality and consent in health and social care, seeking to ensure that every flow of data into, across and out of the NHS and care system is consensual, safe and transparent. Founded in January 2013, medConfidential is an independent, non-partisan organisation working with patients and medics, service users and care professionals.

– ends –

Where are the CAG regulations?

We talk a lot about NHS Digital, and its data releases that continue to ignore opt-outs. But 4 years ago today, Royal Assent of the Care Act 2014 gave NHS Digital a “general duty” to “respect and promote the privacy of recipients of health services and of adult social care in England” – which clearly hasn’t been honoured in some areas of its work. The Act also changed the law specifically so that the Confidentiality Advisory Group (CAG) of the Health Research Authority has the power to advise NHS Digital; advice to which NHS Digital must listen.

Caldicott 3 itself does not require dissent to be honoured when data is disseminated in line with the ICO’s Code of Practice on Anonymisation. (The National Data Guardian – who has been given no enforcement powers – is very careful not to ‘cross wires’ with the UK’s data regulator, who does have such powers.) And, despite well over a million patients clearly indicating their wishes to the contrary, NHS Digital continues to argue its dissemination of pseudonymised data is “anonymised” to the satisfaction of the 1998 Data Protection Act.

The UK is about to get a new Data Protection Act, aligned with and based on the EU General Data Protection Regulation, which says:

(26) … Personal data which have undergone pseudonymisation, which could be attributed to a natural person by the use of additional information should be considered to be information on an identifiable natural person.

The UK’s Information Commissioner will update the Code of Practice on Anonymisation in due course –  she’s just a little busy right now, and the new Data Protection Act is not yet on the statute book – but the Irish Commissioner has already said: (emphasis added)

“Although pseudonymisation has many uses, it should be distinguished from anonymisation, as it only provides a limited protection for the identity of data subjects in many cases as it still allows identification using indirect means. Where a pseudonym is used, it is often possible to identify the data subject by analysing the underlying or related data.

Current practice will have to change.

While IGARD may be the appropriate body to advise whether a request meets NHS Digital’s standards for dissemination, it is not an appropriate body to advise on releasing data which does not honour patients’ objections. The adjudication of the principles of those decisions, by statute, belongs to CAG.

There are legitimate instances where patients’ dissent may be be overridden – but IGARD is not, and never should have been, the body to decide that.

The opt-out is there to protect patients who decide that the safeguard the NHS currently relies upon – pieces of paper, which include for example commercial re-use contracts with commercial companies that service other commercial companies, including pharmaceutical companies that use the data for promoting their products (i.e. marketing) to doctors – is not sufficient for their situation. As is their right.

Another example: in 2014, the Health Select Committee asked for a safe setting for researchers. Only in April 2018 did a remote safe setting begin to be piloted for researchers – that work not only needs to be completed, but it should become the standard means of access.

NHS Digital continues to insist that a piece of paper is sufficient safeguard under which to release copies of the entire nation’s lifelong, linked medical histories to hundreds of organisations. Its own published records show that two-thirds of NHS Digital’s data releases do not respect patient dissent.

It should be CAG which makes such decisions, whenever and wherever it is necessary. The CAG Regulations will make that clear, when they exist. Assurances to patients are less than meaningful when the Regulations to which they relate do not yet exist.

If someone applying for patients’ data cannot do what they need with only 98% of people’s data, they should simply explain to a responsible body why this is the case. Public Health England’s cancer registry already takes this approach with the choice of protections if offers for event dates. NHS Digital simply releases data on every patient, with the medical event dates completely unprotected.

The National Data Guardian was asked to determine a single choice by which patients could express their dissent from their data being used for purposes beyond their direct care. When that choice is disregarded, it must be on a basis clearly and specifically defined in statute, and approved by CAG.

As it is doing around the world, the introduction of the GDPR will force a change, and that change should protect patients’ data that under the new Data Protection Act will be considered identifiable. Those who still need everyone’s data will have to explain why to a competent body – which really isn’t too much to ask.

Given the clear promises given as a consequence of the care.data and HES data scandals – promises much repeated, but yet to be delivered – we’ve been waiting a long time for this to be fixed.

Instant Messaging in Clinical Settings

November 2018 Update: The below is consistent with the new NHS Digital guidance, but paragraphs in blue below are omissions from that official guidance.


As a clinician or nurse, you should not have to keep up with the latest fluff of the apps you might use for work. 

NHS England has put out another attempt at ‘guidance’ on using instant messaging apps. Last year it said WhatsApp was not banned, but failed to provide helpful guidance on what to actually use. It still hasn’t. There is a Do & Don’t list, which is better than nothing, but almost impossible to turn into practice in the real world.  If asked, we would suggest something like this:

Summary

  1. If your employer offers an instant messaging solution, use that.
  2. If you are picking apps to use yourself, you are safest with Signal.
  3. If you are not picking the apps you use, you will probably have to use WhatsApp or Skype. But be aware that someone will be held responsible when Facebook or Skype change their rules – and it’s probably not going to be the person who picked the app…
  4. Don’t use Facebook Messenger, Instagram, or Telegram.

Whatever app you use for work, the vast majority of people should avoid having their phone going ding for work purposes while they are not at work. For most apps, a swipe left on the main list of ‘chats’ should show an option to “hide alerts” for some time period – this should ensure that if you do give your personal number to work colleagues, it doesn’t end up driving you to distraction outside work. If someone really wants to get in touch, they can always just call you normally.

 

The reasoning behind our suggestions

The important step in secure messaging is something called “end-to-end” encryption, which prevents anyone – a third party ‘listening in’, or even the service making the connection –  knowing what you said. It’s the equivalent of having a conversation in a private consultation room, rather than doing it standing next to the nurses station, or in a waiting room. But even with Signal, if you are messaging using your personal device, you should treat any conversation as if it were in a lift where another person might be listening.

Signal allows you to decide for how long you will keep messages from any particular person or group, and will automatically delete the stored messages after that. But what happens with the stored message history in other apps? WhatsApp, for example, wants you to give it a full copy of all your messages and send them to its servers as a ‘backup’ (though at some point it will show you ads against them – it is part of Facebook after all).

You may also have set your phone itself to backup to somewhere. Do you know where the backup goes, and what’s in it?

Of course, it is best practice to backup everything on your phone, and most apps assume (probably correctly) that you don’t want to lose every message or photo you receive of your kids. This doesn’t necessarily translate neatly to a clinical setting – anything that must be kept should be recorded elsewhere, so that if you lose your phone, the only thing you won’t have kept was ward chit-chat. WhatsApp wants everything – it doesn’t offer clinical reassurance. And while Snapchat has deletion as a feature, it has other problems akin to Facebook and Skype.

The longer-term security of your messaging is dependent upon who makes the app – and when, and why, they will change the rules on you. We (also) recommend Signal because it is produced by a charitable foundation whose sole mission is to provide secure, usable, communications. One key reason why the NHS England guidance is so terrible is that WhatsApp has lobbyists telling NHS England that it should allow their product; Signal doesn’t.

Since Facebook (the owner of WhatsApp) lies to regulators about its intentions, you clearly cannot rely on the company not to do tomorrow what it denies it will do today.  As a consequence of this, any official guidance must in future be kept up to date by NHS Digital. And, as corporate policies change, so must the guidance – removing from the equation NHS England’s fear of the deluge of lobbying that created this mess in the first place.

Clinicians deserve better tools than those that NHS England chooses to recommend, where a national body prioritises its own interests over the needs of those delivering direct care.

(This post will be kept under review as technologies change; it was last updated in November 2018)

Data and AI in the Rest of Government: the Rule of Law

medConfidential spoke about the Framework for Data Processing by Government at the All Party Parliamentary Group on the Rule of Law. The topic of the APPG provides a useful perspective for much work on data in the public sector, and the wider use of AI by anyone. The meeting was on the same day as the launch of the AI Select Committee Report, which addresses similar key issues of  ‘data ethics’.

The ‘Rule of Law’ is defined in 8 principles as identified by Lord Bingham. The principles are not themselves law, but rather describe the process that must be followed for the Rule of Law to be respected.

Public bodies must already follow that process, and also be able to show how that process has been followed. As a result, those developing AIs (and data processing tools) for use by public bodies must also show how these processes have been followed. This is necessary to satisfy the lawful obligations of the bodies to which they are trying to sell services.

The principles identified by Lord Bingham are a model for testing whether an explanation of an AI and its output, or a data model, is sufficient for use by a public body.

While debates on ethics and society, and on politics and policy, focus on whether a technology should be used – the Rule of Law is about the evidence for and integrity of that debate. As Departments implement the Framework for data processing, to deliver on their obligations under the Rule of Law, it must be compliant with the Principles identified by Lord Bingham – not just the ethics and policies of the Minister in charge that day.

Public bodies are already bound by these rules – unless Parliament legislates to escape them. The principles are widely understood, they are testable, and they are implementable in a meaningful way by all necessary parties, with significant expertise available to aid understanding.

 

Companies and other non-public bodies

Companies (i.e. non-public bodies) are not subject to the same legal framework as public bodies. A Public Body must be able to cite in law the powers it uses; a Private Body may do (almost) anything that is not prohibited by law. This is why facebook’s terms and conditions are so vague and let it get away with almost anything – such a data model does not apply to the tax office.

Some of those looking to make money – to “move fast and break things” – would like the standard to be ethics, and ethics alone. There are currently many groups and centres having money poured into them, with names involving ‘data and society’, ‘ethics and society’, and DCMS’s own ‘Centre for Data Ethics’. The latter is led by a Minister in a Government that will always have political priorities, and – given recent revelations about Facebook – the consequences of incentives to lower standards should be very clear.

Ethics may contribute to whether something should be done – but they are not binding on how it is done, and they offer no actual accountability. After all, no tyrant ever failed to justify their actions; it is the rule of law that ultimately holds them accountable, and leads to justice for those harmed. Ethics alone do not suffice, as facebook and others have recently shown.

There is a great deal more work to do in this area. But unlike other AI ‘ethics’ standards which seek to create something so weak no-one opposes it, the existing standards and conventions of the Rule of Law are well known and well understood, and provide real and meaningful scrutiny of decisions – assuming an entity believes in the Rule of Law.

The question to companies and public bodies alike is therefore simple: Do you believe in the Rule of Law?

[notes from APPG talk]
[medConfidential (updated) portion of the APPG briefing]

Response to the House of Lords AI Select Committee Report

The AI Select Committee of the House of Lords published their report this morning.

In respect of the NHS, it suggests nothing the NHS wasn’t already doing anyway.

The suggestion that ‘data trusts’ be created for public sector datasets – such as tax data – will likely cause fundamental distrust in AI amongst the public (paragraphs 82 & 84). The NHS has shown how that model ends badly when the prime drivers are commercial, not ‘human flourishing’.

Sam Smith, a coordinator at medConfidential said (referring to paragraphs 99, 129, 317-318, 386, 419-420) :

“A week after Facebook were criticised by the US Congress, the only reference to the Rule of Law in this report is about exempting companies from liability for breaking it.

“Public bodies are required to follow the rule of law, and any tools sold to them must meet those legal obligations. This standard for the public sector will drive the creation of tools which can be reused by all.

 

-ends-

medConfidential are speaking at the APPG Rule of Law in Parliament from 11 – 12:30, and more details are now available.

NHS Digital failing to uphold patient interest


The Health Select Committee has published a report on data sharing which “raises serious concerns about NHS Digital’s ability to protect patient data” under the headline “NHS Digital failing to uphold patient interest”.  The Home Office is “treating GP patient data like the Yellow Pages” according to the RCGP.

The NHS has been trying to rebuild trustworthiness around data since the last big NHS data project collapsed in 2014. This report shows that all promises can be undermined by the narrow minded view of one office in Whitehall

The Health Select Committee is clear that NHS Digital has again failed in its statutory duties, and has put patients at risk by the processes it has adopted and refuses to change.

HSCIC rebranded into NHS Digital in an attempt to avoid the history of past failures, but this report shows actions are unchanged…

We submitted written evidence to the inquiry.

medConfidential Bulletin, 9th March 2018

It has been a while since we last sent a newsletter. Our apologies for that – we have been kept busy on a number of fronts, but rather than spam you with speculations we believe it’s better to communicate when there are significant developments.

 

New national opt-out for medical records

An announcement has been delayed for some months and there’s still some time until action is taken, but to quote NHS Digital last week:

The Secretary of State has agreed that the national data opt-out will be introduced alongside the new data protection legislation on 25 May 2018. It has also been agreed to present the national data opt-out as a single question to cover both research and planning. Type 2 opt-outs (which currently prevent identifiable data from leaving NHS Digital) will be converted to the new national data opt-out when it is introduced in May. Patients with type 2 opt-out will be contacted directly about this change.

There are still a number of important questions to be answered, but we’re working on those for you. For example, at this point, the Government has not yet confirmed that every data release that would be covered by the Type 2 opt-out will be covered by the new opt-out.

medConfidential has yet to see the final wording of the question, but this announcement is clear confirmation that if you opted out in 2014 (or subsequently), you will be sent a letter about what happened. We also haven’t yet seen the wording of the letter, as we and the other members of CDAG (the care.data Advisory Group) would previously have done, but apparently we are to be consulted on that too. When we have the ability to cite formal statements on the new process, we will update our website – this is likely to be in May.

So, if you have already opted out, the NHS will write to you about the new opt-out model. Whether anyone will tell other people remains unclear. We do hope the Secretary of State won’t snatch defeat from the jaws of a victory which could improve patient confidentiality and everyone’s confidence in how the NHS uses data.

 

This week: Data Protection Bill

The Data Protection Bill was delayed by political squabbling, but must pass by early May, and is now on a very tight timescale.

medConfidential’s concerns with the Bill relate to something called the “Framework for Data Processing by Government” which, in effect, creates a ‘Data Controller in Chief’ who can ignore the Information Commissioner, and the fact that the Government wishes to deny your ability to access information on how your records are used, if that might be used by someone else at another time in a way which may “prejudice… effective immigration control”.

Thanks to a great deal of work by many concerned groups and organisations, the Government no longer considers this framework above the law, just above enforcement of the law. The Rule of Law requires that justice both be done, and be seen to be done – requiring transparency that Governments and companies often prefer to avoid.

 

What you can do

Many parts of England have local elections in May. The ongoing stealth reorganisation of the NHS in England (into 44 “Sustainability and Transformation Partnerships” and “Integrated Care Systems”) will give your local council more responsibility for data re-use in your area. No details will be given until after the elections – of course! – but if anything does emerge before that, we’ll let you know.

The health and care issues that most burden the NHS differ from place to place, sometimes quite widely. So when local politicians ask for your vote in the next few weeks, you might ask them what their council would do about the biggest issues in your area.

You can see the top three issues most impacting health in your local authority, and those nearby, on this map: http://bit.ly/2FVYVE1

(Created thanks to current data from Public Health England, and with the help of tools provided by Democracy Club whose volunteers collate and share information on elections across the UK.)

 

What’s next?

medConfidential keeps working even when we’re not sending newsletters; we won’t spam you if there’s nothing important to say. As you can see from this Bulletin, we are approaching another critical time for patient confidentiality that we hope can be negotiated with far greater success than in 2014! If you appreciate our ongoing efforts, we accept donations. Thank you for your support.

 

Phil Booth & Sam Smith
9th March 2018

AI and demonstrations of political power

Last September, a company which helps institutions understand data started a new project. What their client wanted, was to tell whether one category of videos could be distinguished from some others. The project was successful on a test dataset, and they produced a demo. The very happy client forwarded this to their boss, who sent it to their boss, and so on, until then the Home Secretary went on TV to say that the Home Office had better technology than Google for blocking ISIS videos. At no point, was there a need to test or explain whether the demo worked beyond the test data. That seems to be the standard for AI – is data processing a place where the rule of law doesn’t matter?

The Department of Health also launched guidance on “decision making” AIs for use in the NHS. Innovations have always come to healthcare and spread based on understanding – there is no need to throw all past lessons out because a PR budget gets spent. Separately, the “Malicious AI report” is worth reading – but already feels dated as the risks are both real and timely, and political imperatives are rarely spun as malicious.

Given the option for a quick headline and a cheap political point, politicians will choose to score it. With digital systems of any kind, there is a temptation to take a shortcut and claim victory.  The Home Office claimed to have an AI which did what it wanted – by ignoring any real world requirements that made things harder. This is not the greatest of precedents for public bodies using AI tools to make decisions, especially on groups who do not command easy political support.

Explainability” is just putting in the extra time to test models and understand how they work – rather than selling the first thing that seems to meet the goal. That faster approach may have short term financial benefits, but it can be more widely toxic as an outcome generally best avoided. The Home Office can make this claim for this AI, as it as the first Department to do so; the next claim will be treated with the greater scepticism that it deserves. We’ve put the Home Office statements into the Government’s ethics framework – which again shows the failures of that framework.

‘Trustworthy technology’ needs to address systemic harms. The first mover advantage on AI will go to those with the least moral concerns about real world effectiveness, until there is a clear reputational harm for continuing to work on systems which are known to be damaging. This is why the ‘most successful’ public sector AI project in the UK is from the Home Office – harms to others are something they have never bothered to avoid.

What started out as a legitimate technology project – can AI help identify ISIS videos? –  demonstrating potential, was spun as something else. Had explainability been a prerequisite of that project as being considered a success as it was claimed, (rather than simply a stage of a trial). Where an entity refuses to follow such processes, as in the drug discovery arena, reputable actors should simply refuse to deal with them as that should be one of the requirements of being seen as a reputable actor.  The Partnership on AI was supposed to consider how to address such issues – but companies outside the partnership aren’t bound by their rules… But many of the staff of those outside would not wish to be barred from working there due to other associations (there, of course, must be a way to demonstrate lessons have been learnt)…

The AI guidance from the NHS contains a checklist, written by “industry” and the Wellcome Trust, which is so vague it barely addresses previous problems, let alone handling future questions. There are no considerations of the principles of ‘trustworthy technology’ by developers, nor any references to equivalent protocols for decision making AIs as we have for determining doctors are trained or new medicines are safe. Claiming you have a phase 0 success is one thing (whether a drug or AI), claiming you have a phase 3 success is quite another – and so it should be with machine learning tools that need to be explained.

Many of the greatest failures of the Home Office are due to technical ineptitude. While their policy can not correctly distinguish an arse from an elbow, technology has moved on sufficiently to do it for them, letting Marsham Street ignore the details while delivering the opposite of human flourishing.

Does HMG wish to be permanently excluded from buying from working with partnership members because it chased a cheap headline?  Does the partnership have the willingness to ensure members deliver “responsible” AI? It is the public headlines and narrative that matters, and the biggest headline about AI in Government is that of the Home Office wanting to choose content purely based on a single suprious claim. Government acts as a single customer to their suppliers; and the reverse must be true for AI & ethics.

 

Data Protection and Datasets of National Significance

Second reading of the Data Protection Bill is in a week – and Government has still not explained the effects of their proposals to centralise data policy in the periphery of Whitehall. As DCMS struggle with a politically led centre for AI data and ethics, announcements like the one from the Home Office will grow. Not because they have solved any problems, but they have done something which redefines the problem as sufficient for the box to get ticked, political claims to be made, and someone else to pick up the pieces. The Home Office does not care about DCMS politics or policy, but which way will Google DeepMind be lobbying DCMS on this?

Lord Mitchell amended the Data Protection Bill to require public bodies estimate the worth of their “datasets of national significance”. Lord Darzi is thinking along similar lines about a new deal with patients. While both good and worthy initiatives that are deserving of time, there is a risk other policies will make them irrelevant.

Lord Mitchell’s amendment mandates an assessment that should be written down, but under current rules, what NHS England or any public body will be forced to write by HM Treasury is that giving data to private companies that employ UK staff, will create new tax revenues from those staff (since company profits go offshore). One NHS trust working with Google might create a nice deal for themselves from some data – but the rest of the NHS will still have to pay a much higher rate.

What will happen when the public understand that this is how their data gets used, and where the money goes?

Even if the Government take the clause out of the Data Protection Bill, whether UK data should be flowing to tax havens is likely an increasingly important question for public debate.  This question is not going away – NHS Digital already do some checks that they’re not dealing with an empty shell company (PHE’s only meaningful step is to check that the fees are in their bank account). Does Government wish to ignore an issue that will resonate with the public on data, or leave in place the small and sensible steps Lord Mitchell added to the Bill?

Enclosed:

 

The Data Protection Bill reaches the Commons

Updated: 16 April: the Bill has been renumbered again. All clauses 185-188 are now numbered 183-186. No other meaningful changes…

Updated 11 March: short briefing for commons committee stage

The Data Protection Bill has reached the Commons. We have 3 briefings on the Bill and an annex on the proposal to make DCMS the lead department for data processing by Government:

(We were expecting 2nd reading this Tuesday/Wednesday, but it’s possible the Whitehall bickering over the DCMS data grab has delayed it; if DCMS has put the politics of empire building ahead of the legislative schedule is a really good indicator that they shouldn’t take over the GDS data function…). Those two links (which were published after the briefing was first circulated), confirm that what is the Cabinet Office’s ‘data science ethics framework’ may get rewritten by DCMS to become the ‘Framework for Data Processing by Government’. For that task, even the iteration that has been discussed is entirely unfit for purpose.

GDPR and Transparency in Government

The EU’s Article29 Working Party held a consultation on their transparency guidance, and with an efficiency that probably infuriates Boris Johnson, ignores late submissions.

For the UK’s NHS, the GDPR is generally just a restatement of the existing ethical good practice that medical bodies should have been following anyway – but it does provide an opportunity (and necessity) to review past decisions and bring them up to scratch (and blame the EU for having to do it).

The main new provision for the NHS, and the topic of A29WP’s recent transparency consultation, are the provisions about what transparency and provision of information to the data subject means. Even that isn’t that new – but it is something that Government has paid lip service to for some time (remember the care.data junk mail leaflets?). That leaves a simple question:

What should transparency look like in practice?

For the NHS, there must be an electronic report on how data was used. NHS Digital keeps track, and with a digital login to the NHS (via patient online), the patient can see where data went, why, and what the benefits of those projects turned out to be, and if they wish to read the published papers (and simpler explanations) that resulted from those uses.

The rest of UK Government lags behind the NHS and is far more murky. Clearly stated in the “Better Use of Data” section of the Technology Code of Practice is a requirement that “the service should clearly communicate how data will be used”, which is akin to the GDPR. Unusually for a GDS recommendation, there is no exemplar given – here is ours.

The best way for an ongoing transactional service to communicate how data will be used next month, is to show how it was used last month.  For any data derived from a digital service behind a login (e.g. any eIDAS compliant system, such as Gov.UK Verify), on any subsequent login, a full accounting of how data on that data subject was accessed, copied, analysed or disseminated, should be available to that data subject.

The best way to know how your data will be used next month is to see how it was used last month. Processes will change over time, but not that rapidly.

This information must also be accurate. It is unclear what the consequence of providing misleading information currently is, but there should be some in a post-GDPR world. Mistakes are a prime facie breach of fair processing, and potentially cause serious distress which is a clear breach of current law.

Taking an example of where information could and should be provided, let’s look at Universal Credit: How much burden is placed on the entire system by the fact that how data is used inside UC & DWP is clouded in secrecy and consequent distrust?

The transparency obligations from GDPR do not extend to investigation of fraud or crimes, so it is not universal, but there are many other consequences of the current system which can be mitigated by informing citizens. UC is already a fully digital service, where users login repeatedly, and access and reuse of data by DWP is already (mostly) logged.

UC used to have such a screen visible to claimants – but the DWP civil servants insisted it be turned off as the Minister might like it. Of course the Minister would like it, as it would be an evidence base of facts and accurate information for a citizen on what the Department actually did – the thing for which the Minister gets held publicly accountable.  With an audit trail, visible to those involved, there will be fewer scandals that land on the Secretary of State’s desk when the stated policy was one thing but the actions of the Department were contradictory.

It is only where ministers deliberately mislead the House that GDPR accountability is a negative…

Access to Individual level Data

As part of transparency, it must be clear how promises to citizens are met. While the NHS does audits on recipients of data, companies regularly fail them with negligible consequences

Population scale citizen level datasets include an administrative census such as the cancer registry (everyone with any cancer for the last ~30 years), HES (everyone who has been treated in hospital since the early ’90s), or the National Pupil Database (everyone who has attended a state school since the mid-90s), or other large scale sensitive datasets (the rest of the NHS data estate).

When population scale data (that does not respect dissent) is copied out of the infrastructure of the data controller, it is impossible to ensure that promises to patients are kept. There are no technical measures which can provide assurance that what should have happened, actually did. That assurance is what the ‘limited environment’ of an independently run safe setting provide.

It is already standard process to allow access to detailed (decennial population) Census data in a safe setting where queries can be audited. The transparency and information provisions of GDPR should be read as requiring that where queries on a dataset can not be audited, that state must be available to a data subject since it makes much more likely that the promises of a data controller may be broken – because the controller has no means to know they are kept.

The 2017 Annual Report from the National Data Guardian again calls for “no surprises”. As the GDPR brings more data controllers closer to the standards already required in the NHS, the best way to inform a data subject how their data is likely to be used next month, is to show how it was used last month. From accountability, can come trustworthiness.

As the Whitehall machine grinds on, as the opt out moves to DH from NHS England, and as data moves from CO to DCMS, the forgetting happens: institutions forget what happened, and institutional memory is what they wished happened. Care.data was just a communications failure, and not a policy failure; etc. Where they forget, we will have to remind them.