ICO Guidance on AI Published – AI and Data Protection

On 30 July 2020, the Information Commissioner’s Office (ICO) published its long-awaited guidance on artificial intelligence (AI) and data protection (ICO guidance on AI), which forms part of its AI auditing framework. However, recognising that AI is still in its early stages and is developing rapidly, the ICO describes the guidance as foundational guidance. The ICO acknowledges that it will need to continue to offer new tools to promote privacy by design in AI and to continue to update the guidance to ensure that it remains relevant.

The need for ICO guidance on AI

Whether it is helping to tackle the coronavirus disease (COVID-19), or managing loan applications, the potential benefits of AI are clear. However, it has long been recognised that it can be difficult to balance the tensions that exist between some of the key characteristics of AI and data protection compliance, particularly under the General Data Protection Regulation (679/2016/EU) (GDPR).

The Information Commissioner Elizabeth Denham’s foreword to the ICO guidance on AI confirms that the underlying data protection questions for even the most complex AI project are much the same as with any new project: is data being used fairly, lawfully and transparently? Do people understand how their data is being used and are they being kept secure?

That said, there is a recognition that AI presents particular challenges when answering these questions and that some aspects of the law require greater thought. Compliance with the data protection principles around data minimisation, for example, can seem particularly challenging given that many AI systems allow machine learning to decide what information is necessary to extract from large data sets.

Scope of the ICO guidance on AI

The guidance forms part of the ICO’s wider AI auditing framework, which also includes auditing tools and procedures for the ICO to use in its audits and investigations and a soon-to-be-released toolkit that is designed to provide further practical support for organisations auditing their own AI use.

It contains recommendations on good practice for organisational and technical measures to mitigate AI risks, whether an organisation is designing its own AI system or procuring one from a third party. It is aimed at those within an organisation who have a compliance focus, such as data protection officers, the legal department, risk managers and senior management, as well as technology specialists, developers and IT risk managers. The ICO’s own auditors will also use it to inform their statutory audit functions.

It is not, however, a statutory code and there is no penalty for failing to adopt the good practice recommendations if an alternative route can be found to comply with the law. It also does not provide ethical or design principles; rather, it corresponds to the data protection principles set out in the GDPR.

Structure of the guidance

The ICO guidance on AI is set out in four parts:

Part 1. This focuses on the AI-specific implications of accountability; namely, responsibility for complying with data protection laws and demonstrating that compliance. The guidance confirms that senior management cannot simply delegate issues to data scientists or engineers, and are responsible for understanding and addressing AI risks. It considers data protection impact assessments (which will be required in the majority of AI use cases involving personal data), setting a meaningful risk appetite, the controller and processor responsibilities, and striking the required balance between the right to data protection and other fundamental rights.

Part 2. This covers lawfulness, fairness and transparency in AI systems, although transparency is addressed in more detail in the ICO’s recent guidance on explaining decisions made with AI (2020 guidance). This section looks at selecting a lawful basis for the different types of processing (for example, consent or performance of a contract), automated decision making, statistical accuracy and how to mitigate potential discrimination to ensure fair processing.

Part 3. This section covers security and data minimisation, and examines the new risks and challenges raised by AI in these areas. For example, AI can increase the potential for loss or misuse of large amounts of personal data that are often required to train AI systems or can introduce software vulnerabilities through new AI-related code. The key message is that organisations should review their risk management practices to ensure that personal data are secure in an AI context.

Part 4. This covers compliance with individual rights, including how individual rights apply to different stages of the AI lifecycle. It also looks at rights relating to solely automated decisions and how to ensure meaningful input or, in the case of solely automated decisions, meaningful review, by humans.

ICO guidance on AI – headline takeaway

According to the Information Commissioner, the headline takeaway from the ICO guidance on AI is that data protection must be considered at an early stage. Mitigation of risk must come at the AI design stage as retrofitting compliance rarely leads to comfortable compliance or practical products.

The guidance also acknowledges that, while it is designed to be integrated into an organisation’s existing risk management processes, AI adoption may require organisations to reassess their governance and risk management practices.

A landscape of guidance

AI is one of the ICO’s top three strategic priorities, and it has been working hard over the last few years to both increase its knowledge and auditing capabilities in this area, as well as to produce practical guidance for organisations.

To develop the guidance, the ICO enlisted technical expertise in the form of Doctor (now Professor) Reuben Binns, who joined the ICO as part of a fellowship scheme. It produced a series of informal consultation blogs in 2019 that were focused on eight AI-specific risk areas. This was followed by a formal consultation draft published in February 2020, the structure of which the guidance largely follows. Despite all this preparatory work, the guidance is still described as foundational.

From a user perspective, practical guidance is good news and the guidance is clear and easy to follow. Multiple layers of guidance can, however, become more difficult to manage. The ICO has already stated that the guidance has been developed to complement its existing resources, including its original Big Data, AI and Machine Learning report (last updated in 2017), and its more recent 2020 guidance.

In addition, there are publications and guidelines from bodies such as the Centre for Data Ethics and the European Commission, and sector-specific regulators such as the Financial Conduct Authority are also working on AI projects. As a result, organisations will need to start considering how to consolidate the different guidance, checklists and principles into their compliance processes.

Opportunities and risks

“The innovation, opportunities and potential value to society of AI will not need emphasising to anyone reading this guidance. Nor is there a need to underline the range of risks involved in the use of technologies that shift processing of personal data to complex computer systems with often opaque approaches and algorithms.” (Opening statement of ICO guidance on AI and data protection.)

If you have any questions on data protection law or on any of the issues raised in the ICO guidance on AI please get in touch with one of our data protection lawyers.