On April 21, 2021, the European Commission proposed Europe’s first comprehensive legal framework to regulate Artificial Intelligence and promote trustworthy AI.[1] The framework issued by the executive arm of the European Union aims to regulate AI technologies including autonomous vehicles, facial recognition, biometric recognition, and others, according to a four-tier scale based on the potential risk of harm to user safety and fundamental rights.[2] The proposed additional requirements in the regulations range from increased transparency requirements for limited-risk applications, to new requirements regarding data quality, documentation and oversight for high-risk systems, to an outright ban on those considered unacceptably risky.
I. The New Legislative Proposal
The regulation proposed by the European Commission classifies Artificial Intelligence systems under four levels of risk: unacceptable, high, limited and minimal. The framework applies to public and private actors inside and outside the European Union when the AI system is placed on the EU market or its use affects people located in the EU.[3] The regulation would not apply to private and non-professional uses of AI technologies.
1) Unacceptable risk: Systems that represent “a clear threat to the security, livelihoods and rights of individuals” are considered an unacceptable risk. This category includes “AI systems or applications that manipulate human behavior to circumvent users’ free will (for example, toys that use voice assistance to encourage unsafe behavior by minors) and systems that allow governments to give a social score.”[4] These systems are banned in their entirety.
2) High risk: AI systems identified as high risk include those used in:
- Critical infrastructures (e.g. transport), that could put the life and health of citizens at risk;
- Educational or vocational training, that may determine the access to education and professional course of someone's life (e.g. scoring of exams);
- Safety components of products (e.g. AI application in robot-assisted surgery);
- Employment, workers management and access to self-employment (e.g. CV-sorting software for recruitment procedures);
- Essential private and public services (e.g. credit scoring denying citizens opportunity to obtain a loan);
- Law enforcement that may interfere with people's fundamental rights (e.g. evaluation of the reliability of evidence);
- Migration, asylum and border control management (e.g. verification of authenticity of travel documents);
- Administration of justice and democratic processes (e.g. applying the law to a concrete set of facts)
- Remote biometric identification (e.g. identifying people in a crowd).[5]
These categories are expected to be adjusted as AI technologies develop. For services classified as high risk, the proposed regulation provides mandatory requirements for approval and market access. Those obligations include:
- High quality datasets;
- Adequate risk assessment and mitigation systems;
- Traceability, including logging of activity;
- Human oversight;
- And transparency.
Providers of these high-risk systems will be expected to demonstrate conformance with these obligations before their product will be approved for the market. Any substantial modification of the AI system will require a new assessment.
3) Limited risk: AI with limited risk, on the other hand, will only provide for an obligation of transparency, indicating explicitly to the user the use of Artificial Intelligence systems. This category includes systems such as chatbots, applications that use Artificial Intelligence to recognize the user’s emotions or use biometric data for their categorization, or systems that offer content created using technologies such as deep fake that present images or sounds created artificially but appearing as real.[6]
4) Minimal risk: Finally, minimal risk systems include “applications such as video games or AI-based spam filters.”[7] For this category, the proposed framework imposes no additional regulations, as theses uses are not expected to compromise the rights or safety of citizens. According to the Commission, the vast majority of AI systems fall into this category.
II. Potential Implications of the Legislative Proposal
The Commission’s approach sets out strict requirements, especially with regards to biometric identification through AI. The European Commission has not decided to ban higher-risk uses of AI, but has classified those systems as such and prohibited their indiscriminate use at least in public areas. However, there are “strictly defined and regulated” exceptions that fall within the sphere of public security and for which authorization by a judicial body is required, with “time limits, geographical scope and researched databases.”[8] Examples include the search for missing minors, the prevention of imminent terrorist attacks, the location of perpetrators of serious crimes.
Offenders, according to the Commission’s text, could incur administrative fines of up to 30 million euros or, in the case of companies, fines of up to 6% of their total turnover.
III. Regulatory Adoption
The European Parliament and the Member States will have to adopt the Commission’s proposals under ordinary legislative procedures. Once adopted, the regulations will be directly applicable across the EU. The regulation expressly introduces the possibility of creating “regulatory sandboxes” to develop AI services under the guidance of the competent authorities, in order to more easily implement solutions that are “legal by design.”
Member States will be asked to designate one or more national authorities to supervise the application and implementation of this regulatory framework, as well as carry out market surveillance activities.
IV. Conclusions
Europe is planning far-reaching and forward-thinking regulations to guide the development of Artificial Intelligence and protect people from the unintended consequences of AI. By the first quarter of 2022, the EU is also planning to release rules to address liability issues related to new technologies, including AI systems.[9] The EU is creating an ambitious reference framework so that these technologies can contribute to achieving ambitious sustainability goals, while respecting the fundamental rights of individuals.
***
If you have any questions about the issues addressed in this memorandum, or if you would like a copy of any of the materials mentioned in it, please do not hesitate to reach out to:
Sam Stake
Email: samstake@quinnemanuel.com
Phone: +1 (415) 875-6387
Jerome Kommer
Email: jeromekommer@quinnemanuel.com
Phone: +49 89 20608 3002
James Judah
Email: jamesjudah@quinnemanuel.com
Phone: +1 (415) 875-6420
Federica Combariati
Email: federicacombariati@quinnemanuel.com
Phone: +49 89 20608 3008
ENDNOTES:
- Proposal for a Regulation on a European approach for Artificial Intelligence: https://digital-strategy.ec.europa.eu/en/library/proposal-regulation-european-approach-artificial-intelligence
- New rules for Artificial Intelligence – Questions and Answers: https://ec.europa.eu/commission/presscorner/detail/en/QANDA_21_1683#1
- New rules for Artificial Intelligence – Questions and Answers: https://ec.europa.eu/commission/presscorner/detail/en/QANDA_21_1683#1
- Article 5 of the proposed regulation.
- Article 6 of the proposed regulation.
- Article 7 of the proposed regulation.
- Article 8 of the proposed regulation.
- Article 5 of the proposed regulation.
- https://digital-strategy.ec.europa.eu/en/policies/european-approach-artificial-intelligence