+AI & Autonomous Technologies High Risk Designations
|
AI & Autonomous Technologies High Risk Designations
Description
Mechanisms exist to designate Artificial Intelligence (AI) and Autonomous Technologies (AAT) "High Risk" if one(1), or more, of the following criteria are met:
(1) AAT is used as a safety component of a product or service;
(2) AAT poses a significant risk of harm to an individual's health, safety or fundamental rights; and/or
(3) AAT materially influences the outcome of an individual's decision making.
Possible Solutions & Considerations
Micro-Small Business (<10 staff) / BLS Firm Size Classes 1-2
∙ Document criteria for high-risk AI use and review before deployment
Small Business (10-49 staff) / BLS Firm Size Classes 3-4
∙ AI risk classification checklist with high-risk criteria
Medium Business (50-249 staff) / BLS Firm Size Classes 5-6
∙ Formal AI risk classification policy
∙ High-risk designation process
Large Business (250-999 staff) / BLS Firm Size Classes 7-8
∙ AI risk management framework with formal high-risk designation process
∙ Risk committee review
Enterprise (> 1,000 staff) / BLS Firm Size Class 9
∙ Enterprise AI risk classification framework
∙ Automated risk scoring
∙ Legal/compliance review for high-risk AI
∙ Regulatory compliance mapping (e.g., EU AI Act)
SCR-CMM
Level 0 Not Performed
Practices are non-existent, based on the inability to demonstrate an implemented and operational capability. A reasonable person would conclude the control is not being performed.
Level 1 Performed Informally
SCR-CMM Level 1 criteria definitions are not available for this control:
▪ A reasonable person would conclude this control requires a structured process.
▪ At this level of maturity, the "ad hoc" nature of performing a capability informally would indicate the intent of the control is not met due to a lack of consistency and formality.
Level 2 Planned Tracked
Artificial Intelligence and Autonomous Technology (AAT) capabilities are requirements-driven, but are not standardized across the entity (e.g., local/regional level consistency). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are documented and maintained by process owners.
▪ IT and/or cybersecurity personnel work with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address applicable statutory, regulatory and/or contractual requirements for Technology Assets, Applications, Services and/or Data (TAASD).
▪ Artificial Intelligence (AI)-related controls are primarily administrative and preventative in nature (e.g., policies, standards, procedures & guidelines).
▪ Asset management may be a defined function (e.g., team or department) or assigned as an additional duty to existing IT and/or cybersecurity personnel.
Level 3 Well Defined
Artificial Intelligence and Autonomous Technology (AAT) capabilities are standardized across the entity for applicability to People, Processes, Technologies, Data and/or Facilities (PPTDF) to ensure consistency for Technology Assets, Applications, Services and/or Data (TAASD). Capability criteria associated with this control reasonably expect the following criteria to exist:
▪ Policies and standards associated with AAT domain capabilities are formally documented and centrally-managed by the entity's Governance, Risk & Compliance (GRC) team, or similar function.
▪ Standardized Operating Procedures (SOP) associated with AAT domain capabilities are well-documented and kept current by process owners.
▪ An Artificial Intelligence Governance (AIG) team, or similar function, is appropriately staffed and supported to implement and maintain AAT domain capabilities.
▪ Technology is leveraged to enhance the efficiency and accuracy of AI governance, risk management and compliance operations (e.g., dedicated AI governance platform).
▪ The entity's Governance, Risk & Compliance (GRC) team, or similar function, works with business stakeholders and process owners to appropriately scope and reasonably implement cybersecurity and data protection controls associated with AAT domain capabilities to address Minimum Compliance Requirements (MCR) (e.g., applicable statutory, regulatory and/or contractual requirements) and Discretionary Security Requirements (DSR) (e.g., entity-required controls).
▪ An implemented and operational capability exists to designate AAT "High Risk" if one(1), or more, of the following criteria are met:
(1) AAT is used as a safety component of a product or service;
(2) AAT poses a significant risk of harm to an individual's health, safety or fundamental rights; and/or
(3) AAT materially influences the outcome of an individual's decision making.
Level 4 Quantitatively Controlled
Utilize SCR-CMM Level 3 criteria definitions:
▪ There are no defined Level 4 criteria, since it is reasonable to assume a quantitatively-controlled process is not necessary to operationalize this control.
▪ While it may be possible to develop “metrics-driven” capabilities for this control, the criteria would be organization-specific to define.
Level 5 Continuously Improving
Utilize SCR-CMM Level 3 or Level 4 (if available) criteria definitions:
▪ There are no defined Level 5 criteria, since it is reasonable to assume a continuously-improving process is not necessary to operationalize this control.
▪ Level 5 capabilities should be considered “world-class” where the control builds on Level 4 capabilities, but are continuously improving through Artificial Intelligence (AI) and/or Machine Learning (ML) technologies.
▪ While it may be possible to develop responsive capabilities for this control through the use of AI and/or ML technologies, the criteria would be organization-specific to define.
1. Overview
1.1 References
1.2 Identified Requirements
1.3 Related Regulations
2. Identified Requirements
Requirements
| Source |
Requirement |
3. Related Regulations
Regulations
| Source |
Regulation |
|
EULAW
|
Article 6 Classification rules for high-risk AI systems
Article 6
Classification rules for high-risk AI systems
1. Irrespective of whether an AI system is placed on the market or put into service independently of the products referred to in points (a) and (b), that AI system shall be considered to be high-risk where both of the following conditions are fulfilled:
|
(a)
|
the AI system is intended to be used as a safety component of a product, or the AI system is itself a product, covered by the Union harmonisation legislation listed in Annex I;
|
|
(b)
|
the product whose safety component pursuant to point (a) is the AI system, or the AI system itself as a product, is required to undergo a third-party conformity assessment, with a view to the placing on the market or the putting into service of that product pursuant to the Union harmonisation legislation listed in Annex I.
|
2. In addition to the high-risk AI systems referred to in paragraph 1, AI systems referred to in Annex III shall be considered to be high-risk.
3. By derogation from paragraph 2, an AI system referred to in Annex III shall not be considered to be high-risk where it does not pose a significant risk of harm to the health, safety or fundamental rights of natural persons, including by not materially influencing the outcome of decision making.
The first subparagraph shall apply where any of the following conditions is fulfilled:
|
(a)
|
the AI system is intended to perform a narrow procedural task;
|
|
(b)
|
the AI system is intended to improve the result of a previously completed human activity;
|
|
(c)
|
the AI system is intended to detect decision-making patterns or deviations from prior decision-making patterns and is not meant to replace or influence the previously completed human assessment, without proper human review; or
|
|
(d)
|
the AI system is intended to perform a preparatory task to an assessment relevant for the purposes of the use cases listed in Annex III.
|
Notwithstanding the first subparagraph, an AI system referred to in Annex III shall always be considered to be high-risk where the AI system performs profiling of natural persons.
4. A provider who considers that an AI system referred to in Annex III is not high-risk shall document its assessment before that system is placed on the market or put into service. Such provider shall be subject to the registration obligation set out in Article 49(2). Upon request of national competent authorities, the provider shall provide the documentation of the assessment.
5. The Commission shall, after consulting the European Artificial Intelligence Board (the ‘Board’), and no later than 2 February 2026, provide guidelines specifying the practical implementation of this Article in line with Article 96 together with a comprehensive list of practical examples of use cases of AI systems that are high-risk and not high-risk.
6. The Commission is empowered to adopt delegated acts in accordance with Article 97 in order to amend paragraph 3, second subparagraph, of this Article by adding new conditions to those laid down therein, or by modifying them, where there is concrete and reliable evidence of the existence of AI systems that fall under the scope of Annex III, but do not pose a significant risk of harm to the health, safety or fundamental rights of natural persons.
7. The Commission shall adopt delegated acts in accordance with Article 97 in order to amend paragraph 3, second subparagraph, of this Article by deleting any of the conditions laid down therein, where there is concrete and reliable evidence that this is necessary to maintain the level of protection of health, safety and fundamental rights provided for by this Regulation.
8. Any amendment to the conditions laid down in paragraph 3, second subparagraph, adopted in accordance with paragraphs 6 and 7 of this Article shall not decrease the overall level of protection of health, safety and fundamental rights provided for by this Regulation and shall ensure consistency with the delegated acts adopted pursuant to Article 7(1), and take account of market and technological developments.
|
|
EULAW
|
Article 13 Transparency and provision of information to deployers
Article 13
Transparency and provision of information to deployers
1. High-risk AI systems shall be designed and developed in such a way as to ensure that their operation is sufficiently transparent to enable deployers to interpret a system’s output and use it appropriately. An appropriate type and degree of transparency shall be ensured with a view to achieving compliance with the relevant obligations of the provider and deployer set out in Section 3.
2. High-risk AI systems shall be accompanied by instructions for use in an appropriate digital format or otherwise that include concise, complete, correct and clear information that is relevant, accessible and comprehensible to deployers.
3. The instructions for use shall contain at least the following information:
|
(a)
|
the identity and the contact details of the provider and, where applicable, of its authorised representative;
|
|
(b)
|
the characteristics, capabilities and limitations of performance of the high-risk AI system, including:
|
(i)
|
its intended purpose;
|
|
(ii)
|
the level of accuracy, including its metrics, robustness and cybersecurity referred to in Article 15 against which the high-risk AI system has been tested and validated and which can be expected, and any known and foreseeable circumstances that may have an impact on that expected level of accuracy, robustness and cybersecurity;
|
|
(iii)
|
any known or foreseeable circumstance, related to the use of the high-risk AI system in accordance with its intended purpose or under conditions of reasonably foreseeable misuse, which may lead to risks to the health and safety or fundamental rights referred to in Article 9(2);
|
|
(iv)
|
where applicable, the technical capabilities and characteristics of the high-risk AI system to provide information that is relevant to explain its output;
|
|
(v)
|
when appropriate, its performance regarding specific persons or groups of persons on which the system is intended to be used;
|
|
(vi)
|
when appropriate, specifications for the input data, or any other relevant information in terms of the training, validation and testing data sets used, taking into account the intended purpose of the high-risk AI system;
|
|
(vii)
|
where applicable, information to enable deployers to interpret the output of the high-risk AI system and use it appropriately;
|
|
|
(c)
|
the changes to the high-risk AI system and its performance which have been pre-determined by the provider at the moment of the initial conformity assessment, if any;
|
|
(d)
|
the human oversight measures referred to in Article 14, including the technical measures put in place to facilitate the interpretation of the outputs of the high-risk AI systems by the deployers;
|
|
(e)
|
the computational and hardware resources needed, the expected lifetime of the high-risk AI system and any necessary maintenance and care measures, including their frequency, to ensure the proper functioning of that AI system, including as regards software updates;
|
|
(f)
|
where relevant, a description of the mechanisms included within the high-risk AI system that allows deployers to properly collect, store and interpret the logs in accordance with Article 12.
|
|
|
EULAW
|
Article 51 Classification of general-purpose AI models as general-purpose AI models with systemic risk
Article 51
Classification of general-purpose AI models as general-purpose AI models with systemic risk
1. A general-purpose AI model shall be classified as a general-purpose AI model with systemic risk if it meets any of the following conditions:
|
(a)
|
it has high impact capabilities evaluated on the basis of appropriate technical tools and methodologies, including indicators and benchmarks;
|
|
(b)
|
based on a decision of the Commission, ex officio or following a qualified alert from the scientific panel, it has capabilities or an impact equivalent to those set out in point (a) having regard to the criteria set out in Annex XIII.
|
2. A general-purpose AI model shall be presumed to have high impact capabilities pursuant to paragraph 1, point (a), when the cumulative amount of computation used for its training measured in floating point operations is greater than 1025.
3. The Commission shall adopt delegated acts in accordance with Article 97 to amend the thresholds listed in paragraphs 1 and 2 of this Article, as well as to supplement benchmarks and indicators in light of evolving technological developments, such as algorithmic improvements or increased hardware efficiency, when necessary, for these thresholds to reflect the state of the art.
|
Linked Issues
- Secure Controls Framework -
"The SCF is the Common Controls Framework™ (CCF), the world's most comprehensive cybersecurity and data privacy metaframework - it is also free to use. The entire concept is building secure, compliant and resilient capabilities in the most efficient and cost-effective manner possible.
The SCF is more than just a unified control catalog, since its included content creates a playbook for Governance, Risk & Compliance (GRC) capabilities. Used globally by organizations of every size, the SCF is a robust and scalable solution for security, compliance and resilience controls. As a comprehensive security framework, the SCF maps 1,400+ controls across 200+ laws, regulations, and industry frameworks so you can implement once and comply everywhere.
Like it or not, cybersecurity is a protracted war on an asymmetric battlefield, where the threats are everywhere and as defenders we have to make the effort to work together to help improve cybersecurity and data privacy practices, since we all suffer when massive data breaches occur or when cyber attacks have physical impacts. Hackers share information on attack methods with other hackers, so why shouldn’t the good guys share information on how to best protect an organization? We decided to take action and make a difference, since we feel it is too important to wait for someone else to fix the problems that exist.
The SCF is made up of volunteers, mainly specialists within the cybersecurity profession, who focus on GRC and the cybersecurity side of data privacy. These are auditors, engineers, architects, incident responders, consultants and other specialists who live and breathe these topics on a daily basis. The end product is "expert-derived content" that makes up the SCF." https://securecontrolsframework.com/
Terms & Conditions
The SCF End User License Agreement (EULA) governs the use of the Secure Controls Framework® (SCF) under the Creative Commons Attribution-No Derivatives 4.0 International Public License.
|