{"id":897,"date":"2025-09-16T08:30:49","date_gmt":"2025-09-16T08:30:49","guid":{"rendered":"https:\/\/naaia.ai\/news\/generative-ai-regulation-risks-ai-act\/"},"modified":"2026-06-02T13:45:28","modified_gmt":"2026-06-02T13:45:28","slug":"generative-ai-regulation-risks-ai-act","status":"publish","type":"post","link":"https:\/\/naaia.ai\/en\/generative-ai-regulation-risks-ai-act\/","title":{"rendered":"Generative AI and regulation: understanding the risks and obligations of the AI Act"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">With the rise of&nbsp;<strong>AI systems<\/strong>, capable of acting, interacting, and sometimes deciding, questions of security, transparency, and responsibility become urgent. What are the&nbsp;<strong>risks of generative AI<\/strong>? How can they be anticipated without hindering innovation? Where do we stand on the regulation of generative AI?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The&nbsp;<strong>AI Act<\/strong>, the first attempt at&nbsp;<strong>AI regulation<\/strong>&nbsp;at the European level, seeks to respond to this challenge by introducing&nbsp;<strong>AI risk management<\/strong>&nbsp;based on the level of potential impact. The objective is to explore the notion of&nbsp;<strong>high-risk AI systems<\/strong>, the obligations for providers, specific cases related to&nbsp;<strong>AI agents<\/strong>, and the limits of this emerging regulatory framework. An essential insight for all those who design, use, or oversee technologies based on AI.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What is a high-risk AI system according to the AI Act?<\/strong><\/h2>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>A dual classification of high-risk systems<\/strong><\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The&nbsp;<strong>AI regulation<\/strong>&nbsp;distinguishes two main categories of&nbsp;<strong>high-risk AI systems<\/strong>:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Those linked to&nbsp;<strong>regulated products<\/strong>&nbsp;(Annex I)<\/li>\n\n\n\n<li>Those classified&nbsp;<strong>according to their purpose<\/strong>&nbsp;(Annex III)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">In both cases, the systems are subject to all the requirements of&nbsp;<strong>Title III of the AI Act regulation<\/strong>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Generative AI and regulation: necessary oversight in the face of sensitive uses<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The&nbsp;<strong>AI regulation<\/strong>&nbsp;no longer concerns only industrial players or researchers. It now affects companies from all sectors that integrate&nbsp;<strong>generative AI<\/strong>&nbsp;technologies into their tools, services, or decision-making processes. These autonomous agents, sometimes capable of complex actions, arouse both fascination and concern.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The&nbsp;<strong>AI Act<\/strong>&nbsp;acknowledges this reality by classifying certain uses as \u201c<strong>high risk<\/strong>\u201d when they may affect fundamental rights or public safety. This is notably the case for AI systems used in recruitment, education, healthcare, justice, or essential public services. This classification is not symbolic: it entails strong&nbsp;<strong>regulatory obligations<\/strong>&nbsp;for the designers and providers of these systems.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>A classification based on the risks of AI agents<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The text distinguishes two main types of situations in which an&nbsp;<strong>AI agent<\/strong>&nbsp;may be considered high risk. On the one hand, when it is integrated into a product regulated by the European Union, such as a medical device or an automated vehicle. On the other hand, when its&nbsp;<strong>purpose<\/strong>&nbsp;relates to sensitive fields such as biometrics, surveillance, education, or policing. It is&nbsp;<strong>Annex III of the AI Act<\/strong>&nbsp;that lists these critical uses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This classification aims to prevent the&nbsp;<strong>risks linked to generative AI<\/strong>, notably those related to the opacity of models, the biased reproducibility of certain decisions, or the uncontrolled automation of human processes. It is not a question of prohibition, but of requiring these systems to be designed in a more robust, ethical, and traceable manner.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Article 6(2): a safety net for generative AIs with limited use<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Not all&nbsp;<strong>generative AIs<\/strong>&nbsp;are automatically considered high risk. The&nbsp;<strong>AI Act<\/strong>&nbsp;provides for an&nbsp;<strong>exception to classification<\/strong>, defined in&nbsp;<strong>Article 6(2)<\/strong>. If an AI agent is limited to carrying out a technical task, such as converting documents or sorting files, without influencing or replacing a human decision, it may be excluded from the scope of the strictest obligations.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This clause makes it possible to distinguish between a&nbsp;<strong>generative assistance tool<\/strong>&nbsp;and an&nbsp;<strong>autonomous AI agent<\/strong>, capable of acting with a strong impact on users. It thus avoids over-regulating modest uses while strengthening oversight of the most sensitive cases.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>What obligations for designers of high-risk AI?<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Providers of&nbsp;<strong>high-risk AI<\/strong>&nbsp;must implement continuous&nbsp;<strong>AI risk management<\/strong>. This involves documenting the system\u2019s functions, assessing its risks at each stage of its life cycle, ensuring appropriate human oversight, and proving that the system is used in accordance with its declared purpose.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They must also comply with criteria of transparency, robustness, cybersecurity, and provide for a post-market monitoring system. Failure to meet these obligations may result in sanctions, but also a loss of credibility with users, partners, or regulators.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>Towards responsible and compliant generative AI<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The purpose of the&nbsp;<strong>AI Act<\/strong>&nbsp;is not to hinder innovation, but to enable the&nbsp;<strong>responsible development of generative AI<\/strong>. By overseeing the most sensitive uses, clarifying the responsibilities of stakeholders, and imposing technical and ethical safeguards, Europe seeks to lay the foundations for a safe, fair, and sustainable digital space.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Companies that anticipate this evolution by structuring their&nbsp;<strong>compliance with AI regulation<\/strong>&nbsp;today gain a head start. They protect themselves against abuses while strengthening the trust of their clients, users, and partners.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n","protected":false},"excerpt":{"rendered":"<p>With the rise of&nbsp;AI systems, capable of acting, interacting, and sometimes deciding, questions of security, transparency, and responsibility become urgent. What are the&nbsp;risks of generative AI? How can they be&hellip; <a href=\"https:\/\/naaia.ai\/en\/generative-ai-regulation-risks-ai-act\/\">Lire la suite<\/a><\/p>\n","protected":false},"author":2,"featured_media":891,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[46],"tags":[],"class_list":["post-897","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-governance-blog"],"_links":{"self":[{"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/posts\/897","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/comments?post=897"}],"version-history":[{"count":1,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/posts\/897\/revisions"}],"predecessor-version":[{"id":2375,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/posts\/897\/revisions\/2375"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/media\/891"}],"wp:attachment":[{"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/media?parent=897"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/categories?post=897"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/naaia.ai\/en\/wp-json\/wp\/v2\/tags?post=897"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}