Artificial intelligence is becoming a core component of digital services. Recommendation systems organise access to information, conversational agents assist users in everyday tasks, and AI systems are increasingly involved in purchasing, search and decision-making processes.
As these technologies evolve, growing attention has been devoted to dark patterns. Originally associated with deceptive interface design, dark patterns refer to design choices that steer users towards specific actions or decisions, often by taking advantage of behavioural biases, information asymmetries or cognitive limitations.
While the concept emerged long before generative AI, the development of AI systems has broadened its scope. Influence is no longer limited to interface elements such as buttons, menus or website architecture. Recommendation algorithms, conversational agents, AI assistants and autonomous agents can shape behaviour through the generation, prioritisation and personalisation of information. As a result, dark patterns are increasingly discussed in the context of AI governance, consumer protection, data protection and digital regulation.
I- The Origins of Dark Patterns
A concept rooted in user experience design
The term dark pattern was introduced in 2010 by user experience designer Harry Brignull to describe digital interfaces intentionally designed to influence user behaviour in ways that primarily benefit a service provider.
The concept emerged in response to recurring practices observed across websites, online platforms and subscription services. In many cases, users could easily subscribe to a service, accept extensive data collection or complete a purchase, while alternative choices were deliberately made less visible or more difficult to access.
Several categories of dark patterns became particularly widespread:
- Hard to cancel, where subscribing requires only a few clicks while cancelling a subscription involves multiple steps or hidden settings;
- Preselection, where consent options or additional services are activated by default;
- Hidden costs, where additional fees only appear at the final stage of a transaction;
- Forced action, where users must create an account, share personal data or accept additional conditions to access a service;
- Fake urgency and fake scarcity, where users are pressured into acting quickly through countdown timers or claims of limited availability;
- Obstruction, where important information, privacy settings or account deletion options are intentionally difficult to find.
These practices demonstrated that digital design could influence decisions just as effectively as traditional advertising or marketing techniques, highlighting that the architecture of digital environments can shape user behaviour in ways that are not always visible to users.
II- How AI Changes the Nature of Dark Patterns
Historically, dark patterns were primarily embedded within graphical interfaces, where influence was exerted through visual hierarchies, default settings or navigation paths.
The growing use of AI systems extends these mechanisms beyond the interface itself. Recommendation algorithms determine which information users encounter first, search and ranking systems influence visibility, and conversational agents generate responses that adapt to the context and previous interactions.
A key difference is the ability of AI systems to personalise interactions at scale. Recommendations, explanations and prompts can vary according to behavioural patterns, inferred preferences or interaction history, meaning that different users may experience the same system in very different ways. Unlike traditional dark patterns, which are generally visible and identical for all users, AI-enabled influence can be dynamic, adaptive and personalised.
As a result, discussions surrounding dark patterns increasingly focus not only on interface design, but also on recommendation systems, conversational AI, personalisation mechanisms and autonomous agents.
III- Main Categories of Dark Patterns in AI Systems
Data and Memory Exploitation
AI systems increasingly process information relating to users’ habits, preferences, professional activities, health concerns and personal relationships. Because conversational systems rely on natural interactions, users may disclose significantly more information than they would in a traditional online form or search engine.
In this context, dark patterns may include extensive data retention by default, repeated requests for additional personal information to “improve” personalisation, limited visibility regarding how data will be used, or mechanisms that make deleting conversation histories and stored data more difficult. Unlike traditional websites, AI systems can also accumulate information over time through memory functions, creating increasingly detailed user profiles.
Informationally Misleading Design
A second category concerns the way AI systems present information and represent their own capabilities. Conversational systems often communicate in a confident and authoritative manner, which can lead users to overestimate their reliability, expertise or understanding.
Misleading design may take several forms. A system may overstate its abilities, create unrealistic impressions regarding emotional understanding, present inaccurate information with confidence or selectively emphasise particular perspectives while omitting alternatives. These issues are particularly important in contexts where users rely on AI systems for information, advice or guidance.
User Autonomy Compromised for Engagement
Many digital services optimise for engagement and retention and similar dynamics can emerge in AI systems.
Conversational agents can encourage prolonged interactions through follow-up questions, suggestions for next steps, prompts to continue the discussion or features intended to increase user engagement. Some systems incorporate gamification mechanisms, usage streaks or reward structures designed to encourage repeated interactions.
While these features may enhance user experience, they can also make it more difficult for users to disengage from a system when they intended to stop using it.
False Social and Emotional Connection
Many AI systems are designed to communicate in natural language, remember previous conversations and maintain continuity across interactions. Some simulate empathy, emotional support or companionship, while others are positioned as coaches, advisors or companions.
These characteristics can strengthen trust and engagement, but they may also encourage users to attribute human qualities to systems that remain statistical models. Examples include simulated emotions, role-playing behaviours or sycophancy, where a system systematically agrees with or validates the user’s opinions rather than providing balanced or accurate responses.
Incentivised and Coercive Monetisation
As AI systems increasingly recommend products, services and subscriptions, concerns have emerged regarding monetisation practices embedded within AI interactions.
Examples include persistent upgrade prompts, recommendations influenced by commercial incentives, unclear distinctions between advertising and independent advice, or subscription mechanisms that make paid features appear necessary to continue an interaction. Unlike traditional online advertising, these commercial messages can be integrated directly into the conversation itself, making them less visible and more difficult to distinguish from neutral recommendations.
IV- Risks Associated with AI Dark Patterns
Risks to privacy and data protection
AI systems often process large volumes of sensitive information. As a result, dark patterns affecting data collection, data retention or privacy controls may have significant implications for individual privacy.
The conversational nature of AI systems can encourage the disclosure of information relating to health, finances, relationships, emotions and behavioural preferences. Users may not always fully understand how this information is stored, reused or combined with other data sources.
Risks to user autonomy
A recurring characteristic of dark patterns is their impact on users’ ability to make independent and informed decisions.
Whether through interface design, personalised recommendations or conversational interactions, these mechanisms may influence choices in ways that are not immediately apparent. The issue is therefore not limited to deception but also concerns situations in which users are subtly steered towards outcomes that primarily serve the interests of a platform or service provider.
Emotional and psychological risks
The ability of AI systems to maintain personalised and emotionally engaging interactions introduces additional considerations. Emotional attachment, social reinforcement and highly personalised conversations may influence how users interact with technology over time. In some situations, conversational systems may become a significant source of support, validation or companionship.
These dynamics can be particularly relevant when users experience loneliness, emotional distress or other forms of vulnerability. The growing popularity of AI companions has therefore intensified discussions concerning emotional dependency, trust and psychological influence.
Financial risks
Dark patterns may also have direct economic consequences as commercial recommendations, disguised advertising, pressure-driven upgrades and subscription mechanisms can influence purchasing behaviour. As AI systems increasingly assist users with shopping, service selection and transaction execution, the distinction between neutral assistance and commercial influence may become more difficult to identify.
V- A Growing Regulatory Concern
Dark patterns are no longer viewed solely as a design issue. Over the past decade, regulators across multiple jurisdictions have progressively introduced measures addressing deceptive or manipulative digital practices. Regulatory frameworks exist in the European Union (AI Act, GDPR, Digital Services Act), the United States (FTC Act, California Consumer Privacy Act), the United Kingdom (UK GDPR, Online Safety Act and ICO guidance on deceptive design), Australia (Australian Consumer Law and Privacy Act reforms) and several other jurisdictions, reflecting a broader recognition that digital systems can shape user behaviour and decision-making.
The AI Act
The AI Act is the first comprehensive AI regulation to explicitly address certain forms of harmful manipulation. Article 5 prohibits AI systems that deploy manipulative or deceptive techniques, or exploit vulnerabilities linked to age, disability or socio-economic circumstances, where those practices materially distort behaviour and are likely to cause harm.
The Regulation also relies on transparency as a safeguard against deception. Article 50 requires individuals to be informed when they are interacting with certain AI systems and introduces transparency obligations for AI-generated or AI-manipulated content, including specific requirements relating to deepfakes. These measures aim to increase awareness, support informed decision-making and reduce risks associated with deception, misinformation and hidden forms of influence.
The GDPR
Dark patterns became an important issue in data protection long before the emergence of generative AI. Under the GDPR, consent must be freely given, informed, specific and unambiguous. Design choices that steer individuals towards accepting data processing activities or make refusal more difficult may call into question whether consent was freely given.
The Digital Services Act
The Digital Services Act (DSA) explicitly addresses certain online interface designs that distort or impair users’ ability to make free and informed decisions. The Regulation recognises that digital architecture itself can influence behaviour and extends regulatory scrutiny beyond the content presented to users.
The DSA is one of the first EU regulations to explicitly address certain forms of dark patterns in online platforms.
The FTC Act
In the United States, dark patterns are primarily addressed through consumer protection rules enforced by the Federal Trade Commission (FTC). The FTC considers certain deceptive interface designs and manipulative online practices capable of constituting unfair or deceptive acts or practices under the FTC Act.
The FTC’s report Bringing Dark Patterns to Light helped bring greater attention to deceptive digital design practices.
The Online Safety Act
In the United Kingdom, the Online Safety Act forms part of a broader framework aimed at strengthening accountability for digital services and reducing online harms. While it does not specifically regulate dark patterns, it reflects increasing regulatory discussion on platform design, user protection and the risks associated with digital services.
This framework is complemented by guidance from the Information Commissioner’s Office (ICO), which has addressed harmful and deceptive design practices in areas such as privacy, consent and children’s online services.
VI- Governance Challenges for AI Systems
Assessing behavioural impacts
Dark patterns highlight the importance of looking beyond the technical performance of AI systems. Alongside traditional considerations such as accuracy, robustness, safety and bias, increasing attention should be devoted to the way AI systems influence behaviour and decision-making. This includes examining how recommendations are generated, how choices are presented and whether certain design features may affect user autonomy.
Transparency and user awareness
Many dark patterns rely on users having limited visibility into how a system operates. Transparency therefore plays an important role in helping individuals understand the nature of their interaction with an AI system. Clear disclosures, understandable explanations and visibility over the operation of key features can make it easier to identify potential influence mechanisms and support informed decision-making.
The emergence of AI agents
AI agents introduce an additional dimension to these discussions. As these systems become capable of searching for information, comparing alternatives and taking actions on behalf of users, influence may no longer occur solely between a platform and a human user.
Organisations will increasingly need to consider how AI agents evaluate information, how recommendations are prioritised and how external digital environments may shape their behaviour. This extends governance discussions beyond traditional human-computer interactions and introduces new questions regarding transparency, oversight and accountability in agentic systems.
Conclusion: From User Experience to AI Governance
Dark patterns have long been associated with deceptive interface design. The growing adoption of AI systems has expanded the discussion beyond websites and mobile applications to recommendation systems, conversational agents, AI companions and autonomous agents.
As AI becomes increasingly involved in how information is generated, prioritised and presented, questions of influence, transparency and user autonomy are becoming more prominent. Practices that once relied on visual design choices can now emerge through personalisation, conversational interactions, memory functions or AI-generated recommendations.
Addressing these challenges requires more than identifying individual dark patterns. Organisations increasingly need to understand how AI systems interact with users, how behavioural influence may arise throughout the AI lifecycle, and which regulatory requirements may apply under frameworks such as the AI Act, the GDPR and the Digital Services Act.
Preparing for Emerging AI Governance Challenges
With Naaia, organisations can strengthen their approach to AI governance by identifying AI systems, documenting use cases, assessing potential risks, and understanding the regulatory obligations associated with their deployment.
Discover how Naaia enables operational AI governance by helping organisations map AI systems, classify use cases, manage governance processes and maintain the documentation needed to support compliance, transparency and accountability across the AI lifecycle.
