From Democracies to Autocracies: How AI Systems Enable Authoritarianism by Design

From Democracies to Autocracies: How AI Systems Enable Authoritarianism by Design

AI-enabled authoritarianism is not confined to autocracies. In this paper, we provide greater transparency by investigating and mapping the lifecycles of six AI systems deployed in different political regimes, ranging from the US to China. By drawing on an extensive range of sources (academic publications, investigative research reports, third-party evaluations, media interviews, government procurement notices), we conduct a systematic, qualitative comparison across systems to identify the critical technical and operational features that enable authoritarianism within their respective political contexts. We find that enabling features include the centralization and co-optation of administrative data for law enforcement and political punishment, regulatory gaps that fail to deter misuse, weak user compliance that nullifies human oversight mechanisms, and the encoding of protected group traits that identify members of vulnerable populations. We find that these features are present across systems deployed in autocratic and democratic regimes, albeit in varying configurations. We also find that both centralized and fragmented AI systems can contribute to authoritarianism by exploiting governance gaps: centralized systems directed by executive authorities, particularly within security and military institutions, are often not subjected to formal oversight mechanisms, while fragmented systems diffuse accountability between stakeholders, paving the way for entrenchment. These findings reveal that AI-enabled authoritarianism is distributed, resulting from design and operational choices made by developers, administrators, and users alike. We conclude with recommendations for developers and policymakers to mitigate these risks.

Researchers Detail How AI Systems Can Enable Authoritarianism

A new preprint study of authoritarianism-enabling AI presents another set of evidence of the gaps in safeguards, writes Tim Bernard.

The rise of AI sovereignty: Authoritarian technological imaginaries as a form of reflexive control

Recent literature increasingly examines generative artificial intelligence's (AI's) role in disinformation and propaganda as a key driver of digital authoritarianism. Yet, less attention has been paid to how authoritarian regimes may influence the global governance of AI itself. This essay addresses this gap by analysing the impact of authoritarian technological imaginaries, focusing on the rise of ‘AI sovereignty’ as a central concept shaping global AI governance. It adopts a conceptual framework that combines the notion of imaginaries with the concept of reflexive control, originating in Soviet strategic thought and referring to influencing an adversary's perceptions so they act in ways aligned with the initiator's goals, to study authoritarian diffusion and the global politics of AI. Through an examination of public statements by the Russian president, the study shows how Russia constructs an AI imaginary that extends Cold War logics of nuclear competition into the digital era, framing AI as a matter of security and survival rather than ethics or rights. Drawing on the Russian case, the analysis identifies three frames that underpin the authoritarian AI imaginary: AI as a tool for power and domination; Western AI as a cultural threat; and AI as a guarantor of state efficiency. The article argues that such imaginaries shape the cognitive environment of global policy debates, promoting sovereignty-centred and securitised approaches to AI governance. It suggests that recognising the role of reflexive control in authoritarian diffusion helps distinguish legitimate Global South concerns from authoritarian influence and supports more inclusive and resilient models of AI governance.

The rise of AI sovereignty: Authoritarian technological imaginaries as a form of reflexive control

Recent literature increasingly examines generative artificial intelligence's (AI's) role in disinformation and propaganda as a key driver of digital authoritarianism. Yet, less attention has been paid to how authoritarian regimes may influence the global governance of AI itself. This essay addresses this gap by analysing the impact of authoritarian technological imaginaries, focusing on the rise of ‘AI sovereignty’ as a central concept shaping global AI governance. It adopts a conceptual framework that combines the notion of imaginaries with the concept of reflexive control, originating in Soviet strategic thought and referring to influencing an adversary's perceptions so they act in ways aligned with the initiator's goals, to study authoritarian diffusion and the global politics of AI. Through an examination of public statements by the Russian president, the study shows how Russia constructs an AI imaginary that extends Cold War logics of nuclear competition into the digital era, framing AI as a matter of security and survival rather than ethics or rights. Drawing on the Russian case, the analysis identifies three frames that underpin the authoritarian AI imaginary: AI as a tool for power and domination; Western AI as a cultural threat; and AI as a guarantor of state efficiency. The article argues that such imaginaries shape the cognitive environment of global policy debates, promoting sovereignty-centred and securitised approaches to AI governance. It suggests that recognising the role of reflexive control in authoritarian diffusion helps distinguish legitimate Global South concerns from authoritarian influence and supports more inclusive and resilient models of AI governance.

Loss of Oversight: How AI systems may become harder to audit, monitor, and investigate

The safety of advanced AI systems increasingly depends on the ability to oversee them: to audit models for concerning behaviours before deployment, monitor their activity during operation, and investigate incidents after they occur. This report maps the landscape of AI oversight and assesses how it is likely to change. Drawing on 25 expert interviews across frontier AI developers, government, NGOs, and academia, together with a literature review and internal analysis, we examine five sources of oversight signal: model behaviour, chain-of-thought reasoning, internals activations and circuits, memory architectures, and honesty training. For each source, we identify the properties that current oversight relies on, the pathways by which these properties could degrade, and the technical levers available to preserve them. Our central finding is that literature and expert opinion support the conclusion that current oversight rests on foundations that are likely to erode, absent effective intervention. We recommend that developers track and report shifts in oversight-relevant properties, preserve oversight affordances by design, and invest in emerging oversight techniques as fallbacks against continued degradation of current methods.

Artificial intelligence in government: why people feel they lose control

The use of Artificial Intelligence (AI) in public administration is expanding rapidly. While AI promises greater efficiency and responsiveness, its integration into government and administration ra...

From chatbots to assistants: governance is key for AI agents

AI's shift into agentic technology ushers in a new set of governance and security challenges that will mean defining to what extent they should be autonomous

Human-Computer Insurrection: Notes on an Anarchist HCI

The HCI community has worked to expand and improve our consideration of the societal implications of our work and our corresponding responsibilities. Despite this increased engagement, HCI continues to lack an explicitly articulated politic, which we argue re-inscribes and amplifies systemic oppression. In this paper, we set out an explicit political vision of an HCI grounded in emancipatory autonomy - an anarchist HCI, aimed at dismantling all oppressive systems by mandating suspicion of and a reckoning with imbalanced distributions of power. We outline some of the principles and accountability mechanisms that constitute an anarchist HCI. We offer a potential framework for radically reorienting the field towards creating prefigurative counterpower - systems and spaces that exemplify the world we wish to see, as we go about building the revolution in increment.

Human-Computer Insurrection: Notes on an Anarchist HCI

The HCI community has worked to expand and improve our consideration of the societal implications of our work and our corresponding responsibilities. Despite this increased engagement, HCI continues to lack an explicitly articulated politic, which we argue re-inscribes and amplifies systemic oppression. In this paper, we set out an explicit political vision of an HCI grounded in emancipatory autonomy - an anarchist HCI, aimed at dismantling all oppressive systems by mandating suspicion of and a reckoning with imbalanced distributions of power. We outline some of the principles and accountability mechanisms that constitute an anarchist HCI. We offer a potential framework for radically reorienting the field towards creating prefigurative counterpower - systems and spaces that exemplify the world we wish to see, as we go about building the revolution in increment.

To Repress or to Co‐opt? Authoritarian Control in the Age of Digital Surveillance

This article studies the consequences of digital surveillance in dictatorships. I first develop an informational theory of repression and co-optation. I argue that digital surveillance resolves dicta...

Predictive policing AI is on the rise − making it accountable to the public could curb its harmful effects

AI that anticipates where crimes are likely to occur and who might commit them has a troubling track record. Democratic accountability could shine a light on the technology and how it’s used.

Predictive policing AI is on the rise − making it accountable to the public could curb its harmful effects

AI that anticipates where crimes are likely to occur and who might commit them has a troubling track record. Democratic accountability could shine a light on the technology and how it’s used.

Information Access of the Oppressed: A Problem-Posing Framework for Envisioning Emancipatory Information Access Platforms

Online information access (IA) platforms are targets of authoritarian capture. These concerns are particularly serious and urgent today in light of the rising levels of democratic erosion worldwide, the emerging capabilities of generative AI technologies such as AI persuasion, and the increasing concentration of economic and political power in the hands of Big Tech. This raises the question of what alternative IA infrastructure we must reimagine and build to mitigate the risks of authoritarian capture of our information ecosystems. We explore this question through the lens of Paulo Freire's theories of emancipatory pedagogy. Freire's theories provide a radically different lens for exploring IA's sociotechnical concerns relative to the current dominating frames of fairness, accountability, confidentiality, transparency, and safety. We make explicit, with the intention to challenge, the dichotomy of how we relate to technology as either technologists (who envision and build technology) and its users. We posit that this mirrors the teacher-student relationship in Freire's analysis. By extending Freire's analysis to IA, we challenge the notion that it is the burden of the (altruistic) technologists to come up with interventions to mitigate the risks that emerging technologies pose to marginalized communities. Instead, we advocate that the first task for the technologists is to pose these as problems to the marginalized communities, to encourage them to make and unmake the technology as part of their material struggle against oppression. Their second task is to redesign our online technology stacks to structurally expose spaces for community members to co-opt and co-construct the technology in aid of their emancipatory struggles. We operationalize Freire's theories to develop a problem-posing framework for envisioning emancipatory IA platforms of the future.

Runtime Governance for AI Agents: Policies on Paths

AI agents -- systems that plan, reason, and act using large language models -- produce non-deterministic, path-dependent behavior that cannot be fully governed at design time, where with governed we mean striking the right balance between as high as possible successful task completion rate and the legal, data-breach, reputational and other costs associated with running agents. We argue that the execution path is the central object for effective runtime governance and formalize compliance policies as deterministic functions mapping agent identity, partial path, proposed next action, and organizational state to a policy violation probability. We show that prompt-level instructions (and "system prompts"), and static access control are special cases of this framework: the former shape the distribution over paths without actually evaluating them; the latter evaluates deterministic policies that ignore the path (i.e., these can only account for a specific subset of all possible paths). In our view, runtime evaluation is the general case, and it is necessary for any path-dependent policy. We develop the formal framework for analyzing AI agent governance, present concrete policy examples (inspired by the AI act), discuss a reference implementation, and identify open problems including risk calibration and the limits of enforced compliance.

How malicious AI swarms can threaten democracy

The fusion of agentic AI and LLMs marks a new frontier in information warfare , Advances in artificial intelligence (AI) offer the prospect of manipulating beliefs and behaviors on a population-wide level ( 1 ). Large language models (LLMs) and autonomous agents ( 2 ) let influence campaigns reach unprecedented scale and precision. Generative tools can expand propaganda output without sacrificing credibility ( 3 ) and inexpensively create falsehoods that are rated as more human-like than those written by humans ( 3 , 4 ). Techniques meant to refine AI reasoning, such as chain-of-thought prompting, can be used to generate more convincing falsehoods. Enabled by these capabilities, a disruptive threat is emerging: swarms of collaborative, malicious AI agents. Fusing LLM reasoning with multiagent architectures ( 2 ), these systems are capable of coordinating autonomously, infiltrating communities, and fabricating consensus efficiently. By adaptively mimicking human social dynamics, they threaten democracy. Because the resulting harms stem from design, commercial incentives, and governance, we prioritize interventions at multiple leverage points, focusing on pragmatic mechanisms over voluntary compliance.

How malicious AI swarms can threaten democracy

The fusion of agentic AI and LLMs marks a new frontier in information warfare , Advances in artificial intelligence (AI) offer the prospect of manipulating beliefs and behaviors on a population-wide level ( 1 ). Large language models (LLMs) and autonomous agents ( 2 ) let influence campaigns reach unprecedented scale and precision. Generative tools can expand propaganda output without sacrificing credibility ( 3 ) and inexpensively create falsehoods that are rated as more human-like than those written by humans ( 3 , 4 ). Techniques meant to refine AI reasoning, such as chain-of-thought prompting, can be used to generate more convincing falsehoods. Enabled by these capabilities, a disruptive threat is emerging: swarms of collaborative, malicious AI agents. Fusing LLM reasoning with multiagent architectures ( 2 ), these systems are capable of coordinating autonomously, infiltrating communities, and fabricating consensus efficiently. By adaptively mimicking human social dynamics, they threaten democracy. Because the resulting harms stem from design, commercial incentives, and governance, we prioritize interventions at multiple leverage points, focusing on pragmatic mechanisms over voluntary compliance.