Chapter 4: Deterrence with Mutual Assured AI Malfunction (MAIM). Rapid advances in AI are beginning to reshape national security. Destabilizing AI developments could rupture the balance of power and raise the odds of great-power conflict, while widespread proliferation of capable AI hackers and virologists would lower barriers for rogue actors to cause catastrophe.
Action Plan to increase the safety and security of advanced AI
The first U.S. government-commissioned assessment on catastrophic national security risks from advanced AI on the path to AGI.
Top AI Security Incidents of 2025 Revealed | Adversa AI
Discover how AI systems are being hacked in the wild — from prompt injection to agent abuse — with real breaches, lessons, and defenses in Adversa AI’s 2025 report.
Artificial Intelligence Safety and Security License Requirements
Artificial Intelligence Safety and Security License Requirements Artificial Intelligence (AI) is now broadly capable, which can produce both benefits and dangers. AI will hasten the design and proliferation of bioweapons, cyberweapons, nuclear weapons, progressively more general intelligence,...
Securing internal systems against increasingly capable and imperfectly aligned AI
Discover our AI Control Roadmap: a defense-in-depth system to securely manage advanced, potentially misaligned AI agents.
Building Political Superintelligence
Amidst fears of dystopia, a blueprint for how we use AI to reinvent the way we govern ourselves
Inside the suddenly explosive world of AI safety
AI safety researchers warned that popular models would go rogue. This is only the beginning.
The Malicious Use of Artificial Intelligence: Forecasting,...
This report surveys the landscape of potential security threats from malicious uses of AI, and proposes ways to better forecast, prevent, and mitigate these threats. After analyzing the ways in...
Gradual disempowerment | 80,000 Hours
The proliferation of advanced AI systems may lead to the gradual disempowerment of humanity, even if efforts to prevent them from becoming power-seeking or scheming are successful. Humanity may be incentivised to hand over increasing amounts of control to AIs, giving them power over the economy, politics, culture, and more. Over time, humanity's interests may be sidelined and our control over the future undermined, potentially constituting an existential catastrophe. There's disagreement over how serious a problem this is and how it relates to [other concerns about AI alignment](https://80000hours.o
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
Import AI 447: The AGI economy; testing AIs with generated games; and agent ecologies
What might a superintelligence arcology be like?
AI's Catastrophic Risk Isn't Rogue Machines, It's Cognitive Surrender
For many, AI introduces the suspicion that self-investment is inherently a losing proposition, writes recent graduate Evan Liu.
Statement from Dario Amodei on our discussions with the Department of War
A statement from our CEO on national security uses of AI
Import AI 446: Nuclear LLMs; China's big AI benchmark; measurement and AI policy
Will AIs be jealous of one another?
Safety Co-Option and Compromised National Security: The Self-Fulfilling Prophecy of Weakened AI Risk Thresholds
Risk thresholds provide a measure of the level of risk exposure that a society or individual is willing to withstand, ultimately shaping how we determine the safety of technological systems. Against the backdrop of the Cold War, the first risk analyses, such as those devised for nuclear systems, cemented societally accepted risk thresholds against which safety-critical and defense systems are now evaluated. But today, the appropriate risk tolerances for AI systems have yet to be agreed on by global governing efforts, despite the need for democratic deliberation regarding the acceptable levels of harm to human life. Absent such AI risk thresholds, AI technologists-primarily industry labs, as well as "AI safety" focused organizations-have instead advocated for risk tolerances skewed by a purported AI arms race and speculative "existential" risks, taking over the arbitration of risk determinations with life-or-death consequences, subverting democratic processes. In this paper, we demonstrate how such approaches have allowed AI technologists to engage in "safety revisionism," substituting traditional safety methods and terminology with ill-defined alternatives that vie for the accelerated adoption of military AI uses at the cost of lowered safety and security thresholds. We explore how the current trajectory for AI risk determination and evaluation for foundation model use within national security is poised for a race to the bottom, to the detriment of the US's national security interests. Safety-critical and defense systems must comply with assurance frameworks that are aligned with established risk thresholds, and foundation models are no exception. As such, development of evaluation frameworks for AI-based military systems must preserve the safety and security of US critical and defense infrastructure, and remain in alignment with international humanitarian law.
Do AI Risks Require Extraordinary Government Intervention?
Let’s not skip the hard work of AI governance
How Shifting Responsibility for AI Harms Undermines Democratic Accountability | TechPolicy.Press
The moralization of individual AI use deflects responsibility away from powerful actors like corporations and governments, Suvradip Maitra and others write.
