The first U.S. government-commissioned assessment on catastrophic national security risks from advanced AI on the path to AGI.
Artificial Intelligence Safety and Security License Requirements
Artificial Intelligence Safety and Security License Requirements Artificial Intelligence (AI) is now broadly capable, which can produce both benefits and dangers. AI will hasten the design and proliferation of bioweapons, cyberweapons, nuclear weapons, progressively more general intelligence,...
AI in 2025: gestalt — LessWrong
This is the editorial for this year’s "Shallow Review of AI Safety". (It got long enough to stand alone.) …
Deterrence with Mutual Assured AI Malfunction (MAIM) — Chapter 4 of Superintelligence Strategy
Chapter 4: Deterrence with Mutual Assured AI Malfunction (MAIM). Rapid advances in AI are beginning to reshape national security. Destabilizing AI developments could rupture the balance of power and raise the odds of great-power conflict, while widespread proliferation of capable AI hackers and virologists would lower barriers for rogue actors to cause catastrophe.
OWASP GenAI Security Project Releases Top 10 Risks and Mitigations for Agentic AI Security
Culmination of over 100 industry leaders’ input and extensive published resources to deliver critical guidance to address Agentic AI Security risks WILMINGTON, Del. — Dec. 10, 2025 — The OWASP GenAI Security Project (genai.owasp.org), a leading global open-source and expert community dedicated to delivering practical guidance and tools for securing generative and agentic AI, […]
Top AI Security Incidents of 2025 Revealed | Adversa AI
Discover how AI systems are being hacked in the wild — from prompt injection to agent abuse — with real breaches, lessons, and defenses in Adversa AI’s 2025 report.
Safety Co-Option and Compromised National Security: The Self-Fulfilling Prophecy of Weakened AI Risk Thresholds
Risk thresholds provide a measure of the level of risk exposure that a society or individual is willing to withstand, ultimately shaping how we determine the safety of technological systems. Against the backdrop of the Cold War, the first risk analyses, such as those devised for nuclear systems, cemented societally accepted risk thresholds against which safety-critical and defense systems are now evaluated. But today, the appropriate risk tolerances for AI systems have yet to be agreed on by global governing efforts, despite the need for democratic deliberation regarding the acceptable levels of harm to human life. Absent such AI risk thresholds, AI technologists-primarily industry labs, as well as "AI safety" focused organizations-have instead advocated for risk tolerances skewed by a purported AI arms race and speculative "existential" risks, taking over the arbitration of risk determinations with life-or-death consequences, subverting democratic processes. In this paper, we demonstrate how such approaches have allowed AI technologists to engage in "safety revisionism," substituting traditional safety methods and terminology with ill-defined alternatives that vie for the accelerated adoption of military AI uses at the cost of lowered safety and security thresholds. We explore how the current trajectory for AI risk determination and evaluation for foundation model use within national security is poised for a race to the bottom, to the detriment of the US's national security interests. Safety-critical and defense systems must comply with assurance frameworks that are aligned with established risk thresholds, and foundation models are no exception. As such, development of evaluation frameworks for AI-based military systems must preserve the safety and security of US critical and defense infrastructure, and remain in alignment with international humanitarian law.
The risks of advanced AI
Could working on AI risks be the highest-impact career choice today? Explore why AI may trigger rapid, dramatic societal change — and what you can do about it.
IIIb. Lock Down the Labs: Security for AGI - SITUATIONAL AWARENESS
The nation’s leading AI labs treat security as an afterthought. Currently, they’re basically handing the key secrets for AGI to the CCP on a silver platter. Securing the AGI secrets and weights against the state-actor threat will be an immense effort, and we’re not on track. They met in the evening in Wigner’s office. “Szilard
AIRO (Automated AI Risk Outlook)
AI Act Scientific Panel
The Scientific Panel advises the AI Office and national authorities on the implementation the AI Act and assessment of the impacts and risks of General-Purpose AI (GPAI) models.
AI Act
The AI Act is the first-ever legal framework on AI, which addresses the risks of AI and positions Europe to play a leading role globally.
Statement from Dario Amodei on our discussions with the Department of War
A statement from our CEO on national security uses of AI
AI.Gov | President Trump's AI Strategy and Action Plan
Explore President Trump’s AI initiatives focused on innovation, infrastructure, international engagement, and youth education in artificial intelligence.
U.S. Must Act Quickly to Avoid Risks From AI, Report Says
The U.S. government must move “decisively” to avert an “extinction-level threat" to humanity from AI, says a government-commissioned report
Our updated Preparedness Framework
Sharing our updated framework for measuring and protecting against severe harm from frontier AI capabilities.
