Purpose-designed AI agents with a focus on safety can provide critical assistance to vulnerable populations.
How to Deploy AI Agents for Safety
How AI assistance impacts the formation of coding skills
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
The 2025 AI Agent Index
Agentic AI systems are increasingly capable of performing complex tasks with limited human involvement. The 2025 AI Agent Index documents the origins, design, capabilities, ecosystem, and safety features of 30 prominent AI agents based on publicly available information and correspondence with developers.
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
Do AI Risks Require Extraordinary Government Intervention?
Let’s not skip the hard work of AI governance
Taking AI Welfare Seriously
In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood, i.e. of AI systems with their own interests and moral significance, is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that language model outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are, or will be, conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.
Responsible AI
Discover how AWS is committing to developing AI responsibly – to built trust, promote the safe development of AI, and act as a force for good.
Deterrence with Mutual Assured AI Malfunction (MAIM) — Chapter 4 of Superintelligence Strategy
Chapter 4: Deterrence with Mutual Assured AI Malfunction (MAIM). Rapid advances in AI are beginning to reshape national security. Destabilizing AI developments could rupture the balance of power and raise the odds of great-power conflict, while widespread proliferation of capable AI hackers and virologists would lower barriers for rogue actors to cause catastrophe.
From chatbots to assistants: governance is key for AI agents
AI's shift into agentic technology ushers in a new set of governance and security challenges that will mean defining to what extent they should be autonomous
The lethal trifecta for AI agents: private data, untrusted content, and external communication
If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of …
Who decides when AI is too dangerous?
Anthropic asked for AI regulation, but not like this.
Building Political Superintelligence
Amidst fears of dystopia, a blueprint for how we use AI to reinvent the way we govern ourselves
Labor market impacts of AI: A new measure and early evidence
Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.
How Shifting Responsibility for AI Harms Undermines Democratic Accountability | TechPolicy.Press
The moralization of individual AI use deflects responsibility away from powerful actors like corporations and governments, Suvradip Maitra and others write.
The 2025 AI Agent Index Documenting Technical and Safety Features of Deployed Agentic AI Systems
Agentic AI systems are increasingly capable of performing professional and personal tasks with limited human involvement. However, tracking these developments is difficult because the AI agent ecosystem is complex, rapidly evolving, and inconsistently documented, posing obstacles to both researchers and policymakers. To address these challenges, this paper presents the 2025 AI Agent Index. The Index documents information regarding the origins, design, capabilities, ecosystem, and safety features of 30 state-of-the-art AI agents based on publicly available information and email correspondence with developers. In addition to documenting information about individual agents, the Index illuminates broader trends in the development of agents, their capabilities, and the level of transparency of developers. Notably, we find different transparency levels among agent developers and observe that most developers share little information about safety, evaluations, and societal impacts. The 2025 AI Agent Index is available online at https://aiagentindex.mit.edu.