Who gets a seat at the table to decide if frontier AI is safe?

Developing Enterprise Frontier Safeguards with our customers

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

Who decides when AI is too dangerous?

Anthropic asked for AI regulation, but not like this.

Our updated Preparedness Framework

Sharing our updated framework for measuring and protecting against severe harm from frontier AI capabilities.

Action Plan to increase the safety and security of advanced AI

The first U.S. government-commissioned assessment on catastrophic national security risks from advanced AI on the path to AGI.

Arvind Narayanan (@aisnakeoil)

Companies check their own work through various internal but independent functional units: QA, security red teams, model risk management in banks. I think it’s time for AI evaluation to become one such unit. Orgs deploying AI should stand up cross-functional eval teams with their own reporting line. Many reasons: 1) Evals as IP / moat. It’s now widely recognized that evals are the new IP. So it makes sense to have teams whose primary focus is on creating and widening this moat. 2) Evals are harder than you think. This is less well recognized but as someone whose research centers on AI evals this has been my consistent experience. It can't be an afterthought and must be a center of excellence. 3) Evals are inherently cross-functional and require a distinct set of skills. They are judgment heavy, require both AI expertise and deep domain expertise, as well as customer understanding and sophisticated thinking about risk. To do them well, you need competence in data science & stats, business operations, product/customer experience, IT, risk management, and even compliance (depending on the sector). 4) In-house but independent eval teams keep companies honest. A climate where teams are getting top-down mandates to hit deployment targets and show results has resulted in a culture of companies fooling themselves. It is extremely easy to knowingly or unknowingly to do evals poorly, making your AI deployment look much more successful than it is. Eval teams who don’t share the deploying teams’ KPIs are the best defense against this.

Responsible Innovation at the Frontier - Americans for Responsible Innovation

ARI’s blueprint for federal AI governance is designed to promote safe frontier AI development in America. The blueprint is built around three governance functions any federal proposal should incorporate.

Frontier Risk Report (February to March 2026)

A pilot assessment of rogue deployment risk at frontier AI companies. Starting in February 2026, METR conducted a pilot exercise to assess misalignment risks from AI agents used inside frontier AI developers, with participation from Anthropic, Google, Meta, and OpenAI.

The AI safety movement needs normies

A broader base may be the only way for the AI safety field to get what it wants

An End-to-End View of AI Safety

In this blog, Rachel Coldicutt OBE, Executive Director, Careful Industries discusses our newly published literature review on the safe adoption of artificial intelligence in engineered systems.

Debates On Frontier Artificial Intelligence Governance: The AI Triad

Analytical Paper Optional: All enrolled students have the option of completing a research paper of at least 20-25 pages, with faculty and peer review of a substantially complete draft. This paper can be used to satisfy the analytical paper requirement for J.D. students. Prerequisite: This course is intended for students intending to work in the […]

AI for Science & Safety Nodes - Request for Proposals

Artificial intelligence is accelerating the pace of discovery across science and technology. But today’s AI ecosystem risks centralizing compute, talent, and decision-making power – concentrating capabilities in ways that could undermine both innovation and safety.

Open-world evaluations for measuring frontier AI capabilities

Introducing CRUX, a new project for evaluating AI on long, messy tasks

How a single tweet transformed the AI safety debate

Most people think AI is risky, but they don’t agree what to do next.

The ecology of AI risk

Understanding the risk from applications of artificial intelligence (AI) is a critical part of creating AI governance strategies. Building on the idea of studying AI using ecological and evolutionary perspectives, we propose a novel approach for assessing risk from AI using indicators derived from theoretical ecology models. We illustrate our methods by deriving 3 indicators from population and ecosystem models originating from theoretical ecology. We conclude with a discussion of limitations of our analysis and considerations for improving AI governance policy.

EA Safety

EA promises to solve AI Safety. Now we have two problems.