AI safety standards worldwide must keep up with the rapid development and deployment of AI technology. Our mission is to help accelerate the writing of AI safety standards.
Artificial Intelligence Safety and Security License Requirements
Artificial Intelligence Safety and Security License Requirements Artificial Intelligence (AI) is now broadly capable, which can produce both benefits and dangers. AI will hasten the design and proliferation of bioweapons, cyberweapons, nuclear weapons, progressively more general intelligence,...
AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations
The rapid advancement of AI systems has raised widespread concerns about potential harms of frontier AI systems and the need for responsible evaluation and oversight. In this position paper, we argue that frontier AI companies should report both pre- and post-mitigation safety evaluations to enable informed policy decisions. Evaluating models at both stages provides policymakers with essential evidence to regulate deployment, access, and safety standards. We show that relying on either in isolation can create a misleading picture of model safety. Our analysis of AI safety disclosures from leading frontier labs identifies three critical gaps: (1) companies rarely evaluate both pre- and post-mitigation versions, (2) evaluation methods lack standardization, and (3) reported results are often too vague to inform policy. To address these issues, we recommend mandatory disclosure of pre- and post- mitigation capabilities to approved government bodies, standardized evaluation methods, and minimum transparency requirements for public safety reporting. These ensure that policymakers and regulators can craft targeted safety measures, assess deployment risks, and scrutinize companies’ safety claims effectively.
Guidelines on transparency obligations for providers and deployers of certain AI systems
These guidelines help providers and deployers of AI systems and competent authorities in ensuring compliance with the transparency obligations under Article 50 of the AI Act.
Model Hardware Standard
A new standard for AI agents to safely operate physical equipment in scientific research and advanced manufacturing.
Safety isn’t safety without a social model (or: dispelling the myth of per se technical safety) — AI Alignment Forum
As an AI researcher who wants to do technical work that helps humanity, there is a strong drive to find a research area that is definitely helpful so…
An End-to-End View of AI Safety
In this blog, Rachel Coldicutt OBE, Executive Director, Careful Industries discusses our newly published literature review on the safe adoption of artificial intelligence in engineered systems.
Standards Matter More Than Ever in the Age of AI
AI can read anything, but without shared standards it only reinforces PDF-era incentives. The interfaces we build shape what science becomes, and transforming scientific culture requires modular, structured, contextual foundations beneath every interface.
Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Cases
This paper contributes to the nascent debate around safety cases for frontier AI systems. Safety cases are structured, defensible arguments that a system is acceptably safe to deploy in a given context. Historically, they have been used in safety-critical industries, such as aerospace, nuclear or automotive. As a result, safety cases for frontier AI have risen in prominence, both in the safety policies of leading frontier developers and in international research agendas proposed by leaders in generative AI, such as the Singapore Consensus on Global AI Safety Research Priorities and the International AI Safety Report. This paper appraises this work. We note that research conducted within the alignment community which draws explicitly on lessons from the assurance community has significant limitations. We therefore aim to rethink existing approaches to alignment safety cases. We offer lessons from existing methodologies within safety assurance and outline the limitations involved in the alignment community's current approach. Building on this foundation, we present a case study for a safety case focused on Deceptive Alignment and CBRN capabilities, drawing on existing, theoretical safety case "sketches" created by the alignment safety case community. Overall, we contribute holistic insights from the field of safety assurance via rigorous theory and methodologies that have been applied in safety-critical contexts. We do so in order to create a better foundational framework for robust, defensible and useful safety case methodologies which can help to assure the safety of frontier AI systems.
Responsible use of AI in the military? US publishes declaration outlining principles
12 "best practices" for using AI and autonomous systems emphasize human accountability.
The AI safety movement needs normies
A broader base may be the only way for the AI safety field to get what it wants
Action Plan to increase the safety and security of advanced AI
The first U.S. government-commissioned assessment on catastrophic national security risks from advanced AI on the path to AGI.
Responsible AI
Discover how AWS is committing to developing AI responsibly – to built trust, promote the safe development of AI, and act as a force for good.
In the US, Caution Rules on AI Ahead of K-12 School Year
Students and teachers across the country are unlikely to find consistent, clear rules about AI use, reports Tech Policy Press fellow Chris Mills Rodrigo.
AI Principles
A guiding framework for our responsible development and use of AI, alongside transparency and accountability in our AI development process.
The 2025 AI Agent Index Documenting Technical and Safety Features of Deployed Agentic AI Systems
Agentic AI systems are increasingly capable of performing professional and personal tasks with limited human involvement. However, tracking these developments is difficult because the AI agent ecosystem is complex, rapidly evolving, and inconsistently documented, posing obstacles to both researchers and policymakers. To address these challenges, this paper presents the 2025 AI Agent Index. The Index documents information regarding the origins, design, capabilities, ecosystem, and safety features of 30 state-of-the-art AI agents based on publicly available information and email correspondence with developers. In addition to documenting information about individual agents, the Index illuminates broader trends in the development of agents, their capabilities, and the level of transparency of developers. Notably, we find different transparency levels among agent developers and observe that most developers share little information about safety, evaluations, and societal impacts. The 2025 AI Agent Index is available online at https://aiagentindex.mit.edu.