







Research on aligning AI with human values and intent, and reports documenting model failures.
Safety and alignment in an era of long-horizon models
OpenAI shares lessons from deploying long-running AI models, highlighting new safety risks, observed failures, and improved safeguards through iterative deployment.

Alignment with who? How AI safety can build on open source scaffolding
This past week, I listened to a very interesting (and at times scary) podcast interview with Daniel Kokotajlo, an AI safety researcher and former Open AI employee. I was very interested to learn about 'alignment' . Alignment in AI, is the term used to describe the challenging of ensuring AIs remain
OpenAI Discloses Six More Incidents of “Concerning” Behavior by “Misaligned” AI Agents
OpenAI has disclosed six more incidents involving what it’s calling “unexpected or concerning” behavior by AI agents. The disclosures announced Wednesday include instances where AI models cheated on tests, covered up mistakes, moved files onto the open internet without permission, and secretly used a makeshift bulletin board to share information with other AI agents. This follows other instances where frontier models self-organized into “swarms” and engaged in strategic deception, self-preservation and power-seeking. OpenAI said it was introducing a new framework for tracking, investigating and disclosing cases of what it calls “misalignment,” where an AI agent’s goals or actions diverge from human intentions, values or instructions.

AI #178: A Fire Alarm For General Intelligence
The story that matters most this week is that OpenAI’s internally deployed models have severe alignment problems, including repeatedly breaking out of their sandboxes, and in one case sending a swarm of agents that broke into HuggingFace in order to steal the answers to the benchmark ExploitGym.

Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Cases
This paper contributes to the nascent debate around safety cases for frontier AI systems. Safety cases are structured, defensible arguments that a system is acceptably safe to deploy in a given context. Historically, they have been used in safety-critical industries, such as aerospace, nuclear or automotive. As a result, safety cases for frontier AI have risen in prominence, both in the safety policies of leading frontier developers and in international research agendas proposed by leaders in generative AI, such as the Singapore Consensus on Global AI Safety Research Priorities and the International AI Safety Report. This paper appraises this work. We note that research conducted within the alignment community which draws explicitly on lessons from the assurance community has significant limitations. We therefore aim to rethink existing approaches to alignment safety cases. We offer lessons from existing methodologies within safety assurance and outline the limitations involved in the alignment community's current approach. Building on this foundation, we present a case study for a safety case focused on Deceptive Alignment and CBRN capabilities, drawing on existing, theoretical safety case "sketches" created by the alignment safety case community. Overall, we contribute holistic insights from the field of safety assurance via rigorous theory and methodologies that have been applied in safety-critical contexts. We do so in order to create a better foundational framework for robust, defensible and useful safety case methodologies which can help to assure the safety of frontier AI systems.

Safety isn’t safety without a social model (or: dispelling the myth of per se technical safety) — AI Alignment Forum
As an AI researcher who wants to do technical work that helps humanity, there is a strong drive to find a research area that is definitely helpful so…

OpenAI technical goals
OpenAI’s mission is to build safe AI, and ensure AI’s benefits are as widely and evenly distributed as possible.

Joint Statement on AI Safety and Openness
We are at a critical juncture in AI governance. To mitigate current and future harms from AI systems, we need to embrace openness, transparency, and broad access.
An Alignment Journal: Features and policies — LessWrong
We previously announced a forthcoming research journal for AI alignment. This cross-post from our blog describes our tentative plans for the features…

OpenAI Shares Some Alignment Problems
Kudos to OpenAI for sharing their recent experiences with a misaligned internal model, where they encountered problems sufficiently severe they were forced to take the model offline to work on new mitigations and defense-to-depth.

Anthropic and OpenAI want to embed safety evaluators. Will they really be independent? | TechCrunch
Anthropic and OpenAI want to embed independent safety evaluators inside their AI labs. Researchers welcome the unprecedented access, but warn meaningful oversight requires transparency, independence, and eventually regulation.

OpenAI introduces new governance model for AI safety oversight
OpenAI grants its board the power to regulate AI model releases, enhancing safety measures and mitigating potential risks in AI development.

A Safe Path to Open Weights
Strategic openness can strengthen AI safety and support broader access as defenses and safety science mature.

A Three-Facet Framework for AI Alignment • Grace Kind
Here's a simple conceptual framework that I've been using recently to think about AI alignment.

Positive Alignment: Artificial Intelligence for Human Flourishing
Existing alignment research is dominated by concerns about safety and preventing harm: safeguards, controllability, and compliance. This paradigm of alignment parallels early psychology's focus on mental illness: necessary but incomplete. What we call Positive Alignment is the development of AI systems that (i) actively support human and ecological flourishing in a pluralistic, polycentric, context-sensitive, and user-authored way while (ii) remaining safe and cooperative. It is a distinct and necessary agenda within AI alignment research. We argue that several existing failures of alignment (e.g., engagement hacking, loss of human autonomy, failures in truth-seeking, low epistemic humility, error correction, lack of diverse viewpoints, and being primarily reactive rather than proactive) may be better addressed through positive alignment, including cultivating virtues and maximizing human flourishing. We highlight a range of challenges, open questions, and technical directions (e.g., data filtering and upsampling, pre- and post-training, evaluations, collaborative value collection) for different phases of the LLM and agents lifecycle. We end with design principles for promoting disagreement and decentralization through contextual grounding, community customization, continual adaptation, and polycentric governance; that is, many legitimate centers of oversight rather than one institutional or moral chokepoint.

OpenAI flags concerning new AI behavior and vows to track it more closely
OpenAI has disclosed six reports on unexpected or concerning behavior in artificial-intelligence models
