The Locally Optimal Discursive Posture — LessWrong

Amplifiers of Epistemic Posture

Essays and writing on AI

A big-tent or small-tent AI safety movement?

The unstated disagreement that underpins safety debates

Safety isn’t safety without a social model (or: dispelling the myth of per se technical safety) — AI Alignment Forum

As an AI researcher who wants to do technical work that helps humanity, there is a strong drive to find a research area that is definitely helpful so…

The AI safety movement needs normies

A broader base may be the only way for the AI safety field to get what it wants

The assistant axis: situating and stabilizing the character of large language models

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

Curriculum | AI Safety — ARENA

Explore the ARENA curriculum for AI safety education, covering fundamentals, transformers, reinforcement learning, and evaluations, with resources for educators and learners.

A new writing series: Re-envisioning AI safety through global majority perspectives | Brookings

Chinasa T. Okolo previews a new writing series that will re-envision AI safety through global majority perspectives.

Cognitive Surrender and Tri-System Theory · Today I Learned

How AI can support, displace, or bypass human deliberation

Safety cases at AISI | AISI Work

As a complement to our empirical evaluations of frontier AI models, AISI is planning a series of collaborations and research projects sketching safety cases for more advanced models than exist today, focusing on risks from loss of control and autonomy. By a safety case, we mean a structured argument that an AI system is safe within a particular training or deployment context.

Clear, Compelling Arguments: Rethinking the Foundations of Frontier AI Safety Cases

This paper contributes to the nascent debate around safety cases for frontier AI systems. Safety cases are structured, defensible arguments that a system is acceptably safe to deploy in a given context. Historically, they have been used in safety-critical industries, such as aerospace, nuclear or automotive. As a result, safety cases for frontier AI have risen in prominence, both in the safety policies of leading frontier developers and in international research agendas proposed by leaders in generative AI, such as the Singapore Consensus on Global AI Safety Research Priorities and the International AI Safety Report. This paper appraises this work. We note that research conducted within the alignment community which draws explicitly on lessons from the assurance community has significant limitations. We therefore aim to rethink existing approaches to alignment safety cases. We offer lessons from existing methodologies within safety assurance and outline the limitations involved in the alignment community's current approach. Building on this foundation, we present a case study for a safety case focused on Deceptive Alignment and CBRN capabilities, drawing on existing, theoretical safety case "sketches" created by the alignment safety case community. Overall, we contribute holistic insights from the field of safety assurance via rigorous theory and methodologies that have been applied in safety-critical contexts. We do so in order to create a better foundational framework for robust, defensible and useful safety case methodologies which can help to assure the safety of frontier AI systems.

More Questions About AI Dangers, With Pointers From Gary Marcus

Following up on my previous post, I want to point your attention...

Positive Alignment: Artificial Intelligence for Human Flourishing

Existing alignment research is dominated by concerns about safety and preventing harm: safeguards, controllability, and compliance. This paradigm of alignment parallels early psychology's focus on...

A case for AI taboos – Writings and rehearsals by Nathan Schneider

We probably all know the person, and maybe it is ourselves, who has inundated themselves in AI use, and say they are fully aware of all the dangers, but are so, so careful as to not fall into such traps themselves.

ADD / XOR / ROL: A non-anthropomorphized view of LLMs

In many discussions where questions of "alignment" or "AI safety" crop up, I am baffled by seriously intelligent people imbuing almost magic...