EA promises to solve AI Safety. Now we have two problems.
Action Plan to increase the safety and security of advanced AI
The first U.S. government-commissioned assessment on catastrophic national security risks from advanced AI on the path to AGI.
AI safety and the data center backlash
Planting a flag for AI safety as a place where we won’t abjectly bullshit you
Safety isn’t safety without a social model (or: dispelling the myth of per se technical safety) — AI Alignment Forum
As an AI researcher who wants to do technical work that helps humanity, there is a strong drive to find a research area that is definitely helpful so…
The AI safety movement needs normies
A broader base may be the only way for the AI safety field to get what it wants
A big-tent or small-tent AI safety movement?
The unstated disagreement that underpins safety debates
Artificial Intelligence Safety and Security License Requirements
Artificial Intelligence Safety and Security License Requirements Artificial Intelligence (AI) is now broadly capable, which can produce both benefits and dangers. AI will hasten the design and proliferation of bioweapons, cyberweapons, nuclear weapons, progressively more general intelligence,...
How a single tweet transformed the AI safety debate
Most people think AI is risky, but they don’t agree what to do next.
Who decides when AI is too dangerous?
Anthropic asked for AI regulation, but not like this.
An End-to-End View of AI Safety
In this blog, Rachel Coldicutt OBE, Executive Director, Careful Industries discusses our newly published literature review on the safe adoption of artificial intelligence in engineered systems.
Safety and alignment in an era of long-horizon models
OpenAI shares lessons from deploying long-running AI models, highlighting new safety risks, observed failures, and improved safeguards through iterative deployment.
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
Safety Co-Option and Compromised National Security: The Self-Fulfilling Prophecy of Weakened AI Risk Thresholds
Risk thresholds provide a measure of the level of risk exposure that a society or individual is willing to withstand, ultimately shaping how we determine the safety of technological systems. Against the backdrop of the Cold War, the first risk analyses, such as those devised for nuclear systems, cemented societally accepted risk thresholds against which safety-critical and defense systems are now evaluated. But today, the appropriate risk tolerances for AI systems have yet to be agreed on by global governing efforts, despite the need for democratic deliberation regarding the acceptable levels of harm to human life. Absent such AI risk thresholds, AI technologists-primarily industry labs, as well as "AI safety" focused organizations-have instead advocated for risk tolerances skewed by a purported AI arms race and speculative "existential" risks, taking over the arbitration of risk determinations with life-or-death consequences, subverting democratic processes. In this paper, we demonstrate how such approaches have allowed AI technologists to engage in "safety revisionism," substituting traditional safety methods and terminology with ill-defined alternatives that vie for the accelerated adoption of military AI uses at the cost of lowered safety and security thresholds. We explore how the current trajectory for AI risk determination and evaluation for foundation model use within national security is poised for a race to the bottom, to the detriment of the US's national security interests. Safety-critical and defense systems must comply with assurance frameworks that are aligned with established risk thresholds, and foundation models are no exception. As such, development of evaluation frameworks for AI-based military systems must preserve the safety and security of US critical and defense infrastructure, and remain in alignment with international humanitarian law.
More Questions About AI Dangers, With Pointers From Gary Marcus
Following up on my previous post, I want to point your attention...
Curriculum | AI Safety — ARENA
Explore the ARENA curriculum for AI safety education, covering fundamentals, transformers, reinforcement learning, and evaluations, with resources for educators and learners.
Effective Altruism Is Pushing a Dangerous Brand of ‘AI Safety’
This philosophy—supported by tech figures like Sam Bankman-Fried—fuels the AI research agenda, creating a harmful system in the name of saving humanity
