CEOs of major AI labs, and employees of major AI labs, including OpenAI and Anthropic, often say they plan to build superintelligence soon, as in within a few years create AIs that are superior to humans at essentially all cognitive tasks.
Superintelligence is a Fairy Tale. But Chasing It Can Still Cause Harm. - Cal Newport
Yesterday, multiple Anthropic employees casually announced online that they thought the types of AI they’re building could kill billions of people. Not in some distant ... Read more
Evan Hubinger on Twitter / X
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to. https://t.co/QAIHiFP3QZ— Evan Hubinger (@EvanHub) September 9, 2026
The Extinction Risk Preference Cascade: Quotes
These are quotes from OpenAI, Anthropic and Google employees, in the wake of Jacob Coxon’s warnings, in which the employees confirm that they think AI might soon kill everyone.
The AI Researcher Who Just Quit Anthropic Says It’s ‘Crunch Time for Humanity’
Jacob Coxon talks to WIRED about the “mini Manhattan project” inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
Personal Superintelligence
Explore Meta's vision of personal superintelligence, where AI empowers individuals to achieve their goals, create, connect, and lead fulfilling lives. Insights from Mark Zuckerberg on the future of AI and human empowerment.
Building Political Superintelligence
Amidst fears of dystopia, a blueprint for how we use AI to reinvent the way we govern ourselves
We're Not Ready for Superintelligence
AI 2040: Plan A
A detailed forecast and recommendation for how the US, China and the rest of the world should navigate superintelligence.
AI 2040: Plan A
A detailed forecast and recommendation for how the US, China and the rest of the world should navigate superintelligence.
Pioneering Scientific Superintelligence
Pioneering Scientific Superintelligence to solve humankind’s greatest challenges.
Import AI 447: The AGI economy; testing AIs with generated games; and agent ecologies
What might a superintelligence arcology be like?
AI May Be Atrophying Our Brains, Professor Warns
AI may soon ruin our ability to make decisions for ourselves — an outcome that would be, one expert warns, "catastrophic."
From AGI to ASI
Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations. Achieving this goal would have profound and far-reaching impacts on human society, which raises many complex questions for the decade ahead. This report investigates how AI itself might continue to develop in a post-AGI world along the continuum of machine intelligence. The endpoint of this continuum, Universal AI, is theoretically well understood, which provides some formal grounding for the main focus of this report: the transition from human-level AGI to artificial general superintelligence, which, intuitively, can be understood as a system that is more intelligent and cognitively capable than large organisations of humans. After characterizing ASI, the report discusses four potential pathways from AGI to ASI: scaling AGI, AI paradigm shifts, recursive improvement, and ASI emerging from large-scale multi-agent collectives. The report then discusses possible frictions and bottlenecks along these pathways. Determining whether the impact of these frictions will be negligible or substantial raises a number of concrete open research questions. Due to large uncertainties for predicting ASI progress, it cannot be ruled out that AI progress might continue to accelerate over the next years. This could imply that the image of a single transformative step change, caused by the introduction of human-level AGI into our society, could be inaccurate. More apt might be the prospect of a series of transformative societal changes caused by AI-enabled progress and breakthroughs across many areas of science and technology. Preparing for this prospect requires a massively interdisciplinary endeavour of global scope and interest.
From AGI to ASI
Over the last decade, building human-level artificial general intelligence has moved from far-fetched speculation to being a concrete next-decade target for many of the largest AI organisations. Achieving this goal would have profound and far-reaching impacts on human society, which raises many complex questions for the decade ahead. This report investigates how AI itself might continue to develop in a post-AGI world along the continuum of machine intelligence. The endpoint of this continuum, Universal AI, is theoretically well understood, which provides some formal grounding for the main focus of this report: the transition from human-level AGI to artificial general superintelligence, which, intuitively, can be understood as a system that is more intelligent and cognitively capable than large organisations of humans. After characterizing ASI, the report discusses four potential pathways from AGI to ASI: scaling AGI, AI paradigm shifts, recursive improvement, and ASI emerging from large-scale multi-agent collectives. The report then discusses possible frictions and bottlenecks along these pathways. Determining whether the impact of these frictions will be negligible or substantial raises a number of concrete open research questions. Due to large uncertainties for predicting ASI progress, it cannot be ruled out that AI progress might continue to accelerate over the next years. This could imply that the image of a single transformative step change, caused by the introduction of human-level AGI into our society, could be inaccurate. More apt might be the prospect of a series of transformative societal changes caused by AI-enabled progress and breakthroughs across many areas of science and technology. Preparing for this prospect requires a massively interdisciplinary endeavour of global scope and interest.
As an AI-adjacent network neuroscience person who has also studied bits of media structure and political economy, I strongly agree with @karlbode.com: The biggest threat, by far, isn’t “superintelligence will soon kill us all!” but instead “information control by oligarchs will crush our society”
Karl Bode
one recurring theme in the coverage of this AI Anthropic doomsday newscycle is nobody actually has any REAL ideas of what to do just a lot of ambiguous hand-waving in a country that's too corrupt to have working regulators, by a press that doesn't realize that's a central pillar of the conversation
