The proliferation of advanced AI systems may lead to the gradual disempowerment of humanity, even if efforts to prevent them from becoming power-seeking or scheming are successful. Humanity may be incentivised to hand over increasing amounts of control to AIs, giving them power over the economy, politics, culture, and more. Over time, humanity's interests may be sidelined and our control over the future undermined, potentially constituting an existential catastrophe. There's disagreement over how serious a problem this is and how it relates to [other concerns about AI alignment](https://80000hours.o
Gradual Disempowerment: Systemic Existential Risks from...
This paper examines the systemic risks posed by incremental advancements in artificial intelligence, developing the concept of `gradual disempowerment', in contrast to the abrupt takeover...
Gradual Disempowerment: Systemic Existential Risks from...
This paper examines the systemic risks posed by incremental advancements in artificial intelligence, developing the concept of `gradual disempowerment', in contrast to the abrupt takeover...
The Scaling Era: An Oral History of AI, 2019–2025
An inside view of the AI revolution, from the people an…
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
The AI Researcher Who Just Quit Anthropic Says It’s ‘Crunch Time for Humanity’
Jacob Coxon talks to WIRED about the “mini Manhattan project” inside Anthropic, the problem with alignment, and why AI labs have just a few years left to make their systems safe.
The Scaling Era: An Oral History of AI, 2019–2025
An inside view of the AI revolution, from the people and companies making it happen.
3: Consulting the Record: AI Consistently Fails the Public - AI Now Institute
Amid the excitement over AI’s (speculative, hypothetical) potential, we have lost sight of the sobering reality of its present and recent past. AI is already intermediating critical social infrastructures, materially reshaping our institutions in ways that ratchet up inequality and concentrate power in the hands of the already powerful. It is consistently deployed in ways […]
AI FOR EPISTEMICS & COORDINATION
Civilization and technology have radically improved the human condition. Nonetheless, the world sometimes goes in directions which essentially nobody would prefer — e.g., nuclear arms races, unexpected financial crashes, predatory marketing, or ubiquitous political misinformation.
Agentic AI and the next intelligence explosion
For decades, the artificial intelligence (AI) “singularity” has been heralded as a single, titanic mind bootstrapping itself to godlike intelligence, consolidating all cognition into a cold silicon point. But this vision is almost certainly wrong in its most fundamental assumption. If AI development follows the path of previous major evolutionary transitions or “intelligence explosions,” our current step-change in computational intelligence will be plural, social, and deeply entangled with its forebears (us!).
Reverse Alignment
We are building AI faster than we are building the society to receive it. Reverse alignment is the deliberate redesign of institutions, norms, skills, and governance so societies can absorb AI safely and share its gains widely.
Building Political Superintelligence
Amidst fears of dystopia, a blueprint for how we use AI to reinvent the way we govern ourselves
The Impact of Artificial Intelligence on Human Thought
This research paper examines, from a multidimensional perspective (cognitive, social, ethical, and philosophical), how AI is transforming human thought. It highlights a cognitive offloading effect: the externalization of mental functions to AI can reduce intellectual engagement and weaken critical thinking. On the social level, algorithmic personalization creates filter bubbles that limit the diversity of opinions and can lead to the homogenization of thought and polarization. This research also describes the mechanisms of algorithmic manipulation (exploitation of cognitive biases, automated disinformation, etc.) that amplify AI's power of influence. Finally, the question of potential artificial consciousness is discussed, along with its ethical implications. The report as a whole underscores the risks that AI poses to human intellectual autonomy and creativity, while proposing avenues (education, transparency, governance) to align AI development with the interests of humanity.
What Does 'Human-Centred AI' Mean?
While it seems sensible that human-centred artificial intelligence (AI) means centring "human behaviour and experience," it cannot be any other way. AI, I argue, is usefully seen as a relationship between technology and humans where it appears that artifacts can perform, to a greater or lesser extent, human cognitive labour. This is evinced using examples that juxtapose technology with cognition, inter alia: abacus versus mental arithmetic; alarm clock versus knocker-upper; camera versus vision; and sweatshop versus tailor. Using novel definitions and analyses, sociotechnical relationships can be analysed into varying types of: displacement (harmful), enhancement (beneficial), and/or replacement (neutral) of human cognitive labour. Ultimately, all AI implicates human cognition; no matter what. Obfuscation of cognition in the AI context -- from clocks to artificial neural networks -- results in distortion, in slowing critical engagement, perverting cognitive science, and indeed in limiting our ability to truly centre humans and humanity in the engineering of AI systems. To even begin to de-fetishise AI, we must look the human-in-the-loop in the eyes.
What Does 'Human-Centred AI' Mean?
While it seems sensible that human-centred artificial intelligence (AI) means centring "human behaviour and experience," it cannot be any other way. AI, I argue, is usefully seen as a relationship between technology and humans where it appears that artifacts can perform, to a greater or lesser extent, human cognitive labour. This is evinced using examples that juxtapose technology with cognition, inter alia: abacus versus mental arithmetic; alarm clock versus knocker-upper; camera versus vision; and sweatshop versus tailor. Using novel definitions and analyses, sociotechnical relationships can be analysed into varying types of: displacement (harmful), enhancement (beneficial), and/or replacement (neutral) of human cognitive labour. Ultimately, all AI implicates human cognition; no matter what. Obfuscation of cognition in the AI context -- from clocks to artificial neural networks -- results in distortion, in slowing critical engagement, perverting cognitive science, and indeed in limiting our ability to truly centre humans and humanity in the engineering of AI systems. To even begin to de-fetishise AI, we must look the human-in-the-loop in the eyes.
The Age of Wonders and Terrors
Twenty years ago, when the idea of AI taking over the world in our lifetimes still struck most of us as the unconstrained fantasy of those who knew too much science fiction and too little science, …
