What To Fear from AI Agents and How to Reclaim Our Human Agency
We Need to Talk About How We Talk About 'AI'
We share a responsibility to create and use empowering metaphors rather than misleading language, write Emily M. Bender and Nanna Inie.
AI agents pose untold risk to humanity. We must act to prevent that future | David Krueger
The pieces are falling into place for autonomous artificial intelligence. We must stop unregulated development
AI is not superhuman
What metaphor should drive the field of AI research?
AI Epistemic Risks: Emerging Mechanisms & Evidence
<p>Advances in artificial intelligence pose risks to humanity's collective capacity to form accurate beliefs, reason well, and maintain a healthy information en
AI Safety Is a Narrative Problem · Special Issue 5: Grappling With the Generative AI Revolution
This op-ed explores power and narrative dynamics around AI. Drawing on pop-culture references, the professional experiences of the author and examples from 2023’s “Great AI Safety Hype Roadshow,” this piece draws on the literary criticism technique of practical criticism to consider how speeches and announcements from both Silicon Valley executives and research scientists to interrogate the media-friendly nature of p(doom) discourse—which focuses on the existential risks of AI (PauseAI, 2023)—and its likely consequences. The complexities of AI and its numerous social impacts can be difficult for even the most expert analyst to unpack. In spite of this, the potential of “existential threats” has successfully cut through to become a mainstay of mainstream media coverage over the last year. This piece will make the case that this is an effective narrative conceit that has achieved a number of ends that traditional science communication tends to find difficult, if not impossible, to achieve. Firstly, it is easy to understand. Simplification of this nature—that removes jargon and complexity and focuses on a single outcome—is much easier to fit on a TV rolling news ticker or on the cover of a tabloid newspaper than more well-balanced, representative opinions. Secondly, it inherits prior assumptions from well-known dramatic forms. P(doom) plays to stories familiar from Greek tragedy through to Marvel movies, in which lone male heroes battle ineluctable forces. Thirdly, it is imbued with urgency and so becomes difficult to ignore.
The lethal trifecta for AI agents: private data, untrusted content, and external communication
If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of …
Taking AI Welfare Seriously
In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood, i.e. of AI systems with their own interests and moral significance, is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that language model outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are, or will be, conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.
How Shifting Responsibility for AI Harms Undermines Democratic Accountability | TechPolicy.Press
The moralization of individual AI use deflects responsibility away from powerful actors like corporations and governments, Suvradip Maitra and others write.
Trusting your own judgement on ‘AI’ is a huge risk
Writing at the end of the world, from Hveragerði, Iceland
Why We Fear AI: On the Interpretation of Nightmares — Common Notions Press
Industry insiders Hagen Blix and Ingeborg Glimmer dive into the dark, twisted world of AI to demystify the many nightmares we have about it. They combine expertise in cognitive science and machine learning with political and economic analyses to cut through the hype and technobabble to show how fear
The Impact of Artificial Intelligence on Human Thought
This research paper examines, from a multidimensional perspective (cognitive, social, ethical, and philosophical), how AI is transforming human thought. It highlights a cognitive offloading effect: the externalization of mental functions to AI can reduce intellectual engagement and weaken critical thinking. On the social level, algorithmic personalization creates filter bubbles that limit the diversity of opinions and can lead to the homogenization of thought and polarization. This research also describes the mechanisms of algorithmic manipulation (exploitation of cognitive biases, automated disinformation, etc.) that amplify AI's power of influence. Finally, the question of potential artificial consciousness is discussed, along with its ethical implications. The report as a whole underscores the risks that AI poses to human intellectual autonomy and creativity, while proposing avenues (education, transparency, governance) to align AI development with the interests of humanity.
Many Minds: Seven metaphors for AI
If you wanted a petri dish for understanding metaphors—how they emerge and evolve and jostle with each other—it would be hard to do better than the world of AI. We talk about AI systems variously as coaches or co-pilots, little genies or alien intelligences. Some researchers claim that AIs "grow," that they're entering their phase of "adolescence." Critics deride AI products as slop and dismiss LLMs as a kind of autocomplete on steroids. What's behind these different characterizations? Which ones are accurate and which are unfair? And are our metaphors mostly colorful rhetoric or do they matter? Are they shaping how we understand, adopt, and ultimately regulate these new technologies? My guest today is . Melanie is a computer scientist and Professor at the Santa Fe Institute. She is the author of the book, and she writes a by the same name. This episode is a bit of a companion to with Steve Flusberg. In that episode, Steve and I attempted a kind of crash course on metaphor and the human mind. Here, Melanie and I sit down for more of an extended case study: how metaphors are guiding, galvanizing, and maybe deceiving us in the contested realm of AI discourse. We unpack seven of the most widely used metaphors in this space. We consider how these metaphors are shaping not only our everyday understandings of AI, but also law and policy. We also talk about the metaphor and analogy capabilities of AI itself. Can these systems reason abstractly in the way that humans can? Along the way, Melanie and I touch on: AI-generated poetry, anthropomorphism, the original sin of AI research, the myth of Narcissus, psychometric testing and its pitfalls, metaphors for AI that are a bit hard to spot, and the question of whether an AI has ever come up with a decent analogy for itself. Longtime fans of the show will know that we've had Melanie on the show . We invited her back, not only because she's thought about metaphor and analogy in AI discourse for decades, but because she's a voice of calm insight in an area that’s increasingly awash in hype and polemic. Longtime fans of the show may also note that we are now celebrating our 6th birthday at Many Minds. That's right, the show launched in February 2020. If you'd like to support us as we recognize this milestone, you can leave us a rating or a review, recommend us to a friend, or give us a shout out on social media. Your support is always appreciated. Without further ado, on to my conversation with Dr. Melanie Mitchell. Enjoy! Notes 3:30 – For an overview of Douglas Hofstadter’s work on analogy, see . 8:00 – Much of our discussion in this interview draws on Dr. Mitchell’s piece on the in Science magazine. 13:30 – For earlier discussions of anthropomorphism on the show, see our earlier episodes and . 16:00 – See for the original discussion of LLMs as “stochastic parrots.” 17:00 – See for the original discussion of ChatGPT as a “blurry jpeg.” 18:30 – See for the original discussion of LLMs as role players. 22:00 – See for one use of the “LLMs as crowds” metaphor. See also a discussion of this metaphor (and other metaphors for AI) . 25:00 – For one discussion of AI as a “cultural technology” by Alison Gopnik and colleagues, see . For a more recent discussion of the same metaphor by Henry Farrell, Alison Gopnik and others, see . 27:00 – For the podcast series on intelligence that Dr. Mitchell co-hosted for the Santa Fe Institute, see . 28:00 – See for an influential formulation of the idea that AI is an “alien intelligence.” 29:00 – For philosopher Shannon Vallor’s book about AI as “mirror,” see . 31:00 – For the recent study on users’ metaphors for AI systems, see . 33:00 – For more on the rise of social AI, see our earlier episode . 38:00 – For more on what AI researchers might learn from developmental and comparative psychologists, see Dr. Mitchell’s (summarizing her keynote at NeurIPs). 42:00 – For more on the ARC (Abstraction and Reasoning Corpus) and the research that Dr. Mitchell and colleagues have been doing with it, see and . 48:30 – For the study on humans' preference for AI-generated poetry, see . 50:30 – For Brigitte Nerlich’s documentation and discussion of various metaphors for AI (including AI’s metaphors for itself), see . Recommendations , by Shannon Vallor ‘,’ by Murray Shanahan (!) et al. ‘,’ by Henry Farrell et al. Many Minds is a project of the , which is made possible by a generous grant from the John Templeton Foundation to Indiana University. The show is hosted and produced by , with help from Assistant Producer and with creative support from DISI Directors Erica Cartmill and Jacob Foster. Our artwork is by . Subscribe to Many Minds on Apple, Stitcher, Spotify, Pocket Casts, Google Play, or wherever you listen to podcasts. You can also now subscribe to the Many Minds newsletter ! We welcome your comments, questions, and suggestions. Feel free to email us at: manymindspodcast@gmail.com. For updates about the show, visit or follow us on Bluesky ().
The AI safety vibe shift
Once a fringe obsession of Bay Area rationalists, existential risk is suddenly all anyone is talking about
