The System From Nowhere

Debates over AI consciousness are a trap

If AI systems are viewed as too advanced to control, the companies that build them can’t held liable for the harms they cause.

The Unaccountability Machine: Why Big Systems Make Terrible Decisions—and How the World Lost Its Mind

Longlisted for the 2024 Financial Times Book of the Year. How life and the economy became a black box—a collection of systems no one understands, producing outcomes no one likes.   Passengers get bumped from flights. Phone menus disconnect. Automated financial trades produce market collapse. Of all the challenges in modern life, some of the most vexing come from our relationships with automation: a large system does us wrong, and there’s nothing we can do about it. The problem, economist Dan Davies shows, is accountability sinks: systems in which decisions are delegated to a complex rule book or set of standard procedures, making it impossible to identify the source of mistakes when they happen. In our increasingly unhuman world—lives dominated by algorithms, artificial intelligence, and large organizations—these accountability sinks produce more than just aggravation. They make life and economy unknowable—a black box for no reason. In The Unaccountability Machine, Davies lays bare how markets, institutions, and even governments systematically generate outcomes that no one—not even those involved in making them—seems to want. Since the earliest days of the computer age, theorists have foreseen the dangers of complex systems without personal accountability. In response, British business scholar Stafford Beer developed an accountability-first approach to management called “cybernetics,” which might have taken off had his biggest client (the Chilean government) not fallen to a bloody coup in 1973. With his signature blend of economic and journalistic rigor, Davies examines what’s gone wrong since Beer, including what might have been had the world embraced cybernetics when it had the chance. The Unaccountability Machine is a revelatory and resonant account of how modern life became predisposed to dysfunction.

How Shifting Responsibility for AI Harms Undermines Democratic Accountability | TechPolicy.Press

The moralization of individual AI use deflects responsibility away from powerful actors like corporations and governments, Suvradip Maitra and others write.

The Unaccountability Machine, by Dan Davies

I read an online excerpt of this book and was immediately intrigued by the idea of an “accountability sink”, which is a mechanism by which “The communication between the decision-maker and the decided-upon has been broken – they have created a handy sink into which negative feedback can be poured without any danger of it affecting anything.” This lack of accountability where people shrug their shoulders and ask “What else could we do?” really frustrates me, as I believe that we should take responsibility for our actions and their consequences. So I bought the book and inhaled it in a few days.

The Real Existential Risk Is AI CEOs Avoiding Oversight: “Alternative Nobel” Winner Timnit Gebru

We speak to the prominent AI researcher and leading critic of the AI industry, Timnit Gebru, who was recently awarded a 2026 Right Livelihood Award for her work investigating the harms and hype behind the global buildup of artificial intelligence. “Instead of discussing faulty, unreliable systems,” Gebru says tech leaders like SpaceXAI’s Elon Musk, OpenAI’s Sam Altman and Anthropic’s Dario Amodei are “marketing their systems as all-knowing, superpowerful, superintelligent,” and lobbying for “self-policing and self-regulation” to avoid being held “liable through existing laws.” Gebru expresses skepticism over increasingly widespread claims that AI algorithms pose an existential risk to humanity, warning that such claims are overblown and that “what is actually happening is regulatory capture.”

AI Principles

A guiding framework for our responsible development and use of AI, alongside transparency and accountability in our AI development process.

AI Agent Standards: Navigating New NIST Governance | Nemko Digital

NIST's new AI agent standards are here. Learn what they mean for AI governance, compliance, and liability. Get ahead of the new regulations.

ver.ooo

Curious about systems — how they behave, how they fail, and what they get up to when nobody is steering.

Mindless Machines, Mindless Myths | Los Angeles Review of Books

Erik J. Larson thinks about “Mindless: The Human Condition in the Age of Artificial Intelligence,” which traces Robert Skidelsky’s philosophical reckoning with AI, automation, and the illusion of progress.

Mindless Machines, Mindless Myths | Los Angeles Review of Books

Erik J. Larson thinks about “Mindless: The Human Condition in the Age of Artificial Intelligence,” which traces Robert Skidelsky’s philosophical reckoning with AI, automation, and the illusion of progress.

The Systems Thinker

The Systems Thinker works to catalyze effective change by expanding the use of systems approaches. All articles are available free of charge in an effort to expose as wide of audience as possible. Browse, share with others, save your favorites and tell others about this valuable resource!

Code Was Never the Asset - The Phoenix Architecture

Why AI makes the hidden economics of software unavoidable

AI Slopageddon and the OSS Maintainers

AI slop is ripping up the social contract between maintainers and contributors essential to open source development. Practitioners have been repeatedly assured that AI would supercharge their communities, but so far that hasn’t been the case. Just look at what happened last month. Mitchell Hashimoto’s Ghostty implemented a zero-tolerance policy where submitting bad AI-generated code

Taking AI Welfare Seriously

In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood, i.e. of AI systems with their own interests and moral significance, is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that language model outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are, or will be, conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.