The System From Nowhere

Debates over AI consciousness are a trap

If AI systems are viewed as too advanced to control, the companies that build them can’t held liable for the harms they cause.

The Unaccountability Machine: Why Big Systems Make Terrible Decisions—and How the World Lost Its Mind

Longlisted for the 2024 Financial Times Book of the Year. How life and the economy became a black box—a collection of systems no one understands, producing outcomes no one likes.   Passengers get bumped from flights. Phone menus disconnect. Automated financial trades produce market collapse. Of all the challenges in modern life, some of the most vexing come from our relationships with automation: a large system does us wrong, and there’s nothing we can do about it. The problem, economist Dan Davies shows, is accountability sinks: systems in which decisions are delegated to a complex rule book or set of standard procedures, making it impossible to identify the source of mistakes when they happen. In our increasingly unhuman world—lives dominated by algorithms, artificial intelligence, and large organizations—these accountability sinks produce more than just aggravation. They make life and economy unknowable—a black box for no reason. In The Unaccountability Machine, Davies lays bare how markets, institutions, and even governments systematically generate outcomes that no one—not even those involved in making them—seems to want. Since the earliest days of the computer age, theorists have foreseen the dangers of complex systems without personal accountability. In response, British business scholar Stafford Beer developed an accountability-first approach to management called “cybernetics,” which might have taken off had his biggest client (the Chilean government) not fallen to a bloody coup in 1973. With his signature blend of economic and journalistic rigor, Davies examines what’s gone wrong since Beer, including what might have been had the world embraced cybernetics when it had the chance. The Unaccountability Machine is a revelatory and resonant account of how modern life became predisposed to dysfunction.

How Shifting Responsibility for AI Harms Undermines Democratic Accountability | TechPolicy.Press

The moralization of individual AI use deflects responsibility away from powerful actors like corporations and governments, Suvradip Maitra and others write.

AI Principles

A guiding framework for our responsible development and use of AI, alongside transparency and accountability in our AI development process.

AI Agent Standards: Navigating New NIST Governance | Nemko Digital

NIST's new AI agent standards are here. Learn what they mean for AI governance, compliance, and liability. Get ahead of the new regulations.

ver.ooo

Curious about systems — how they behave, how they fail, and what they get up to when nobody is steering.

Mindless Machines, Mindless Myths | Los Angeles Review of Books

Erik J. Larson thinks about “Mindless: The Human Condition in the Age of Artificial Intelligence,” which traces Robert Skidelsky’s philosophical reckoning with AI, automation, and the illusion of progress.

Mindless Machines, Mindless Myths | Los Angeles Review of Books

Erik J. Larson thinks about “Mindless: The Human Condition in the Age of Artificial Intelligence,” which traces Robert Skidelsky’s philosophical reckoning with AI, automation, and the illusion of progress.

The Systems Thinker

The Systems Thinker works to catalyze effective change by expanding the use of systems approaches. All articles are available free of charge in an effort to expose as wide of audience as possible. Browse, share with others, save your favorites and tell others about this valuable resource!

Code Was Never the Asset - The Phoenix Architecture

Why AI makes the hidden economics of software unavoidable

AI Slopageddon and the OSS Maintainers

AI slop is ripping up the social contract between maintainers and contributors essential to open source development. Practitioners have been repeatedly assured that AI would supercharge their communities, but so far that hasn’t been the case. Just look at what happened last month. Mitchell Hashimoto’s Ghostty implemented a zero-tolerance policy where submitting bad AI-generated code

Taking AI Welfare Seriously

In this report, we argue that there is a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future. That means that the prospect of AI welfare and moral patienthood, i.e. of AI systems with their own interests and moral significance, is no longer an issue only for sci-fi or the distant future. It is an issue for the near future, and AI companies and other actors have a responsibility to start taking it seriously. We also recommend three early steps that AI companies and other actors can take: They can (1) acknowledge that AI welfare is an important and difficult issue (and ensure that language model outputs do the same), (2) start assessing AI systems for evidence of consciousness and robust agency, and (3) prepare policies and procedures for treating AI systems with an appropriate level of moral concern. To be clear, our argument in this report is not that AI systems definitely are, or will be, conscious, robustly agentic, or otherwise morally significant. Instead, our argument is that there is substantial uncertainty about these possibilities, and so we need to improve our understanding of AI welfare and our ability to make wise decisions about this issue. Otherwise there is a significant risk that we will mishandle decisions about AI welfare, mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not.

A Systems Literacy Manifesto

In 1968, West Churchman wrote, “…there is a good deal of turmoil about the manner in which our society is run. …the citizen has begun to suspect that the people who make major decisions that affect our lives don’t know what they are doing.”[1] Churchman was writing at a time of growing concern about war, civil rights, and the environment. Almost fifty years later, these concerns remain, and we have more reason than ever “to suspect that the people who make major decisions that affect our lives don’t know what they are doing.” Examples abound.

The lethal trifecta for AI agents: private data, untrusted content, and external communication

If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of …

Greyhaven — Sovereign AI Systems for Enterprise

Greyhaven builds custom sovereign AI systems: on-premise inference, private data pipelines, and model-agnostic architecture so enterprises maintain full control over their AI.