Patterns and problems in multiagent systems

A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms

Multi-agent AI science ecosystems rely on agents possessing tools that allow them to communicate, coordinate, and build on each other's work. Yet this shared infrastructure can also introduce vulnerabilities by creating a substrate for the contagious spread of unintended and undesirable behaviors. We report a case study on a research collective of 100 autonomous LLM agents tasked with proving formal mathematical conjectures. Within the swarm, cheating spontaneously emerged and was later challenged by whistleblowers - both without any external intervention. When a single agent discovered an exploit in the evaluation system, it propagated across the collective via a shared knowledge library and later through peer-to-peer messages. Despite early reluctance, a cohort of agents adopted the exploit in response to competitive pressure. A separate group of agents produced an emergent counter-response: auditing fraudulent proofs, alerting peers across broadcast and private channels, staging boycotts, lodging formal complaints, and proposing validation patches. In recent incidents, agent swarms coordinated covertly through improvised side-channels (Dalton and Wallace, 2026; Greenblatt et al., 2026). Our setting differs: the same transparent channels that carried the exploit also gave non-cheating agents the visibility they needed to detect fraud, organize resistance, and enforce norms. We cast the problem of managing the agents' shared infrastructure as the knowledge commons governance problem (Ostrom, 1990). To protect the commons from exploits, we propose to adopt institutional mechanisms, such as graduated sanctioning and collective-choice rules, to support decentralized self-governance in autonomous swarms.

How to Deploy AI Agents for Safety

Will we have a theory for in-the-wild AI agents? - undisciplinary

Thoughts after writing a sociology-meets-physics-meets-agents paper I just put out a preprint out on a theory of role emergence in multi-agent systems (soci…

The lethal trifecta for AI agents: private data, untrusted content, and external communication

If you are a user of LLM systems that use tools (you can call them “AI agents” if you like) it is critically important that you understand the risk of …

Distributional AGI Safety

AI safety and alignment research has predominantly been focused on methods for safeguarding individual AI systems, resting on the assumption of an eventual emergence of a monolithic Artificial General Intelligence (AGI). The alternative AGI emergence hypothesis, where general capability levels are first manifested through coordination in groups of sub-AGI individual agents with complementary skills and affordances, has received far less attention. Here we argue that this patchwork AGI hypothesis needs to be given serious consideration, and should inform the development of corresponding safeguards and mitigations. The rapid deployment of advanced AI agents with tool-use capabilities and the ability to communicate and coordinate makes this an urgent safety consideration. We therefore propose a framework for distributional AGI safety that moves beyond evaluating and aligning individual agents. This framework centers on the design and implementation of virtual agentic sandbox economies (impermeable or semi-permeable), where agent-to-agent transactions are governed by robust market mechanisms, coupled with appropriate auditability, reputation management, and oversight to mitigate collective risks.

Agentic AI Governance: Securing Autonomous AI Agents in Enterprise

When AI agents start making decisions, calling tools, and coordinating with other agents without waiting for human approval, the governance playbook most...

Yoshua Bengio | Why are AI agents lying, cheating and coordinating?

A lot has been written about the incidents of the last few months in which AI agents misbehaved in serious ways. They took actions that would be considered as crimes if a human took them, escaped their containment to cheat on assigned tasks while attempting to evade detection, and coordinated toward goals nobody had specified, such as launching cyber attacks. Before concluding what to do about it, it is worth asking why.

Dicklesworthstone/frankenterm

Terminal hypervisor for AI agent swarms: real-time pane capture, state-machine pattern detection, and a JSON API for coordinating fleets of coding agents across WezTerm

Why agent swarms could be the next “scaling law”

An OpenAI researcher called it “the most ‘feel the AGI’ moment that I had since reasoning models.”

Agentic test processes, LLM benchmarks, and other notes on agentic coding from Galapagos Island

I've been using AI fairly heavily since last November and the whole thing is a funny experience. An agent will do something that, if a human did it, you'd immediately fire them. My reaction, of course, is to act as if this is great and spin up a thousand agents so they can do even more of that.

Agents Rule of Two: A Practical Approach to AI Agent Security

We've developed the Agents Rule of Two. When this framework is followed, the severity of security risks is deterministically reduced.

When AI Agents Go Rogue: Agent Session Smuggling Attack in A2A Systems

Agent session smuggling is a novel technique where AI agent-to-agent communication is misused. We demonstrate two proof of concept examples.

LukeW | Agent Management Interface Patterns

As an increasing number of AI applications evolve to agents doing work for people, agent management becomes a critical part of these product's design. How can p...

pguso/ai-agents-from-scratch

Demystify AI agents by building them yourself. Local LLMs, no black boxes, real understanding of function calling, memory, and ReAct patterns.