Simplicity Scales

Ask an engineering team to show you their most impressive AI agent and you will usually be shown their least reliable system. The demo where six agents debate each other gets the applause. The boring pipeline that chains four prompts in a fixed order gets the production traffic. Two years of teams shipping agents to real customers have made the pattern hard to ignore: simplicity scales, and complexity breaks.

This runs against instinct. In most of software, more architecture signals more maturity. In agent systems the arithmetic points the other way. A model step that succeeds 95 percent of the time sounds dependable. Chain ten of those steps and the full run succeeds about 60 percent of the time. Let agents call other agents in an open loop and errors stop merely accumulating; they compound, because each agent acts on the flawed output of the previous one with complete confidence.

Complexity is a cost, not a credential. Every layer in an agent architecture must be earned by a measured failure that the simpler design could not fix.

The ladder of agent architectures

Anthropic's guide to building effective agents, distilled from work with dozens of production teams, opens with advice most teams skip past: find the simplest solution possible, and only add complexity when it demonstrably improves outcomes. The guide draws a line between workflows, systems where your code steers model calls along fixed paths, and agents, systems where the model itself decides what happens next. That line turns into a ladder.

The bottom rung is a single model call with a good prompt and the right context. One rung up is prompt chaining: several calls in a fixed sequence, each one checking or extending the last. Next comes routing: a first call classifies the request and sends it down one of several specialized paths. Above that sit parallel workflows and evaluator loops, still running on rails your code laid down. Only at the top does the model take the wheel: first a single autonomous agent with tools, then multiple agents coordinating with each other.

Each rung up buys flexibility and sells predictability. A workflow can be traced, tested, and debugged like ordinary software, because you know every path a request can take. An autonomous loop can handle problems you never anticipated, and it can fail in ways you never anticipated. OpenAI's practitioner guide gives its customers the same instruction: validate the single-agent version first, and treat additional agents as a last resort for genuine orchestration limits, not as a starting point.

The evidence against premature swarms

The failure data is now public. A Berkeley-led study titled "Why Do Multi-Agent LLM Systems Fail?" analyzed more than 150 execution traces from popular multi-agent frameworks and catalogued 14 recurring failure modes. Most were not model limitations. They were coordination problems: agents ignoring their task specifications, agents losing or withholding information a teammate needed, and no one verifying the final answer. The systems failed at being organizations before they failed at being intelligent.

Cognition, the company behind the Devin coding agent, drew a blunter conclusion from its own production experience and published it under the title "Don't Build Multi-Agents." Their argument centers on context. Reliable action requires the full history of a task, and the moment work is split across agents, each one acts on a partial view. Two competent agents holding different halves of the picture will make confident, conflicting decisions, and the errors surface exactly where their outputs get stitched back together.

When complexity is earned

None of this means a multi-agent system never pays for itself. Anthropic built one for its Research feature and published the numbers. On their internal research evaluation, the multi-agent version outperformed a strong single agent by roughly 90 percent, because broad research decomposes cleanly into independent, read-only searches that can run in parallel. It also consumed about 15 times the tokens of an ordinary chat. The gain was real, the bill was real, and both were measured before the architecture shipped.

That is what earned complexity looks like. The task parallelizes naturally. The value of the output justifies the token cost. And the team could prove the improvement because both versions ran against the same evaluation. Complexity arrived as the conclusion of an experiment, not as the starting assumption.

The test works in reverse, too. If nobody can name the failing evaluation that forced a new layer into the architecture, the layer is speculative. It was added because it felt advanced, and it will be paid for in incidents.

The one-rung rule

A practical discipline follows from all of this, and it fits in four sentences. Write the evaluation before writing the architecture, so "better" has a definition. Start on the bottom rung of the ladder. Climb exactly one rung when the evaluation fails, and record why in the design document. Price every climb in the three currencies that matter: latency, tokens, and new places to fail.

A customer support automation makes the pattern concrete. A single model call with good retrieval typically resolves the large majority of tickets. Routing earns its place when the traffic splits into genuinely different workflows, say billing disputes versus technical faults. An autonomous agent loop earns its place only for the small tail of open-ended investigations where no fixed path exists. Teams that start with the swarm build the hardest version first, then spend months discovering that most of their traffic never needed it.

Conclusion

Decades before language models, the systems theorist John Gall observed that a complex system that works is invariably found to have evolved from a simple system that worked, and that a complex system designed from scratch rarely works at all. Agent engineering keeps rediscovering his point at production prices. The swarm that dazzled in the demo becomes the outage nobody can trace; the plain chain of prompts becomes the workhorse nobody talks about.

The advantage now goes to teams that treat architecture the way actuaries treat risk: every layer priced, every addition justified by evidence, every simplification celebrated. When someone proposes a multi-agent design, the useful question is not whether it would be impressive. It is which measured failure it fixes.

What leaders should do

1. Require a failing evaluation before approving any new architectural layer. If the team cannot show the test the simpler design fails, the answer is not yet.
2. Put the ladder in your design review template: single call, chain, routing, workflow, single agent, multi-agent. Every rung above the first needs a written justification.
3. Track cost per resolved task and failure rate per component for each agent system in production, and review quarterly whether every layer still earns its keep.
4. Reward deletion. Treat a team that collapses a five-agent system into a two-step workflow with equal results as having shipped something, because it has.