Five orchestration patterns for multi-agent systems (and when to use none)

Add an agent when you need isolation — of context, of authority, or of failure. Adding one for any other reason usually buys you a slower system that is harder to debug.

The default failure mode of multi-agent design is enthusiasm. A problem gets decomposed into seven cooperating specialists because the decomposition is satisfying to draw, and the resulting system is four times slower, six times more expensive, and impossible to debug because no single trace explains what happened.

The useful question is not "how do I decompose this?" It is "what would I gain by adding a boundary here that I cannot get from a tool call?"

There are exactly three good reasons to add an agent: context isolation, authority isolation, and failure isolation. If none of those applies, you want a tool, not an agent.

Context isolation — the subtask needs to read 200 pages that the parent must not carry for the rest of the run. The sub-agent reads them, returns a paragraph, and the pages are discarded with its context.

Authority isolation — the subtask has different permissions. A summarizer that cannot write to the database is a genuinely different security principal from a planner that can.

Failure isolation — the subtask can fail, retry, and time out without taking the parent's progress with it.

The five shapes#

1. Pipeline#

Fixed sequence, each stage's output feeding the next. Extract, then normalize, then validate, then write.

Use when the decomposition is known in advance and does not depend on the data. It is the cheapest pattern, the easiest to test — each stage is a pure function with a typed contract — and the easiest to observe. A remarkable number of systems marketed as "agentic" are pipelines with a language model in one stage, and they are better for it.

The limitation is rigidity: a pipeline cannot decide it needs an extra step.

2. Supervisor#

A coordinator that decides which specialist to invoke, in what order, and when to stop. The specialists do not know about each other.

This is the correct default for open-ended work, and it maps cleanly onto how tool-calling models already behave — the specialists are just tools whose implementation happens to be another model call.

SPECIALISTS = {
    "search":  {"desc": "Find documents. Returns titles + ids, never full text.",
                "schema": SearchArgs},
    "read":    {"desc": "Read one document by id. Returns up to 4k tokens.",
                "schema": ReadArgs},
    "compute": {"desc": "Run a sandboxed calculation. Returns a scalar or table.",
                "schema": ComputeArgs},
}

async def supervise(task, budget: Budget):
    ctx = Context(task)
    while not ctx.done and budget.remaining():
        step = await planner(ctx.render(), tools=SPECIALISTS)
        if step.kind == "finish":
            return step.answer
        result = await run(step.tool, step.args, budget=budget.child(step.tool))
        ctx.append(summarize_for_parent(result))   # never the raw payload
    return ctx.best_effort()

The line that matters is the last one inside the loop. A supervisor that appends raw specialist output to its own context has reinvented the monolith with extra network hops — it now carries every specialist's full working set. Sub-agents must return conclusions, not transcripts.

3. Blackboard#

Agents read from and write to a shared structured workspace rather than passing messages. Each watches for the conditions it can act on.

Good for problems where the order genuinely is not knowable in advance and multiple partial contributions compose — document understanding, incident triage, constraint solving. We used a variant of this on a constraint-aware CAD system, where intent parsing, constraint analysis, operation planning, and validation each contributed to a shared model of the sketch.

The costs are real: shared mutable state means write conflicts, and "why did the system do that" requires reconstructing the interleaving. Version the blackboard, make every write attributable, and you can at least replay it.

4. Debate / ensemble#

Run N agents on the same problem independently and reconcile — by majority, by a judge, or by synthesis.

Genuinely effective for tasks with a verifiable answer and high variance: generation-and-selection beats single-shot generation on most reasoning benchmarks. It costs N times as much, which is fine for a design decision and absurd for a per-request path.

The subtlety is that independence is what buys you the improvement. Three agents with the same prompt and the same model produce correlated errors and a false sense of consensus. Vary the framing — one asked to solve, one asked to refute, one asked to find the missing case — and the ensemble actually covers different failure modes.

5. Market / bidding#

Agents bid on tasks based on estimated fitness; an allocator assigns work.

Almost always over-engineering. It earns its place when agents have genuinely different and dynamic capabilities — heterogeneous robot fleets, multi-tenant scheduling with real resource contention. For a set of prompt-differentiated LLM agents, a supervisor with a good tool description does the same job with a tenth of the machinery.

Choosing#

| Pattern | Use when | Main cost | Debuggability | | --- | --- | --- | --- | | Pipeline | Steps known in advance | Rigid | Excellent | | Supervisor | Open-ended, decomposable | Planner is a bottleneck | Good | | Blackboard | Order unknowable, partial contributions | Shared-state complexity | Poor without versioning | | Debate | Verifiable answer, high variance | N× cost | Good | | Market | Heterogeneous, contended resources | Machinery | Poor |

The failure modes nobody puts in the diagram#

Context bloat in the supervisor. The most common performance cliff. Each sub-agent returns a paragraph, the supervisor accumulates twenty of them, and by step fifteen the planner is reasoning over a context in which the original task is a distant memory. Fix: hard cap the parent's context, summarize aggressively, and keep the task statement pinned at both ends of the window.

Infinite delegation. Agent A decides this is really B's job; B decides it is A's. Fix: a monotonically decreasing depth budget passed down through every call, and a hard refusal at zero. Not a heuristic — a counter.

Lost error provenance. A sub-agent fails, returns "I could not complete that", the supervisor tries something else, and the root cause is gone. Fix: structured errors that propagate a machine-readable cause, and a trace id shared across the whole run.

Cost with no ceiling. A single ambiguous request fans out into two hundred model calls. Fix: a budget object threaded through every call — tokens, wall clock, and depth — that is decremented, not merely observed. If it is not enforceable at the call site, it is not a budget.

@dataclass
class Budget:
    tokens: int
    seconds: float
    depth: int

    def child(self, label: str) -> "Budget":
        if self.depth <= 0:
            raise DelegationLimit(label)
        return Budget(self.tokens // 2, self.seconds * 0.6, self.depth - 1)

Halving the token budget at each level bounds the total: a depth-4 tree cannot spend more than roughly twice the root's allocation no matter how wide it gets.

Observability is the deciding factor#

The pattern you can debug at 3am beats the pattern that is theoretically better. Three things make a multi-agent system debuggable, and they are not optional:

1. One trace id per run, propagated everywhere, so a single query returns every model call, tool call, and error in causal order.
2. Every prompt and completion persisted, with the exact model version and parameters. When behaviour changes after a model update — and it will — this is the only way to know what changed.
3. A replayable transcript. The ability to re-run a failed trace against a new prompt or a new model and diff the outcome. This turns "agents are unpredictable" from a shrug into a test suite.

We would take a pipeline with all three over a beautifully decomposed agent society with none of them, every time.

The honest default#

Start with one agent and good tools. Give it a well-typed tool surface, a budget, and a trace. Watch where it fails.

If it fails because its context filled with material it did not need, extract that work into a sub-agent — context isolation. If it fails because it needed permissions it should not have, split on authority. If it fails because one flaky step took down a long run, split on failure.

Otherwise, keep it as one agent. The best multi-agent architecture is frequently the one you did not build.