How to Write a Multi-Agent Supervisor: The 39% Rewrite and Why the Savings Weren't Where I Expected

My supervisor skill was 3,067 words. That is the file an agent reads before it reads anything about the actual task. By the time a real run began, the model had already taken in 4,615 words of pure process: the skill, the injected project plan, and the brief I wrote for the supervisor. AGENTS.md and the first source file arrived after all of that.

I cut it to 1,872 words. The 39% is the least useful number in this article. What I actually learned is that almost none of the saving came from where I went looking for it.

I went looking in reasoning_effort, because everyone does. It is the one dial that maps directly onto a price list, so it feels like the place waste would live. It was not. The waste was a single sentence in the process description saying that every work package travels the same road: root diagnoses, supervisor plans, implementer writes code, test author writes tests. Four contexts, four briefs, four copies of the same background, for a change that might touch two files.

Deleting that sentence saved more than every model downgrade combined.

What is a supervisor agent, really

The word invites the wrong picture. "Supervisor" sounds like a manager, and once you have a manager you start drawing an org chart, and once you have an org chart you fill the roles because the boxes are sitting there.

A supervisor is a context boundary. Nothing about it requires a persona or a rung on a ladder.

The only thing a subagent physically gives you is a fresh, separate context window. Everything else you might want from it, whether that is specialisation or review or a second opinion, you can get inside one context by writing better instructions. Isolation is the part you cannot write your way around. So the question at each branch point is not who should own the work, but whether the work produces or consumes context that would poison the parent.

Ask it that way and most of the org chart evaporates. A diagnosis and the fix that follows from it share the same reasoning. Splitting them across two agents does not divide the work, it forces you to serialise the reasoning into a brief, ship it, and hope the second agent reconstructs it. You pay for that reconstruction at every hop.

Multi agent vs single agent: what the measurements say

Before rewriting anything I went looking for numbers, because mine were going to be a sample of one.

Anthropic's guidance is blunt about the cost. Multi-agent implementations "typically use 3-10x more tokens than single-agent approaches for equivalent tasks." In their write-up on when to use multi-agent systems they describe watching teams build elaborate agent hierarchies that "suffered from lost context at each handoff and spent more tokens coordinating than executing."

Then there is the line that describes precisely the structure I had built:

In one experiment with agents specialized by software development role (planner, implementer, tester, reviewer), the subagents spent more tokens on coordination than on actual work.

That is a warning about one particular way of cutting the work up, and it happens to be the way most of us reach for first.

A published benchmark backs the same shape with harder numbers. A group compared single-agent and multi-agent RAG pipelines on README generation across 180 repositories. The single agent averaged 7,840 tokens and 40 seconds per file. The multi-agent system spent 56,242 tokens and 78 seconds, a 7.1x increase, and came out marginally worse on quality: 0.1964 against 0.2007 on ROUGE-L F1, with no meaningful difference in BERTScore. The paper is called "The Illusion of Agentic Complexity in README.md Generation", which saves you reading the conclusion.

Now the other side, because it matters and it is also Anthropic's. Their research system, with Opus as lead and Sonnet subagents, beat single-agent Opus by 90.2% on their internal research eval. Multi-agent is not a myth, and a margin that size is not noise.

Look at what it won at, though. Breadth-first search across many independent sources, where each parallel agent burns its own context on its own branch and hands back a paragraph. That is context isolation doing real work. In the same post they add a caveat that gets quoted far less often:

most coding tasks involve fewer truly parallelizable tasks than research, and LLM agents are not yet great at coordinating and delegating to other agents in real time

So the rule comes out of the measurements rather than out of preference. Fan out when the work has genuinely independent branches that each generate context nobody else needs. Do not fan out because you can name four roles.

Split by context, not by job title

The rewrite came down to one substitution. Where the old file decomposed by type of work, the new one decomposes by what context the work needs.

Anthropic calls this context-centric decomposition, and the failure it prevents is specific: "Dividing by type of work (one agent writes features, another writes tests, a third reviews code) creates constant coordination overhead."

Three concrete changes fell out of it.

The implementer now writes the tests. In the old file, production code and tests went to different agents, on the theory that a fresh pair of eyes writes more honest tests. In practice the test author received a brief, not an understanding. It knew the requirement but not the twelve small decisions the implementer had made getting there, so it wrote tests against the requirement as stated and missed the places where the implementation had quietly reinterpreted it. Same context, one agent, and the tests got closer to the code because the code was still in the room.

A single sequential work package never leaves root. If the work is one coupled thread of reasoning, a supervisor sitting in the middle only relays messages. Root can call a bounded scout for a mechanical lookup, or a black-box verifier at the end, without spinning up a supervisor context to sit between them.

Full checks run once. Targeted checks per work package, then the whole test suite, typecheck and build a single time against the integrated diff. Running everything after every package was buying the same signal five times over.

The exception worth keeping

One split survived, and it is the one Anthropic recommends: black-box verification.

A verifier gets the artifact, the requirements, the changed files, and a definition of success. It gets no history, and that is the point rather than a compromise. "Verification requires minimal context transfer by nature," which is why this particular handoff loses nothing. There is nothing to lose. The verifier works because it judges the end state without knowing how the end state was reached.

One detail from their post is worth stealing verbatim, because I had this failure and misdiagnosed it:

The instruction 'You MUST run the complete test suite before marking as passed' is essential. Without explicit requirements for comprehensive validation, verification agents take shortcuts.

Mine took shortcuts, and I read that as laziness in the model. It was underspecification in my brief.

The gate: when a supervisor earns its context

The rewritten skill will not spawn a supervisor unless one of four things is true.

1. Two or more work packages are genuinely independent and can run at the same time without touching shared files or synchronising often.
2. A subtask produces a large amount of context that the parent should only ever see as a compact result: searches, logs, big test output.
3. Several executors would otherwise flood the root context with intermediate output, and one supervisor can absorb it and own recovery.
4. An interrupted run already has supervisors recorded in durable state, or the owner explicitly asked for supervised parallel work.

Everything else stays in root. Diagnosis, architecture, the parts of the codebase I refuse to delegate at all, and most single-package changes. One line in that section does more work than all the others: a three-step checklist is not a reason to build a three-level agent tree.

AWS arrives at the same place from the cost side. Their Agentic AI Lens frames it as a determinism question rather than an architecture question. At every orchestration point, either the routing decision picks from a finite enumerable set, or it is a judgment call over open-ended input. The first kind costs nothing to route with a rule. Only the second needs a model. Their thresholds are refreshingly unfashionable: rules under ten branches, a lightweight classifier for ten to fifty, an AI supervisor only when the category space is genuinely unbounded.

They also name the metric I did not have and now do, the orchestration overhead ratio, which is supervisor tokens divided by total workflow tokens. Track orchestrator spend separately from worker spend, because "tracking only aggregate workflow cost without decomposing orchestrator compared to worker spend" hides exactly the problem you are trying to find. When the ratio drifts up, the fix is a different pattern rather than a bigger budget.

When you do want an independent verifier

Removing the mandatory test author meant writing down the conditions under which I want one back. The gate fires when any of these hold:

• the package is high-risk by signal rather than by vibe: async or concurrency, cross-layer state, persistence or migrations, auth, destructive operations, money, public API or schema compatibility, flaky failures, or a first attempt that already failed;
• the implementer's tests mirror the implementation instead of observable behaviour;
• the change needs an adversarial pass on GUI, accessibility, race conditions, or a public contract;
• the supervisor cannot explain why these tests would have caught the original bug;
• the supervisor is running below the model tier the work expects.

Otherwise the implementer writes the tests and the supervisor reviews them against the real diff, not against the summary. An agent reporting a green build and an actual green build are different objects, and only one of them counts as evidence.

The model matrix, and why it is second-order

Here is the routing table the rewrite settled on. It carries both harnesses because the same skill runs under Claude Code and Codex.

Role	Claude Code	Codex default	Escalate when
Supervisor	opus	gpt-5.6-sol, low	medium for ambiguous integration, contradictory evidence, or a failed package
Implementer	sonnet	gpt-5.6-terra, medium	low for mechanical edits, high for a real risk signal or a second attempt
Verifier / test author	sonnet	gpt-5.6-terra, medium	high when a separate verifier was justified by the gate
Scout	haiku	gpt-5.6-luna, low	only if the job stopped being mechanical

This matches OpenAI's own guidance, which puts medium as the "balanced starting point," reserves low for latency-sensitive work, and gives high and xhigh only "when more reasoning produces a measured quality gain." Their migration advice is the useful bit for anyone tuning a harness: keep your current effort as the baseline, then compare one level lower.

The price spread is real. On the published model catalog, Sol runs $5 per million input tokens against Terra's $2 and Luna's $0.20. Sol costs 2.5x Terra and 25x Luna on input. Billing inside a Codex plan may work out differently, so treat the ratio as a shape rather than an invoice.

Moving a supervisor from medium to low, though, shaves a fraction off a context that should not have existed. Not spawning that supervisor removes the whole thing: its brief, its copy of the plan, its summarisation pass, and the round trip back to root. Effort tuning is real money, and it is worth doing once the topology is right.

Brief narrowly

The last thing that shrank was the briefs, and this one surprised me most.

I had been handing each child agent the full plan. It felt responsible. Give it the whole picture and it makes better local decisions. What it actually did was hand a model working on one server-side function several hundred words about locale rules, three unrelated work packages, and a set of constraints that could never bind it, all of which had to be read and weighed and held.

A child now gets six things: the objective, the cause with a file:line and the mechanism, the files it may touch, the success criteria, the exact commands to run and what passing looks like, and the traps diagnosis actually found. Plus the fences, meaning what it must not do.

It does not get the full project plan, the other work packages, the rules for layers it will never touch, or a transcript of any previous agent's reasoning.

The supervisor's own brief looks similar and adds ownership: which packages it owns, the routing table, the no-production-code boundary, the read-first paths, the targeted checks against the once-per-run integration checks, and a reporting contract with one clause I would not ship without. Do not report completion for anything you did not verify. If a check fails, quote the output.

What I measured, and what I cannot claim

Honest accounting, because this site is called what it is called.

What I measured: the skill went from 3,067 to 1,872 words. A cold run now opens with 4,615 words of instruction rather than roughly 5,800, before the repo rules and the source files land on top. The mandatory four-context chain is gone.

What I have not measured: whether any quality was lost. I cannot claim it was not, and neither can anyone else who has rewritten a harness and reported a token saving. Word counts are easy to measure, which is exactly why they are the number people publish.

So the file now carries its own calibration clause. Every run logs model and effort per role, token usage where the runtime exposes it, the number of rebriefs a package needed, and defects found after implementation was declared done. Same class of work, current setting against one rung lower. When misses and rework eat the saving, the setting goes back up.

The defect count is the one I expect to hurt. Token counts tell you what a run cost. Defects found afterwards tell you what it cost you later, and that is the bill a leaner harness can quietly run up while its own numbers look excellent.

FAQ

What is a supervising agent? An agent whose job is to route work, integrate results, and verify them, rather than to produce the work itself. It is most usefully understood as a context boundary: it exists so its children's intermediate output never reaches the parent, and so someone owns recovery when a child fails.

What is a multi-agent system? Any setup where more than one model context collaborates on a single task, usually with a lead or supervisor delegating to subagents that have their own context windows, tools, and instructions.

Multi agent vs single agent, which is cheaper? Single agent, by a wide margin, and often by more than you would guess. Anthropic reports 3-10x more tokens for multi-agent on equivalent tasks, and the README benchmark measured 7.1x with slightly worse output. Multi-agent wins on cost only when parallelism collapses wall-clock time, or when the work genuinely will not fit in one context.

Do I need a separate agent to write tests? Usually not. Tests and implementation share the same context, and splitting them makes the test author work from a description instead of an understanding. Bring in a separate verifier for high-risk work, for adversarial passes, or when the first attempt failed, and give it black-box criteria rather than the implementation history.

How do I tell if my supervisor is overhead? Measure the orchestration overhead ratio, supervisor tokens over total tokens. Then count how many delegations were genuinely concurrent. If most of your work packages ran one after another through the hierarchy, the hierarchy was a relay, and the token bill was for message passing.