Orchestration: Sequential, Handoff, Supervisor

Orchestration is the policy over policies: whose turn it is, what they receive, when the job ends. If you leave this to “whoever feels inspired,” you will get loops and silence. The coordination tax named this line item who_next. This part of the track pays it.

Roles with teeth are not enough. A planner, a worker, and a critic still need a rule that moves the blackboard. That rule is not a persona named Manager with the same tools as everyone else. That rule is code you can log: mode, path, assigned.

Three patterns cover most products. Learn how they fail differently even when a toy ticket prints the same FAQ answer. Production is the failure, not the happy path.

Three patterns

Sequential (pipeline). Planner → worker → critic → done, or intake → specialist → finish. Control flow is your code. Agents do not pick the next agent. Testable. Matches HITL gates: you can put a human node in the list and no worker can vote it out. Default for a stable graph.

Handoff (peer). The current agent names the next agent and a payload. Useful when routing is genuinely content-dependent in a way a rules engine cannot see yet. Risks: ping-pong, nobody calls finish, illegal reverse edges. Mitigations: max hops, typed brief, allow-list of who can call whom, duplicate payload hashes. Do not start here.

Supervisor (dispatcher). Assigns tasks to workers, collects results, decides to reassign or stop. Workers do not talk to each other. That restriction is a feature. Traces are star-shaped: one hub, many spokes. The supervisor should be stingy. If a rules engine can assign based on ticket category, do not pay a 70B model to be a switch statement.

A mature product often combines sequential and supervisor: the supervisor picks a subgraph (billing vs tech), and that subgraph is a pipeline with HITL. That is a later lesson, not a purity failure.

Swarms and debate are not a fourth and fifth orchestration religion. A swarm is map-reduce under a parent that already has a mode. Debate is a subgraph you run on high-stakes items. Both still need who-next inside the subgraph.

Walkthrough: “Where is my invoice refund?”

Three functions, same workers: intake classifies billing vs tech; billing answers 5-7 days; tech says restart the runner.

Sequential. Code runs intake, then picks billing because the category is billing, then returns. Path is ["intake", "billing"]. Nobody named the next hop in natural language.

Handoff. A loop. Current name starts at intake. Intake’s result chooses the next name. A max_hops of 4 stops a runaway. In the happy path you still land on billing. In a bad graph, tech hands back to intake forever until the cap.

Supervisor. Intake (or a rules function) runs at the hub. Hub assigns billing. Billing never sees tech. Path is an assignment, not a peer walk.

In this toy they print similar answers. In production they fail differently. Sequential fails closed when you forget a node (no HITL in the list — that is a code review). Handoff fails as a ping-pong. Supervisor fails as a bad assignment, or as a peer leak if you turn that flag on.

Do not let the similarity of the FAQ answer fool you into thinking the patterns are equivalent. Log mode so next month’s cost spike can be blamed on the right machine.

Run to execute this in your browser. Nothing is sent to a server.

What printed: sequential shows mode: sequential, path intake then billing, refund delay answer. Handoff shows mode: handoff and a path that also ends on billing. Supervisor shows assigned: billing and the same answer. Change the message to a timeout with no “invoice” and all three should go to tech. The lesson is the fields: path versus assigned, and max hops waiting in the handoff function for the day the graph grows a cycle.

This toy still uses lambdas as fake workers. Real workers are separate jobs with allow-lists. The orchestrator must not share one Python closure’s globals as a side channel — that is a mesh hiding in a module.

How to choose

Start sequential if the graph is stable (billing always HITL, tech always tests). Use a supervisor when you have many worker types and you want no peer talk. Use handoff only when you can write the edge allow-list and you have already been burned by a sequential graph that needed content-dependent extra hops you cannot encode as subgraphs.

Never choose “all three, the model will pick.” That is a fourth pattern: chaos. Stamp orchestration.mode at job start. Changing mode mid-job is an incident unless you have a migration.

A 70B supervisor that only switches on the word “invoice” is a cost bug. Put that switch in code. Save the model for the worker that must read policy.

How agents use this

Log mode, path, and assigned. When cost explodes, you want to know whether you were in handoff soup or a 40-child swarm.

Default new products to sequential. Add supervisor when worker types multiply. Add handoff last, with hop limits from the next lessons. Put the choice in flags and config, same as authz — not in a planner thought.

Eval each mode with a fixture that induces its failure: sequential without HITL in the list; handoff with a reverse edge; supervisor with a peer message. Those fixtures belong in this repo before you add personas.

Do not re-teach a single-agent ReAct loop as orchestration. One agent calling tools is still one policy. Orchestration starts when two policies need a turn-taking rule. If you only have one policy, you do not need this part. Return to the baseline.