Parallel Tool Calls

Some models emit several tool calls in one turn (search A, search B). That is useful. It is also how you double-pay. Parallelism is a runtime policy, not a gift you always honor. The loop is a client: it may ask to run a batch. The dispatcher decides what is legal together.

Rules:

• Independent reads — run together
• Writes — one at a time unless each has an idempotency key
• Delete then read — must be sequential
• A valid search next to a malformed refund — run the search only after you decide the refund failed validation, or fail the whole turn. Pick a policy and test it.

If two calls share a transaction, say so. Two get_job calls are fine together. refund plus refund is an incident unless idempotent. refund plus get_job mixes a write with a read of a world that is about to change. In this classroom, that mix is forbidden. In production you might allow it if the read cannot see the write’s table. Default to no mix. Measure later.

Isolation and order

Parallel means overlapping in time, not “the model listed them in one JSON array.” If your worker is single-threaded, you still need the policy because the next deploy will be async. Policy first. Concurrency second.

Order in the array is not a happens-before. If the model emits close_ticket then get_ticket, running them in parallel can return open. Either serialize writes, or serialize anything that shares a key, or reject the batch.

Shared keys: two writes to the same invoice_id must collapse to one keyed call or run serial under a lock. Two searches with different queries do not share a key. Two searches with the same query can be deduped — optional optimization, not a safety feature.

Validation of a batch

Validate each call independently. A valid search does not excuse a malformed refund sitting next to it. Then apply the parallel policy. Then execute the safe subset or reject the turn. Executing the search and dropping the refund without an observation is a protocol bug: the model will retry the refund. Prefer: observations for every call, including denied: multiple writes.

Caps still apply. Ten parallel searches can be ten timeouts and a bill. Max parallel reads, max total reads per turn, max one write. Put numbers on the worker.

Failures inside a batch

If one read times out, the others may still return. Pack each observation separately. The loop should not treat the whole turn as empty. If one write fails halfway, you needed a key. Without a key you are in incident land: poll status, do not fire the sibling write.

Partial execution is why you log the batch id: which names ran, which were skipped, which errors. Replay needs that.

Classroom policy

The live box is a predicate, not a thread pool. Two searches: ok. Two refunds: no. Refund plus get: no. Search plus search: ok. That is enough to test the label. Wire it to the registry’s read/write bit when you have one.

Run to execute this in your browser. Nothing is sent to a server.

What printed: get_job+search true, two refunds false, refund+get false, two searches true. If your production policy differs, write it as a function this small and test it. Do not bury it in a prompt that says “feel free to call tools together.”

What goes wrong

Honoring every parallel batch because the vendor API supports it. Mixing writes to look fast. Deduping writes with different amounts because the invoice id matched. Validating only the first call. Executing before the policy check. All of these are dispatcher bugs.

How to test parallel policy

A table of name lists to ok/why. Include empty batch, one write, write+write, write+read, read+read, unknown names (unknown should fail before parallel). Integration: two fake slow reads, assert both observations exist. Two refunds without keys, assert at most one handler flag flipped.

Batches are policy objects

A parallel turn is not “the vendor set a flag.” It is a list you validate, classify, and maybe split. Classify each name as read or write using the registry label, not using the first three letters of the string. get_or_create is a write even if it starts with get. search_and_archive is a write. If the label is missing, treat it as write until someone proves otherwise.

Decide a mix policy and test it. This classroom forbids read+write in one batch so the world stays still while you write. Production might allow a read of a different store. Write that exception as a function over names and resource keys, not as a comment. Shared resource keys (invoice_id) serialize even if both calls are labeled write-with-key: two keyed refunds to the same invoice should collapse or run one-at-a-time, not race.

Partial failure needs a batch id in the log: which calls ran, which were skipped, which errors. If you execute the valid search and silently drop the malformed refund, the model will emit the refund again. Give it an observation: denied: invalid sibling or invalid_args on that id. Protocol first. Speed second.

Caps still apply to the whole batch. Ten parallel searches can be ten timeouts and a bill. Max parallel, max total reads per turn, max one unkeyed write. Numbers on the worker. The prompt cannot enforce them.

How agents use this

If you support parallel calls, the assembler still advertised those tools. Policy may refuse the combination. The loop should surface multiple writes as an observation, not as a hang. Prefer many reads, then a separate turn for a write. That shape is easier to approve and to key.

Ids on each call stay mandatory. Parallel without ids is a shuffle. The runtime enforces pairing. The model does not.

Watch out:Two refunds in one turn are an incident unless they share a key or are serial under a lock.