When Zero-Shot Is Enough

Zero-shot is: instructions + context + input + contract, no worked pairs. It is the default, not a lesser mode. Examples cost tokens on every call and they steal window from the real observation. A 2k-token gallery plus a 2k-token stack trace is how the spec falls off the end.

Use zero-shot when:

• The task is well-known (“summarize,” “extract the date,” “translate”)
• The contract is tiny and you have JSON mode or a strict parser
• Examples would leak customer data you have not redacted
• You are still discovering the modes — evals first, shots later
• A small model plus a short prompt already passes the frozen slice (LLM routing still applies)

Use few-shot when:

• Format is picky and JSON mode still drifts
• Labels are your house dialect (infra vs sev1 vs billing)
• Zero-shot fails a frozen eval slice, and a shot of that mode fixes it
• Taste matters and adjectives failed (how you refuse, how terse)

“Just add another example” is the same smell as “just add a sentence to the spec.” It is a PR. It changes every later token. It changes cost. It can change safety if the new shot is a bad last shape.

Measure both quality and weight

A 2-point parse-rate win that triples cost on a high-volume router is not a win. Count tokens in the shot block, not only in the user ask. If shots are bigger than the ask, you are teaching the gallery, not solving the ticket.

Start zero-shot with a contract and an eval. Add shots only for failing modes. Measure parse rate and token count. Remove shots that do not move the score. Dead shots are still paid for.

Pin the model version when you compare. Temperature above 0 flickers. For prompt diffs, temperature 0 when you can.

Run to execute this in your browser. Nothing is sent to a server.

What printed: zero-shot is spec plus ask. Few-shot is much larger. Shots alone exceed the ask. That is the budget you pay on every router call, including the ones that never needed a gallery.

Word-split is not a real tokenizer. It is enough to see the shape of the problem. Production counts vendor tokens. The policy is the same: earn the gallery with a failing eval, not with fear.

Discovery order

1. Write the contract and a tiny golden set (eval-driven lesson).
2. Ship zero-shot against that set.
3. Read the failing clusters, not the average.
4. Add at most one shot per failing mode.
5. Re-run. Keep the shot only if the cluster moves and nothing else regresses.
6. If the cluster does not move, stop prompting that cluster — you may need a tool, a workflow, or a smaller contract (last lesson in this track).

Teams that start with twenty incidents pasted from Slack never learn which sentence mattered. Teams that start at zero learn.

Walkthrough: the gallery that ate the ticket

The router sees 400 tokens of user stack trace and 2,400 tokens of shots. The spec is at the top. Truncation or lost-in-the-middle drops the contract. Parse rate on long tickets falls. Someone adds more shots. Cost triples. The failing cluster was long input, which needed a long-dump shot or a smaller gallery, not twelve billing paraphrases.

Zero-shot with JSON mode already passed short tickets. The honest move: delete the gallery, keep the contract, add one long-dump case to the eval. If that case fails, add one long shot. Measure tokens. If the win is two parse points and a 3x bill, you did not win.

What goes wrong if you skip this

Few-shot becomes the default religion. Every incident becomes a pair. Windows fill. Specs vanish. You cannot tell which example did work because you never ran zero-shot. Discovery order exists so you earn complexity.

Zero-shot is instructions + context + input + contract. It is the default. Examples cost tokens on every call and steal window from the real ticket. Use zero-shot when the task is common, the contract is tiny, examples would leak, or you are still discovering modes. Use few-shot when format still drifts, labels are house dialect, or a frozen slice fails and one mode-shot fixes it.

Start zero-shot. Read failing clusters. Add at most one shot per cluster. Re-run. Keep only if the cluster moves and nothing else regresses. Measure parse rate and token count. A 2-point win that triples the router bill is not a win.

Common mistakes

Discovery order is the whole skill: contract and goldens, zero-shot, read clusters, one shot per failing mode, measure tokens, stop when the cluster does not move. At that point you are no longer in the examples chapter. You are in “prompting is not enough.” Do not skip the measurement just because a gallery feels like progress. Window you spend on shots is window you do not spend on the ticket.

How agents use this

Start zero-shot with a contract and an eval. Add shots only for failing modes. Measure parse rate and token count. A high-volume router should stay short. The LLM routing lesson still applies: small model + short prompt first.

Tip:“Just add another example” is a PR. It changes every later token. Earn it with a failing eval, not with fear.