Few-Shot Prompting

Zero-shot means you describe the task and hope. One-shot means one worked example. Few-shot means a handful of input → output pairs sitting in the window. You are stuffing a miniature dataset into the prompt so the next-token distribution copies the shape of those pairs.

That is not a metaphor. In-context learning is pattern completion. The model does not “remember your examples” as a database. It continues text that looks like them. If the examples are JSON objects with keys status and answer, the next answer is more likely to be that object. If the examples are witty paragraphs, you will get witty paragraphs — including on the ticket that needed a refusal.

Examples program three things that instructions alone do badly:

1. Format — JSON keys, CSV columns, “TAG: value” lines, the exact enum spellings your parser allows
2. Boundary cases — empty input, mixed language, sarcasm, “I do not know,” a dump that is too long
3. Taste — how terse, how many hedges, when to refuse, whether “sorry” appears

Instructions can say “be terse.” An example that answers in nine words shows terse. Instructions can say “refuse secret dumps.” An example that returns status=refused shows the legal shape of a refusal. The model copies what it sees at the end of the list more than what it was told in the middle (recency).

If your examples are all happy-path English and production is tickets with stack traces, you did not few-shot. You decorated.

What few-shot is not

Few-shot is not a knowledge base. Do not paste forty handbook pages as “examples.” That is context, and it belongs in a labeled data block — or, later, in retrieval. Examples that teach facts go stale. Examples that teach shape stay useful.

Few-shot is not a secret store. Ticket IDs, emails, and card suffixes in shots are leaked production data. The model will also imitate mistakes in your shots: a wrong label, an invented citation, a tool you deleted. Treat every example as training data you are willing to ship.

Few-shot is not “the more the better.” Duplicating the same refund five times teaches repetition. Diversity beats volume. The next lesson is which shots to keep. This lesson is what shots are.

How many

Enough to cover the modes of the task, not enough to drown the actual input:

More than that only when evals prove a mode is missing. A 2k-token gallery plus a 2k-token stack trace is how the spec falls off the end.

Pick shots the way you pick unit tests: one per interesting branch, redacted, versioned, reviewed.

A toy that is the same idea

A language model does few-shot by continuing tokens. A nearest-neighbor classifier does it by overlap. The toy below has no neural net. Labels come only from examples. That is the point: if the example set is only billing English, “OOM killed the runner” is a stranger.

Run to execute this in your browser. Nothing is sent to a server.

What printed: the three in-domain queries pick billing, account, infra. The meaning-of-life line is weakly scored or unknown-ish — overlap is accidental. If you drop the infra shots, OOM no longer maps to infra. A language model is smoother and has more ways to cheat. The lesson is the same: shots are the training set you actually shipped.

Walkthrough: the router that only knew refunds

Acme’s ticket router few-shots three paraphrases of “please refund March.” Production is half stack traces: OOM, max_steps, vendor timeout. The spec says labels are billing, account, infra. The model has never seen infra in the window. It maps OOM to billing because “please” and “failed” appeared near money in pretraining and in your shots. Support refunds an outage. The eval that would have caught this was one infra ticket. The shot that would have taught the shape was one redacted OOM pair, last example still legal JSON.

That is few-shot doing its real job: programming modes, not stuffing a handbook. The handbook is context. The pairs are a tiny training set. If you need forty pages of policy, you need a labeled data block, not forty “examples.”

What goes wrong if you skip this

You will treat Slack as the example store. Names leak. A wrong label in a shot becomes next week’s policy. You will paste incidents until the spec falls off the window, then blame the model for ignoring JSON. You will also ship secrets because “it was only an example.” Examples are training data. They are logged. They are stolen with the prompt.

Few-shot is a tiny dataset in the window. It programs format, edge cases, and taste — not facts, not a vector database, not a secret store. Zero-shot is the default. One-shot is one pair. Few-shot is a handful. The model continues the shape it just saw. If the last pair is legal JSON, Tuesday is more likely JSON. If the last pair is a joke, Tuesday is a joke.

Diversity beats volume. Five paraphrases of one refund are one mode. Cover happy path, empty, long, refuse, and the house labels you actually use. Redact names. Version the file. Add a shot only when a frozen eval cluster fails, and add the eval case in the same PR.

Common mistakes

How agents use this

Put shots in a versioned file, not in Slack. When a new failure mode appears, add it as a shot and as an eval case. A prompt that cannot be tested is a mood board. Redact names and ids. Next lesson: which shots to keep.

Note:Examples are leaked production data. Redact ticket IDs and names. The model will also imitate mistakes in your shots.