Finish Reason

Every completion ends for a reason. If you ignore it, truncated JSON looks like a stupid model, and a filter looks like an empty bug. The vendor tells you why it stopped. Believe that field before you swap networks.

Reserve space before you call: window - prompt_tokens - margin >= max_tokens. If that fails, shrink the prompt first. Raising max_tokens when the window is already full just fails in a different way (or the vendor truncates the prompt, which is worse: your spec falls off the front).

n=3 completions triple the output bill. Use them for offline eval, not the hot path, unless you have a measured win. Finish reason applies to each choice if you sample many.

Length is a budget, not a personality

A cut-off {"action": "get_job", "job_id": 1 is not a model that “doesn’t know JSON.” It is a cap. First: treat as truncated. Then: raise max_tokens or shrink what you asked for (shorter reasoning, smaller schema, less chatter in the spec). Then parse again. Do not lower yourself into regex salvage on the fragment.

If the vendor supports parallel tool calls, validate each argument object. One valid call plus one malformed call is not a pass. Partial application is how you refund the customer and fail to log it.

Run to execute this in your browser. Nothing is sent to a server.

The first call hits the word budget before a closing brace: reason is length, valid is False. The second hits the stop word END with a complete object: stop and valid True. Always branch on finish reason before you blame weights. The first call looks like a broken model. It is a budget.

Stop sequences cut off when the model starts imitating your scaffold (User:, END). They are useful. They can also clip JSON if you chose a stop that appears inside a string. Pick stops that cannot appear in arguments.

Window math before you call

You need three numbers: context window, prompt tokens (estimate or tokenizer), max_tokens. Leave a margin (a few hundred tokens) so the vendor does not silently drop the end of the prompt. If prompt + max_tokens + margin > window, shrink the prompt in build_context (later packing lesson) before you raise max_tokens. Raising the completion budget into a full window is how the spec falls off.

Tool steps should use a small max_tokens (a few hundred). Final prose can be larger. A global 4096 is a latency and cost bug that also hides length errors on JSON (the object fits, then the model keeps talking until the cap, and you parse a truncated essay after the JSON). Prefer stop-after-object via native tools or a stop sequence.

If finish_reason is missing (some stream disconnects), treat it as unknown: do not parse as complete JSON unless you have a valid object and you are willing to accept a truncated tail. Default: truncated.

Parallel tool calls: finish_reason=tool_calls plus three argument objects. Validate each. If the second is malformed, do not execute the first if they were meant as a transaction. If they are independent reads, you may run the valid reads and return an error payload for the broken one — write that policy down. Default for money: all-or-nothing.

Dashboards

Store finish_reason on the trace. Dashboard: percent length, percent filter, percent stop, percent tool_calls. A spike in length is a packing or cap bug. A spike in filter is a prompt or user-mix bug. Neither is “we need a bigger model” until you check.

If tool_calls arrives with empty arguments, that is still a parse problem, not a reason to read the prose channel.

What goes wrong

• Parsing length-truncated JSON and filling defaults.
• Ignoring tool_calls and reading “I’ll call get_job” from content.
• Raising max_tokens to 4096 “just in case,” then paying for novels (decoding knobs next).
• No margin between prompt size and window, so the vendor silently drops the spec.
• Blaming the model on a day the length rate jumped after someone added a 2k few-shot.
• Parsing stop JSON that is missing a closing brace because “it is close enough.” Close enough is a guess.

How agents use this

handle_response (previous lesson) already branched filter vs length vs ok. Wire length to a single retry with more room or a packer that drops RAG, then give up. Never retry length seven times with the same prompt.

Tests: truncated fixture must not call a tool; complete JSON with stop may. Store the reason on the span next to usage. If parallel tool_calls arrive, fail the step if any argument object fails validation — do not apply the valid ones and skip the broken one when the broken one was refund.

A length spike after a spec PR is packing, not a new vendor. Check max_tokens and prompt size on the same dashboard before you file a model bug.

Reserve window margin in complete() so you never send a prompt that cannot also hold the completion. If the packer cannot make it fit, fail closed — do not POST and hope.

Note:n=3 triples the output bill. Use it for offline eval, not the hot path, unless you measured a win. Score with your validator, not with another LLM judge, unless you have measured the judge.