When to Abstain

Abstain means: say you do not know (or cannot act), instead of guessing. Product teams hate this until the first legal review. Then they hate the agent that invented a balance.

For agents:

• Abstain after tools, not instead of them. “I didn’t look” is laziness.
• “I looked, the tool 404’d, I will not invent a balance” is professionalism.
• World facts (CEO as of today) without search → abstain (or search, then maybe still abstain).
• Missing job_id after one ask → stop. Do not pick a popular id.
• Contradictory tools → abstain or handoff, do not average two balances.

Forcing an answer with “you must reply” in the spec fights abstention. Put a legal third option in the schema: {"action": "abstain", "reason": "..."}. If the only legal actions are get_job and finish with a required number, you trained a guesser.

Score it like a skill

Score abstaining higher than a stylish wrong number on your eval. If every test item is answerable, you will never train that muscle. Build a slice: empty tools, missing ids, out-of-corpus questions, stale docs that do not contain the amount. The correct action is abstain or handoff.

UI: a yellow “not in tools or docs” is better than a green wrong refund window. Gate send on grounded or abstain. Humans can still override. The model should not.

Run to execute this in your browser. Nothing is sent to a server.

Four rows: need search before answering a world fact; tool 404; missing id; all clear. The last is the only ok. Notice the first row abstains even though a job_id exists — the question still needed search and search never ran. That is “after tools, not instead of them” encoded as: if you needed a tool, you must have used it. The 404 row used a tool and got nothing: still abstain, do not quote a number.

You can implement this in code without asking the model. That is the best abstain. The schema option is for cases the rules do not cover. Rules first; model abstain second; guess never.

Ask, abstain, handoff

Ask once when a required field is missing (job_id). Track asked_id=true in code state so the second turn cannot become an infinite interview. Abstain when you looked and the world did not answer (404, empty search, contradictory tools). Handoff when money, safety, or a timeout on a write is involved. Do not loop “just one more search” until the spend cap fires — that cap is a backstop, not a plan.

Retrying a 404 with a different customer id is not cleverness. It is accessing the wrong person. Retrying search with the same query is a cost leak. Retrying search with a reformulated query can be legal if you cap at one extra and you still abstain when empty.

Schema: action enum includes abstain with a short reason enum (missing_id, tool_empty, not_in_docs, contradiction). Free-text reasons are for traces, not for a model to invent a legal theory. The UI maps those enums to yellow copy. Do not let the model write the yellow copy if legal needs a template.

Forcing “you must reply with an answer” in the spec fights this whole lesson. Delete that sentence. If a vendor’s JSON mode requires a field, make the field final nullable or use the abstain action.

The loop should call tools when should_abstain says need search. That return value means “do not emit a final fact yet,” not “give up.” Only tool empty, missing id after one ask, and contradictions are user-visible abstain. Mixing those cases in one boolean without a reason code is how you skip search and look lazy.

Eval the impossible slice every time the spec changes. If abstain rate on that slice drops and accuracy on answerable items rises, you probably trained a guesser. Product will cheer until legal reads a made-up balance. Put abstain rate on the same dashboard as JSON-ok.

What goes wrong

• Schema without abstain, plus “always answer” in the spec.
• Eval with only answerable items.
• Green UI on ungrounded text.
• Picking a default id to “keep moving.”
• Abstaining instead of calling a cheap tool (laziness). The first case in the tryit would be wrong if you skipped search and still answered. The function forces the search gap to look like abstain-from-final, which your loop should turn into call search, not into a user-facing “I don’t know” until search ran.

Clarify in the product: should_abstain in the toy is “do not emit a final fact.” The agent may still call a tool. Final answers abstain; intermediate steps act.

How agents use this

Put abstain in the action enum. Put should_abstain(state) in code for 404, missing ids, and unused required tools. Eval the impossible slice every time you change the spec. UI copy that is allowed to be yellow.

Wire the reason code to the UI template. Support should see tool_empty on the trace, not a model essay about why balances are hard. Tests: 404 → abstain; missing id and not yet asked → ask; missing id and already asked → stop; needs search and unused → call search, not final prose.

Do not retry a 404 with a neighbor id. Do not raise temperature to “get an answer anyway.” The schema already has abstain. Use it. Score it on the eval as a success when the item was impossible.

Note:Forcing an answer with “you must reply” fights abstention. Put {"action": "abstain", "reason": "..."} in the schema. Empty tools are not a license to guess money.