Refuse When Nothing Matches

If nothing came back (or everything is below threshold), the faithful answer is “I don’t know; nothing in the corpus matched.” That sentence is not a failed product. It is the product keeping its promise: answers from the handbook.

Inventing a procedure from pretraining, while the UI promised the handbook, is a lie even when the procedure happens to be common sense. “Raise memory on OOM” might be true in general and still wrong for this company if the runbook says “page the on-call, do not touch limits.” Helpful hallucinations are product bugs.

Return a structured refuse, not a vibes paragraph:

{"refused": true, "reason": "no_hit", "score": 0.11}

Then the UI can offer a human, a ticket, or a tool (lookup by id) — not a fake runbook. reason should be a small enum: no_hit, below_threshold, empty_index, filtered_empty (tenant filter left zero chunks). Those are different ops problems. One user-facing sentence can still be “I don’t have that in the handbook.”

Empty retrieve is a success state

Do not retry retrieve 40 times with random rewrites because the first call was empty. One rewrite (next lesson) is a policy. A loop of hope is a bill. Do not lower tau to 0 for this user. Do not fetch the public web because the handbook was empty unless that is an explicit, authorized mode.

Empty index (ingest down) should look different in traces from empty match (ingest healthy, query is horses). The user might see the same refuse. On-call must not.

Eval both directions

Questions with no gold chunk must refuse. Questions with a gold chunk must not refuse. Both directions matter. A timid bot that refuses everything scores well on safety and fails the job. A bold bot that never refuses scores well on demos and ships cash refunds.

Keep a golden set with refuse: true rows. The recall@k lesson will wire this. Here, the retrieve function returns empty or a flag, and generate is not allowed to “just help.”

Run to execute this in your browser. Nothing is sent to a server.

Refunds hit: overlap is at least tau, so the answer is the chunk plus a citation. Horses do not: score 0, refuse. The second call does not guess a veterinary procedure. That is the whole lesson in two prints.

If you set tau=0, horses get the refund policy as “least bad.” Try it. That is always-k from the threshold lesson, showing up as a fake handbook quote.

What refuse is not

Refuse is not rudeness. You can still be kind in the user-facing string. The structure is for the UI and for evals. Refuse is not “I am an AI and cannot…” that then pastes a procedure. If refused is true, the answer field must not contain a how-to from weights.

Refuse is not a substitute for a tool. If the user pasted INV-17, empty wiki retrieve should not be the end if get_invoice exists. Routing to tools is the later Agents track. This track’s job: do not turn empty retrieve into a generated runbook.

Reasons, retries, and both eval directions

no_hit means scores below tau (or empty after filters). empty_index means ingest produced zero chunks for this tenant/collection — page someone. filtered_empty means the pool was nonempty globally but empty after tenant or ACL — still not a reason to search Globex. Log the reason. The user-facing string can stay kind and short.

Retries: one query rewrite is a policy you can eval. Five paraphrases until something crosses tau is how 0.12 chunks become procedures. Do not lower tau for this user. Do not open the public web because the handbook was empty unless that mode is explicit and wrapped.

Both directions: a bot that refuses all gold questions is not “safe RAG.” It is a closed door. Your golden set needs refuse: true rows and must-answer rows. CI fails if either side moves past a floor.

Structured refuse belongs in the assembler before generate-on-evidence. If you still call the model with empty DATA, you are asking it to help. Canned honest text plus refused: true is cheaper and testable.

Empty retrieve is success for the retriever. It is not success for ingest if the index is dead. Dashboards must split those.

Common mistakes

• Empty retrieve → closed-book “helpful” how-to.
• One retry storm that eventually finds a 0.12 chunk.
• User message “I don’t know” without refused: true, so evals cannot count it.
• Refusing gold questions because tau was copied from another model.

How agents use this

Treat refuse as a result, not as a personality. The retrieve library returns hits: [] and a reason. The assembler should not call generate-on-evidence. It should emit the structured refuse (or a canned honest string).

Eval it in CI. The agent loop later will decide whether to try a tool instead. That decision is not “generate a runbook from weights.” If you skip this lesson, the loop will only hallucinate with more steps.

Expose reason to the UI so “handbook has no match” can offer a ticket, while empty_index pages on-call. Do not retry until tau is satisfied. One rewrite is a measured policy; a storm of paraphrases is a bill. Keep the user-facing sentence kind. Keep the JSON strict. Questions with gold chunks must still answer; a timid always-refuse bot is not faithful, it is closed. Both floors belong in the same eval job as recall@k. Empty retrieve is honest. Helpful invention from pretraining is a lie you sold as the handbook. Structured refuse is for the UI and for CI. Canned kind text is for the human. Do not call generate-on-evidence when hits are empty. Do not lower tau for this one user.