Citations and Faithfulness

Faithfulness means claims in the answer are supported by retrieved evidence. Fluency is free. Models are trained to write smooth sentences. Faithfulness is the product: a support bot that sounds sure while contradicting the handbook is worse than a slow bot.

A citation is a pointer: source id, and ideally a span (quoted substring) that actually appears in that source. “According to our docs” is not a citation. [billing.md] plus a quote the UI can highlight is. Offsets from the overlap-and-ids lesson are how the highlight works. This lesson is whether the model’s footnote is true.

How to force spans

Ask for structured output, for example JSON: an answer plus citations as a list of {source, quote} objects. Then your code checks that the quote appears in the source (normalize whitespace, compare case-insensitively if you must). If not, drop the citation or regenerate. Models love to invent footnotes. They also love to cite the right file and quote a sentence that is not in it.

The check is a substring (after normalize), not “looks related.” Cosine between quote and source is how fake quotes sneak through. in is the right primitive.

Ban citations to documents that were not in the retrieved (and packed) set. Models “cite” filenames they saw in the system prompt, in tool lists, or in earlier turns. If runbook.md was not in this request’s evidence, it is not a legal source.

Attribution vs grounding

Two words people mix up:

• Grounding: the claim is in the evidence (the answer is faithful to the chunks)
• Attribution: you pointed at the evidence (the user can open it)

You need both. An answer can be true from pretraining and still unfaithful to the corpus you promised. In a support agent, unfaithful-but-true is still a policy bug if it contradicts the runbook — or if it invents a procedure that happens to match industry folklore. You sold “answers from the handbook.” Pretraining is not the handbook.

Partial citation is how a lie sneaks through: the first clause is quoted, the second clause is a cash refund. Multi-claim answers need multiple spans. If a sentence has two facts, you need two quotes or you fail the sentence.

What the UI must do

The UI should highlight the span. If you cannot highlight it, do not show a superscript. A dead footnote trains users to ignore all footnotes. If the source moved (checksum changed), fail the citation rather than highlight a random paragraph.

Run to execute this in your browser. Nothing is sent to a server.

The good answer passes: the quote is a substring of a retrieved source. The bad answer fails because “cash today instantly” is not in billing.md — the model invented a footnote. The sneak fails because runbook.md was not in RETRIEVED, even though the quote would have matched the file on disk. That in check is worth more than “please cite” in a prompt.

The cash-claim heuristic is a tiny extra: it is not a full faithfulness model. It shows that code can catch a class of policy lies. Expand it for your corpus. Do not replace the substring check with only a heuristic.

Evals

Faithfulness evals can start without a judge model: quote-in-source rate, citation-to-unretrieved rate, and (later) refuse rate. The Evals track will add judges. You already have unit tests.

Spans, multi-claim answers, and the UI contract

Normalize before in: collapse whitespace, maybe lowercase. Do not strip all punctuation if the gold quote is “5-7 days.” Do not “fuzzy match” with cosine on the quote; that is how invented footnotes pass.

Multi-claim: “Refunds take 5-7 days and we can pay cash today” needs two checks. The first clause may quote billing.md. The second is a lie. A single citation on the first clause must not bless the sentence. Split claims in the schema (list of {claim, source, quote}) or fail the whole answer if any claim lacks a span.

Unfaithful-but-true: the model knows Hamlet from pretraining while the product promised the handbook. If Hamlet is not in the packed set, either refuse or answer without a handbook citation — and do not put a fake [billing.md] on it. Support bots should refuse off-corpus procedures even when folklore is accurate.

UI: highlight the span; if checksum mismatch, hide the superscript. Dead footnotes train users to ignore live ones. Page number plus quote is enough when byte offsets are painful (PDFs). The locator must round-trip like the overlap lesson.

Ban system-prompt filenames. If the instructions mention runbook.md as an example, the model will cite it. Keep examples out of the instruction text, or use obviously fake names.

Common mistakes

• Superscripts the UI cannot open.
• Accepting a filename without a quote.
• Allowing quotes from the whole corpus, not from this request’s packed set.
• One citation for a three-claim paragraph.

How agents use this

After generate, run the checker before the user sees the message. Drop bad citations. If all citations fail, you may refuse or regenerate once. Do not silently ship a fluent paragraph with fake footnotes.

The retrieve library must pass the packed source map into the checker. The agent loop (later) should treat a failed citation check like a failed schema parse: not a personality problem, a bad output.

Stay in lane: this is grounding the answer in chunks. It is not the think-act loop. It is the difference between a search demo and a product.

Highlight the span or hide the superscript. Multi-claim answers need multiple quotes. Unretrieved filenames are illegal sources. Code checks in, not vibes. Faithfulness is the product; fluency was always free. A footnote the UI cannot open trains people to ignore every footnote.