Query Rewrite

User language is sloppy. Indexes are literal. Query rewrite turns a chat line into search terms (and sometimes into several sub-queries). Do this before retrieve, as a cheap step. It is not a second personality. It is a string transform you can unit-test.

Typical rewrites:

• Expand acronyms you know (OOM → out of memory, plus your product words like worker)
• Drop filler (“please”, “that thing from last week”)
• Split multi-part asks into two retrieves, then merge (MMR or round-robin)
• Keep identifiers verbatim (INV-17 must survive)

A synonym table for your product beats a 40-page thesaurus. Thesauruses introduce neighbors you do not want (“refund” → “rebate” → the wrong policy). Eval rewrite as its own stage: did the gold chunk enter the pile after rewrite, not before?

Cap rewrite + retrieve as one billed hop in traces even if two functions ran. Otherwise cost dashboards lie.

HyDE is a cousin, not a source

HyDE (Hypothetical Document Embeddings) is a cousin: the model writes a fake paragraph, you embed that, then search. It can help paraphrase when the user is vague. It can also search for a hallucinated procedure. Treat HyDE as optional, measured, and never as a source you cite. The fake paragraph does not go in the data block. Only real chunks do.

If HyDE invents “instant cash,” you may retrieve the cash-rumor blog. That is a reason to keep keyword pins and thresholds.

Do not rewrite away the id

If the user typed INV-17, the rewritten query must still contain INV-17. Rewriters love to “help” by expanding to “invoice seventeen” and then hybrid search misses the token. Write a test. Same for error codes and file names.

Run to execute this in your browser. Nothing is sent to a server.

The raw query barely overlaps: “that” and “thing” were dropped in spirit but still, OOM vs the runbook’s tokens is thin if you only split on spaces and the runbook uses OOM:. After rewrite, memory and worker hit the runbook. Print both scores. The first number should be lower. That is the rewrite argument.

This synonym table is tiny on purpose. Add terms from your missed queries, not from a dictionary crawl.

Multi-query

“Refunds for INV-17 and the OOM runbook” is two retrieves. Rewrite can emit a list of queries. Fuse with RRF or pack with MMR. Two queries that each return 10 chunks still need a budget. Do not turn one user sentence into ten searches without a cap.

Synonyms, HyDE, and what must not change

Keep identifiers verbatim. A rewriter that turns INV-17 into “invoice seventeen” is a bug with a unit test. Same for error codes, CVE ids, and filenames.

Product synonym tables beat general thesauruses. Add terms from missed gold questions. “OOM” → worker, memory, runner is yours. “Refund” → rebate, cashback is how you retrieve the wrong policy.

HyDE: embed a fake paragraph, search, discard the paragraph. Never put it in DATA. Never cite it. Measure whether gold recall rises. If it rises by retrieving rumor blogs, turn it off.

Log the raw user string and the rewritten string. Without both, you cannot tell rewrite bugs from index bugs. Bill rewrite + retrieve as one hop so dashboards match cost.

Stopwords: dropping “that thing from last week” helps. Dropping “never” does not — negation is already hard. Be conservative with stop lists.

Rewrite is a retrieve booster. It is not memory write-back. It is not a new instruction. The chunks that come back still get wrapped.

Common mistakes

• Rewriting ids into words.
• Citing a HyDE paragraph.
• A thesaurus that adds 50 terms and retrieves the whole index.
• Not logging the rewritten string, so you cannot see why retrieve changed.

How agents use this

Keep a synonym table for your product. Eval rewrite as its own stage. Cap rewrite + retrieve as one billed hop.

In a later loop, the model might propose a rewritten query as an argument to retrieve. Your code should still apply stopword and id-preservation rules. Untrusted rewrite is still a string into search, not a new instruction. Wrap the chunks, not the query — but do log the query.

Stay in search. Rewrite is a retrieve booster. It is not memory, not tools, not the agent loop.

Unit-test: raw “that OOM thing” misses gold; rewritten string hits. Unit-test: INV-17 still present after rewrite. HyDE off by default until a labeled slice shows recall up without rumor chunks. Multi-query split needs MMR or RRF on the way back in, plus a cap of two or three queries, not ten. Store both strings on every eval row so a miss is diagnosable. Acronym expansion belongs in your table, not in a general dictionary that maps refund to rebate.

Rewrite before retrieve, as a cheap string step you can print. Do not treat the rewritten line as a source to cite. Do not drop “never.” Do not expand SKUs into words. A 40-page thesaurus is how every query retrieves the whole index. Grow the table from missed gold questions only. Cap rewrite plus retrieve as one billed hop so cost dashboards stay honest.

Users do not speak in heading language. Indexes do. The gap is this lesson. If gold still misses after rewrite, the next knobs are hybrid, hops, or chunking — not a longer thesaurus. HyDE remains optional and never cited. Identifiers remain verbatim. Log both strings. The live box is a tiny table on purpose: add only the acronyms you missed in eval, then stop. Users said “that OOM thing.” The index wanted worker, memory, runner. That translation is rewrite. It is not a second corpus and it is not a citation.