Nearest Neighbors

k-nearest neighbors (k-NN) is the simplest supervised model that uses geometry.

1. Store all training points with their labels
2. For a new point, find the k closest training points
3. Predict the majority label (or average, for a number)

There is no training loop. Fit is “remember the table.” Cost is at query time: compare to every stored point (or use an index).

Retrieval is k-NN without the vote: you return the neighbors themselves (the chunks) instead of their labels. A tool router can do the vote: nearest labeled utterances decide search vs sql.

k is a hyperparameter. k = 1 memorizes. Large k smooths, then it starts mixing unrelated neighborhoods. Pick k on validation. Odd k avoids 50/50 ties for two classes; ties still happen with more classes — define a rule (first neighbor wins, or abstain).

Distance is a policy

Euclidean distance (straight-line in the list of numbers) is the default in textbooks. Cosine distance (1 minus cosine, or sort by cosine) is the default for embeddings. They are not the same if norms vary. Pick one, freeze it, eval it.

Distance must be in a space where “close” means “same job.” Raw token counts of different scales will not. Unscaled latency next to a 0/1 flag will not. Embeddings usually will, if they are from one model.

k-NN does not invent features. Garbage geometry in, garbage vote out.

Run to execute this in your browser. Nothing is sent to a server.

What printed: docs question votes [0, 0, 0] -> search — the point sits in the search blob; three neighbors vote 0. revenue question votes [1, 1, 1] -> sql — the other blob. Three neighbors vote. That is a router. No gradient. A table and a distance.

If you set k = 1, a single mislabeled neighbor can flip the call. If you set k = 6 on six points, you always vote the global majority. Middle values are the product.

Cost and indexes

Brute force is: for every query, loop every stored point. Fine for thousands. Painful for millions. Production retrieval uses an index (approximate neighbors). Approximate means you might miss the true nearest. Measure P@k on a freeze through the index you ship, not through a Python loop that only exists in the notebook.

Adding a point is easy (append). There is no “retrain.” There is stale geometry if you change the encoder and forget to rebuild. There is leakage if the table contains test conversations.

Few-shot is k-NN in English

Few-shot prompting is k-NN in disguise: you stuff the nearest labeled examples into the prompt and let the LLM vote in English. The quality still hangs on which neighbors you picked. Log them. If the nearest labeled traces are from a different product, the vote is noise.

The LLM can disagree with the majority of neighbors. Then you have two policies. Log both. If they disagree often, your geometry and your generator are not the same router.

Votes, ties, and a growing table

A weighted vote (closer neighbors count more) is a small upgrade. It is still k-NN. It still needs a validation k. Ties should abstain on a dangerous class rather than break toward shell. Write that rule.

In high dimension, distances bunch: many points look similarly far. Embeddings still work because they were trained so that meaning sits in angles, but you should not assume Euclidean k-NN on raw bag-of-words with 10,000 slots is a good router. That space is where everything is far. Use a small bag, or an embedding, then k-NN.

The table grows. Every labeled trace you add is more memory and more query cost unless you index. It is also more chance of leakage if you add test conversations. Treat the table like a dataset: version it, freeze ids that must not be in it, delete stale labels when policy changes. k-NN has no epoch, but it has stale memory. A wrong label stays until a human removes it. That is a feature (you can fix one row) and a bug (nobody removes rows).

For retrieval, do not vote. Return the neighbors. For routing, vote. Mixing those jobs — stuffing 20 neighbors into a prompt and also taking majority of their labels — can be fine if you log which neighbor won. It is two systems. Measure both: P@k of the retrieve, accuracy of the vote.

k = 1 on duplicate tickets is a cheat: the nearest neighbor is the same ticket. Dedup by conversation id before you celebrate a 99% router.

Query time is the bill. Brute force k-NN that was fine on 2,000 labeled utterances will hurt at 200,000. Then you add an index, and the index is approximate, and you must re-measure P@k or vote accuracy through that index. A notebook loop is not the system you ship. Log neighbor ids in production so you can see when the index skipped a true neighbor that the notebook would have found.

Common mistakes

• k = 1 on a table full of duplicates of the test ticket.
• Distance in unscaled raw counts.
• Rebuilding the table from traces that include the eval.
• Measuring brute-force P@k and shipping an approximate index.
• Forgetting that fit is store: stale labels stay until you delete them.

How agents use this

For retrieval, k is top-k chunks. For a router, k is how many labeled neighbors vote. Both are validation knobs. Memory systems that “find similar tickets” are this page. Evaluate them as ranking (P@k) or as classification (vote), depending on whether you return the neighbors or you return a label.

When the table is small and labels are clean, k-NN is a strong baseline. Beat it on a freeze before you train a net. Many agent routers should stay here.

Tip:For retrieval, k is top-k chunks. For a router, k is how many labeled neighbors vote. Both are validation knobs.