Overfitting

Overfitting is when the model fits the training sample, including accidents, instead of the pattern that will appear tomorrow.

The giveaway is a split personality: training loss goes to zero; validation loss goes the wrong way. Accuracy on the demo is 100%. Accuracy on next week’s tickets is a coin flip.

This is not a moral failing of neural nets. It is what a flexible function does when you give it more knobs than independent facts. A 200-line system prompt that enumerates every anecdote from last month is the same picture without a GPU: you fitted the sample.

Noise teaches the wrong lesson

If labels are noisy, a flexible model will learn the noise. If you have more knobs than independent examples, you can interpolate anything. Neural nets are often in that regime. They still generalize if you stop in time and if the data matches production. They do not get a pass on a 12-trace “eval.”

A high-degree curve that hits every training point and goes wild in between is the textbook drawing. Agents do the same when few-shots are the twelve incidents from Slack, or when retrieval always returns the same three docs that happened to sit in the train conversations.

Memorization is overfitting as a dictionary: if this x appeared, return that y; otherwise guess nothing useful. Perfect train, useless val.

Run to execute this in your browser. Nothing is sent to a server.

What printed: the linear rule (predict twice x) has a small train MSE and a small val MSE — it captured the pattern, not the exact noisy train points. The memorizer has train MSE 0.0 and a large val MSE, because validation x never appeared as keys, so it predicts 0.0 every time. That is overfitting as a dictionary. Train looks perfect. Tomorrow looks like a different job.

The linear function is slightly underfit on the noisy train points (it does not hit 6.4 exactly). That is allowed. Slightly underfit and stable beats perfect on twelve examples.

Underfitting is the other side

A model that is too simple (always predict 0, or a line through a curve) is underfit: train and validation are both bad. You want the middle: simple enough to be stable, rich enough to fit the real pattern.

If train is bad and val is bad, add features, add capacity, or fix labels — do not immediately add more epochs. If train is great and val is bad, you already have too much capacity or too little data or leakage that does not transfer. Early stopping (regularization next) is the practical middle: watch val, stop when it turns.

Capacity is not only net width. For agents: more tools, more prompt clauses, more retrieved chunks, more few-shots, more special-case sentences. Each is a knob that can memorize last week.

Contamination

A huge chat model can “overfit” your forty-example eval without a single gradient step if those examples leaked into pretraining, into the system prompt as few-shots, or into the retrieval index. That is contamination, not a polynomial. Treat few-shot examples as training data. They do not belong in the conversations you report as test.

If you iterate the eval until the model passes, you fitted the eval. Rotate items. Hold out users. Keep a slice that nobody is allowed to “fix” except by bumping the dataset version.

How to fight it (preview)

• More independent data (new tickets, not paraphrases of the same five)
• Simpler functions (keywords, linear router)
• Regularization and early stopping (next part)
• Frozen splits and no test peeking
• Features that exist at decision time only

You cannot fight overfitting by staring at train accuracy. The plot you want is two curves: train and validation, versus steps or versus prompt versions.

Two curves, and the fake “we generalized”

Leakage can hide overfitting. If test is a paraphrase of train, both curves look great and next week still dies. Contamination is that plot in disguise: the published exam was in the prompt, in the index, or in pretraining. You did not generalize to the phenomenon. You recognized the sample.

The honest picture is three numbers, not one:

1. Train metric
2. Validation metric on a freeze that did not guide the last fit
3. A fresh slice from a later week (drift, later)

If 1 is great, 2 is great, and 3 is poor, you overfit the freeze (eval overfitting) or the world moved. If 1 is great and 2 is poor, you overfit train. If all three are poor, you underfit or you are answering the wrong question.

Agent-specific overfitting is often policy memorization: the system prompt lists last month’s outage, the few-shots are the same six Slack threads, retrieval always returns the same three docs because those docs were in the train conversations. The model is a lookup table with extra English. Cutting capacity (shorter prompt, fewer tools, smaller k) is a real fix. Adding another anecdote is not.

Stop rules belong with regularization, but the diagnosis belongs here: whenever someone shows 100% on a demo, ask whether the demo ids sit in train, in the prompt, or in the index. If yes, you have not measured generalization. You have measured a mirror.

Common mistakes

• 100% on the demo script, which is also the train file.
• Adding a prompt paragraph per incident, forever.
• Few-shots copied from the test conversations.
• Celebrating train loss of 0.0.
• Calling a memorizing k-NN (k = 1 on duplicate tickets) a “retrieval win.”

How agents use this

Overfit agents quote yesterday’s outage as if it were physics. They retrieve the same three docs for every question. They pass an internal eval that was updated until it passed. Hold out users. Rotate evals. If train traces and the demo script are the same file, you are the memorizer.

A model that is slightly underfit and stable beats a model that is perfect on twelve examples. Ship the stable one. Put the twelve examples in train or in a gallery of bugs, not in the published exam.

Tip:A model that is slightly underfit and stable beats a model that is perfect on twelve examples.