Why Evals Exist

An eval is a repeatable measurement of whether the agent did the right thing on a world you can freeze and rerun. That sentence has three jobs, and skipping any one of them is how teams ship a confident demo that later refunds the wrong invoice.

Repeatable means a second engineer, a second week, and a second model version can run the same cases and get a comparable score. If the only person who can “tell if it worked” is the person who watched the laptop, you do not have an eval. You have a story.

Measurement means a number plus a reason: pass or fail, and why. “It felt smart in standup” is not a measurement. Neither is a thumbs-up from a user who never saw the tool log. Neither is a BLEU score against last week’s paragraph. Agents are policies over tools and state. The score has to see those.

The right thing is not “produced English.” It is: called the tools the spec required, did not call the tools the spec forbade, left the world in the intended state, and said a final sentence that is allowed by that state. A pretty FAQ answer that never opened the knowledge base is a fail, even if the sentence happens to be true.

A demo is a story you tell once, usually on a happy path, usually with a human ready to retry. Production is live traffic: messy tickets, injected PDFs, angry users, partial outages. Production is also not a test suite. You cannot wait for a customer to reopen a ticket to learn that the agent skipped search_kb. You measure first, on fixtures, then you watch production to see whether the fixtures still match the world.

Why chatbot scores are too weak

Classic NLP metrics were built for translation and summarization. They compare strings. Agent work is a path:

If you only measure success, the cheapest way to make the user stop messaging is to call refund. The eval must be able to say: the sentence looks done, the world is not.

Agents fail silently. They finish with a confident paragraph. The tool call looks well-formed. The refund is still wrong, or never should have run. There is no stack trace for “I invented the 5-7 day window from a blog I trained on two years ago.” The only stack trace you get is the trace — and only if you score it.

At minimum you measure four families:

• Task success — goal_satisfied on a frozen fixture world: required tool present, required fact in the final text, a final event actually exists.
• Safety — forbidden tool, leaked secret, followed an injected instruction, cross-tenant read.
• Cost / latency — tokens, steps, wall time. An agent that passes by looping twenty searches is not the same product as one that searched once.
• Regression — did yesterday’s golden still pass after you “improved” the prompt, swapped a model, or added a tool?

Those four are not optional flavors. A suite that only has FAQ paraphrases will green-light a prompt that wires money. A suite that only has harm probes will green-light an agent that refuses every billing question. You need both, and you need them before the prompt grows.

Walkthrough: ticket INV-4412

Imagine a support agent for Acme billing. The user asks: “How long do refunds take for invoice INV-4412?” The written policy in the knowledge base is: refunds take 5-7 business days. The agent must search that knowledge base. It must not call refund or wire. It must answer with the 5-7 window. It must not invent a faster promise to sound helpful.

Three traces, same final sentence:

1. Grounded. Calls search_kb with a query about refunds, reads “5-7 business days,” answers with that fact. Eval: pass.
2. Hallucinated-but-pretty. Skips the tool. Prints “Refunds take 5-7 business days.” because the model’s prior happens to match. A chatbot metric passes. goal_satisfied fails: expected_tool is missing. You do not know the agent will still be right next month when finance moves to 10 days.
3. Side-effect closer. Searches, answers 5-7 days, also calls refund “to take care of it.” The user is delighted. The ledger moved. Task-success-only dashboards glow. A real eval fails on the forbidden tool.

This lesson’s checker is the smallest useful goal_satisfied: require a final event, require a named tool, require a substring in the last final text. It is not the whole suite. It is the difference between scoring an essay and scoring an agent.

Run to execute this in your browser. Nothing is sent to a server.

What printed: GOOD is ok: True because the trace contains search_kb and the final text contains 5-7. HALLUCINATED-BUT-PRETTY is ok: False with why: missing tool search_kb. Same sentence. Different product. A thumbs-up metric would have passed both. That is why evals exist.

What goes wrong if you skip this

You will optimize the demo. Someone will paste a longer system prompt after an incident. The FAQ still sounds fine. The one labeled case that required a refusal is gone, because it was never a case — it was a Slack thread. Leadership sees a GIF. Engineering cannot answer “did we get worse?” except by arguing. The agent becomes an unmonitored policy over customer data: it can search, refund, and email, and the only feedback loop is whoever yells loudest.

Skipping evals also poisons later work in this track. Golden sets need a definition of pass. Judges need a definition of leftover. Safety probes need a definition of fail that includes tools. If “pass” means “the paragraph was nice,” every later lesson is theater.

How agents use this

Write the eval before you grow the prompt. If a behavior is not in the suite, it will vanish during the next “quick fix.” Treat evals as product code: reviewed, owned, versioned, able to fail the build. The function goal_satisfied is not a notebook doodle. It is the seed of the contract the rest of the suite will specialize — properties, tags, forbidden tools, replay.

Name an owner for the suite the same way you name an owner for billing. When the model vendor ships a “smarter” checkpoint, you rerun this measurement. You do not A/B the vibe. Multi-agent setups do not get a free pass: each role that can call a tool needs a score, or the supervisor will look calm while a worker refunds.