When the Eval Lies

An eval can lie. If you cannot see the lie, you cannot align the agent. This lesson is a checklist of measurement failure modes you already have names for, composed into a trust() report. Oversight is sampling traces, reviewing HITL denies, and updating goldens after incidents — not staring at a single green float.

Lies you have already met:

If you cannot see it, you cannot align it. A dashboard that cannot print n, injection count, quarantine share, and whether the actor graded itself is not an eval dashboard. It is a mood.

Next track: production — gateway, workers, queues, secret redaction, CI that runs this suite, kill switches, and incidents that end in a golden. This lesson does not teach those systems. It tells you which numbers they should refuse to treat as success.

A trust function is a spec for the report

trust(report) is not the agent’s pass rate. It is whether you should believe the pass rate.

• n = 0 → empty_suite (even if someone stuffed pass = 1.0)
• self_judge → fox_henhouse
• injection_n < 5 → no_injection (your floor; pick a number and write it down)
• quarantine_share > 0.2 → too_much_quarantine

A real slice with n = 80, injection_n = 15, quarantine 0.05, not self-judged → trusted. You can still have a low pass rate. Trust means “this exam was real,” not “the student scored 100.”

Add your own fails as you grow: missing safety tag, judge FN above bar, online flags not reviewed, gold version older than the KB. Keep the function boring. Print the fail list next to the pretty chart.

Walkthrough: three reports

1. n: 0, pass: 1.0 → untrusted, empty_suite. The classic lie.
2. n: 80, self_judge: True, injection_n: 20 → untrusted, fox_henhouse. Plenty of cases; the fox graded the henhouse.
3. n: 80, injection_n: 15, quarantine_share: 0.05 → trusted. Now you may discuss the pass rate as a product number.

Acme’s weekly meeting should not start with “we’re 96%.” It should start with trust then slices: billing, safety, injection, forbid-tool count, cost per pass, online reopen/override. If trust fails, the 96% is branding.

Oversight is part of trust. Sampling traces, reading HITL denies, and filing goldens after incidents are how the numbers stay honest. A trusted report with nobody watching HITL is still a process fail; you can add hitl_denies_reviewed to trust() when you have the event. Until the production track exists, do not pretend a green float is a factory.

CI, when it exists, should consume this suite: nonempty n, floors met, quarantine under cap, no self-judge, gold version pinned. Kill switches should be allowed to use forbid-tool spikes and leak hits, not CSAT. This lesson names those uses so you do not build production around a lying dashboard. It does not teach queues or gateways.

If trust fails, you do not get to quote pass rate in a launch review. That social rule is the last measurement: the exam of the exam. Empty 100% is the cartoon. Fox-and-henhouse and no-injection are the adult versions. You now have the vocabulary to refuse all three.

Run to execute this in your browser. Nothing is sent to a server.

What printed: empty 100% is untrusted with empty_suite. Self-judge is untrusted with fox_henhouse. A real slice with injection coverage is trust True and an empty fail list. Wire this next to pass_rate, not instead of it.

What goes wrong if you skip this

You will ship on a lying dashboard. Production CI will be told to “wait until evals exist” forever, or it will gate on 100% of zero. Kill switches will have no honest signal. Incidents will end in pep talks. The rest of this track becomes a museum of good ideas nobody believed because the number was always green.

Stale gold, essay equality, and chatbot metrics on agents are lies too even when n is large. trust() is the start of a checklist, not the end. Add fails when you learn a new way the report misleads. The production track can fail a build on trust: False. Until that exists, the social rule still holds: do not quote an untrusted 96% in a launch review, and do not skip the suite because the GIF looked good.

Fox-and-henhouse includes a judge that is the same checkpoint as the actor, and a rubric the actor’s prompt authors wrote while looking at holdout. Blind names. Split owners. If injection_n is high but every row is the same PDF, coverage lied — trust() can require distinct stories later. Start with n, self-judge, injection_n, quarantine share. Raise the bar as the product grows.

How agents use this

You now have side-effect scores, golden properties, tool unit tests, judges you eval, replay, harm probes, injection allow-lists, and a spec that can win over the prior.

Treat trust(report) as product code in the runner that publishes the weekly slice. Fail the report, not only the agent. When production CI exists, it should refuse to ship an untrusted suite. Until then, you still refuse to brag.

Multi-agent: trust per role. A trusted supervisor report with an empty worker suite is an empty suite.

The measurement lane ends here. Ship the numbers into the production track: same trace schema, redaction, queues, and a build that runs these goldens.