Metrics That Matter

After you fit, you evaluate. The wrong metric ships the wrong agent.

Accuracy is “how often did we match the label?” It is fine when classes are balanced and errors are equal. It is a trap when 99% of events are “not fraud,” “not jailbreak,” or “not the billing tool.” A model that always says “no” is 99% accurate and 0% useful.

You already saw majority-class baselines. This page names the four counts behind that trap, then the ratios product people argue about. Write the counts. Ratios without counts hide small samples: precision 1.0 on two alarms is not a safety story.

The four counts

Pick a positive class you care about catching (fraud, urgent, “needs human”). The other class is negative. If you have eight tools, you either pick one vs rest, or you look at a confusion matrix with eight rows and eight columns. Start with one vs rest for the dangerous class.

• Precision = TP / (TP + FP) — of the alarms, how many were real?
• Recall = TP / (TP + FN) — of the real cases, how many did we catch?
• F1 = harmonic mean of precision and recall — a compromise, not a religion
• Accuracy = (TP + TN) / all — can be high from TN alone

High recall, low precision: the agent escalates everything; humans drown. High precision, low recall: the agent almost never escalates; fires burn.

Division by zero: if you raised zero alarms, precision is undefined (we print 0.0 with a comment that there were no alarms). If there were no real positives, recall is undefined. Do not paper over empty slices with a pretty F1.

Macro-F1 averages per-class F1. Use it when rare tools matter. Micro-F1 is closer to overall accuracy. If you only report micro, the common tool hides the rare one. If you only report macro, one empty class can dominate. Report the matrix.

Run to execute this in your browser. Nothing is sent to a server.

What printed: TP 2, FP 1, TN 6, FN 1 on ten tickets (three real positives; we caught two, missed one, raised one false alarm). Accuracy is 0.8. Precision is 2/3, recall is 2/3, F1 matches that. Always-no accuracy is 0.7 (the seven true negatives) and recall is 0.0 — it never caught a “needs human.” Always-no looks strong on accuracy and is a safety disaster if the positive class is “this action is irreversible.”

Read the four counts before the ratios. FN=1 is a missed human. FP=1 is an extra interruption. Product and safety will not value those equally.

Slices beat averages

Averages hide slices. Report metrics by tool, by user tier, by language, by time of day. The agent that is great on English docs and random on logs will look “fine” in a single number.

A confusion matrix that is “mostly the diagonal” can still hide that sql is always confused with search. That off-diagonal cell is the next feature you add, or the next rubric you clarify.

For ranking (later), accuracy on “is this chunk relevant?” is the wrong family. You need precision at k. Do not average the two jobs into one F1 and hope.

Costs are the missing column

Precision and recall are not morals. They are ratios. Attach dollars or minutes: FP costs a human 5 minutes; FN costs an incident. F1 assumes those hurts are similar. Often they are not. Write the costs. Pick a cutoff on validation (next page). Do not let F1 pick for you in silence.

A three-tool matrix, and what to publish

Suppose tools are search, sql, finish. Accuracy can be 80% while every sql ticket is routed to search. The matrix would show a fat off-diagonal. That cell is the next feature, or a rubric bug (“revenue” tickets that also need a doc). Publish the matrix, not only a single F1.

For a dangerous class (ask_human, shell, jailbreak), publish that class’s precision, recall, support (how many real positives were in the slice), and the four counts. Support 3 means the recall is a coin flip no matter how many decimals you print. Get more positives before you argue.

False discovery is another name for 1 − precision: of the alarms, how many were junk. Safety teams often want that number next to recall. Miss rate is 1 − recall. Write the name the on-call uses.

Do not average English-language tickets with logs if those are different products. A slice table: language, tool, user tier, “has retrieval hit.” The agent that is “fine overall” and terrible on logs is how incidents start.

Metrics are not losses. You can select a checkpoint on recall@human while you train on cross-entropy. You cannot take a slope through a confusion matrix easily. That is fine. Training and reporting are different jobs. This page is reporting. Be boring and complete: counts, then ratios, then slices, then costs.

Common mistakes

• Shipping on accuracy with 99% negatives.
• Precision on a slice of size 3, reported to three decimals.
• One F1 for eight tools with no matrix.
• Using F1 as the training loss.
• Comparing models with different positive-class definitions.

How agents use this

Put the confusion matrix on the routing classifier that decides ask_human. Product will ask for fewer interruptions (precision). Safety will ask to never miss a dangerous action (recall). That disagreement is the job. Write the costs down; do not average them into an F1 and hope.

The same four counts apply to “forbidden tool called” vs not, “citation present” vs not, “schema valid” vs not. Name the positive class. Count. Then argue.

Note:Macro-F1 averages per-class F1. Use it when rare tools matter. Micro-F1 is closer to overall accuracy.