Expectation and Variance

The expected value E[X] is the chance-weighted average of a random variable. For a discrete X that takes values x_i with chances p_i, E[X] = sum p_i x_i. It is not always a value X can take (the expected die roll is 3.5). It is what the average of many independent copies converges to.

Variance is expected squared deviation from the mean: E[(X - mu)^2]. Standard deviation is its square root, in the same units as X. High variance means a single run is a poor guess of the mean — relevant when you quote “the agent costs $0.04 per ticket” from 8 tickets.

Linearity: E[A+B] = E[A]+E[B] even if A and B are dependent. Products do not work that way unless you have independence. If a slow tool is also expensive, ignoring that underprices the bad days.

A useful identity: Var(X) = E[X^2] - (E[X])^2. In code, sum p c for the mean, sum p (c ** 2) for E[X^2], subtract the square of the mean.

A wrong picture

A wrong picture is: “budget the expectation and you will be fine.” Operations wants a high percentile. A few percent of runs hit the human path and cost 5 when the mean is 1.6. If you provision only 1.6, those runs overflow. Finance wants expectation; on-call wants the tail.

Another wrong picture is: “expectation is the typical outcome.” Typical often means the mode (most common). A die’s typical face is not 3.5. An agent’s typical ticket may be “one cheap tool call” while the mean is pulled by rare escalations.

A third: E[total] = E[steps] * E[cost per step] as a law. That needs extra assumptions (uncorrelated number of steps and cost per step, or independence). In practice, measure total cost from traces; use expectation as a model, not as a law of nature. Hard tickets take more steps and more expensive tools.

Evals: the expected pass rate is a Bernoulli mean. Variance of a proportion is p(1-p)/n. Small n means you cannot distinguish 0.80 from 0.88. Ship fewer claims, or gather more tickets.

The formula in words

Expectation: multiply each outcome by its chance, add. That is a weighted average (sums lesson) where the weights are chances.

Variance: how far from the mean, squared, then averaged with the same chances. Std is the square root, back in dollars or tokens.

Tiny numeric. Three outcomes: one call cost 1 with chance 0.70; retry cost 2 with chance 0.20; human cost 5 with chance 0.10.

E[cost] = 0.701 + 0.202 + 0.10*5 = 0.70+0.40+0.50 = 1.60.

E[cost^2] = 0.701 + 0.204 + 0.10*25 = 0.70+0.80+2.50 = 4.00.

Var = 4.00 - 1.60^2 = 4 - 2.56 = 1.44. Std = 1.2.

A few percent? Chance of human is 0.10, cost 5, which is over 3 units. Fraction over 3 is 0.10 here (only the human path). Simulation should match.

Moving parts

Linearity: E[A+B] = E[A]+E[B] even when A and B depend on each other. E[A*B] = E[A]E[B] needs extra assumptions. Hard tickets take more steps and more expensive tools — do not multiply those two means and call it a law.

A second walkthrough (tokens)

Per ticket: 70% cheap (800 tokens), 20% medium (2000), 10% loop (12000).

E[tokens] = 0.70800 + 0.202000 + 0.10*12000 = 560 + 400 + 1200 = 2160.

E[X^2] = 0.70640000 + 0.204000000 + 0.10*144000000 = 448000 + 800000 + 14400000 = 15648000.

Var = 15648000 - 2160^2 = 15648000 - 4665600 = 10982400. Std ≈ 3314 tokens.

The mean is 2160. Ten percent of tickets are 12000 — more than 3 std-ish above a naive “typical” if you forgot the mix is discrete. If you provision only 2160, those loop tickets overflow. Finance wants 2160. On-call wants the 12000 path (or p95).

Zero-cost outcome with chance 0 still needs a row if you might hit it. A missing bucket silently understates the mean.

A Friday ticket

Friday finance provisioned spend from E[cost] = 1.6 units (the tryit). On-call spent the weekend on the 10% human path at 5 units. Mean and tail were both true. The dashboard had only the mean. They added std_tokens, p95_tokens, and fraction_human_path next to expected dollars. Same three-way mix, two audiences.

Price a step

Run to execute this in your browser. Nothing is sent to a server.

E[cost] is 1.6. Var is 1.44, std 1.2. E[dollars] is 0.0032 (1.6 times 0.002). Simulated mean and var should land near 1.6 and 1.44. Fraction over 3 units should land near 0.10 (the human path). The simulation should match the closed form. If you provision only the expectation (~1.6), those 10% runs overflow.

The sampler walks a cumulative sum of chances — the same pattern as token sampling later. random.seed(2) makes the 8000 draws repeatable.

What goes wrong

• Budgeting only the mean: the 10% path overflows. Log a high percentile.
• Mean is not typical: typical is often the mode (one cheap call). 3.5 is not a die face.
• Multiplying means: E[steps] * E[cost per step] underprices tickets where both are large together. Average total cost from traces.
• Small n: expected pass rate from 8 tickets is a noisy Bernoulli mean. Variance of a proportion is p(1-p)/n.
• Empty mix: chances that do not sum to 1. Assert unit sum before you quote E[X].

Production logs: mean, std, p95, n, and the rare-path rate. Assert chances sum to 1, costs ≥ 0, and that a fixture mix reproduces E and Var in closed form (the tryit). Quote mean and spread of pass/fail when T > 0.

Nested loops

If each ticket is a sum of steps, linearity still gives E[sum of step costs] = sum of E[each step]. That is safer than multiplying means of counts and per-step costs. When in doubt, average total cost from traces.

When two designs have the same expected cost, pick the one with smaller variance unless you are deliberately buying a lottery ticket. A new tool with the same mean wait but 10× variance will dominate timeouts. That is a variance bug, not a mean bug — averaging dashboards hide it.

How agents use this

Set max_steps and max_dollars from expectations plus a buffer scaled by observed std. Log both mean and std of tokens per tool.

• Tokens: mean tokens per ticket times price is expected dollars. Std of tokens tells you whether a single bill is a guess. A loop that resends the transcript raises both mean and variance.
• Ranking: expected recall@k is a Bernoulli mean over queries. Variance p(1-p)/n says how many questions you need.
• Loss: batch loss is a sample mean of per-example losses. Noisy batches are high-variance gradient estimates (optimization lesson). Smooth with a moving average.
• Sampling: each run is one draw. Quote the mean and the spread of pass/fail across seeds when temperature > 0. Unseeded T>0 makes “which prompt won?” a coin flip.

Evals: ship fewer claims on small n, or gather more tickets. Expected pass rate without a count is a poster. A production agent that logs only mean dollars will look healthy until the human path clusters on a Friday.

Tip:When two designs have the same expected cost, pick the one with smaller variance unless you are deliberately buying a lottery ticket.