Bayes' Rule

Bayes’ rule is how you update a belief when a new observation arrives:

P(H|E) = P(E|H) P(H) / P(E)

• H is a hypothesis (the API is down; the user wants a refund; this chunk is relevant).
• E is evidence (the tool timed out; the message contains “invoice”; cosine is 0.8).
• P(H) is the prior — belief before the observation.
• P(E|H) is the likelihood — how expected the evidence is if H is true.
• P(H|E) is the posterior — belief after the observation.
• P(E) is the chance of the evidence overall, often a sum over hypotheses.

A well-behaved agent changes its plan when observations arrive. That is Bayes, even if you implement it with if statements.

A wrong picture

The classic swap: people confuse P(E|H) with P(H|E). “90% of down APIs time out” is not “90% of timeouts mean the API is down,” unless the prior is already extreme. Base rates matter.

People skip P(E) and then cannot compare scales. P(E) is the normalizer: total mass of “evidence under each hypothesis.” Without it you have unnormalized masses, not chances.

Another wrong picture: treating two timeouts as two independent observations when they are the same hung connection. You double-counted evidence. Real agents should know whether two observations are copies. Count it as one.

A third: a rare hypothesis plus a noisy detector. Likelihood ratio can look strong and the posterior still modest. That is alert fatigue: most alerts are still “not fraud” if fraud is rare.

The formula in words

Posterior is proportional to likelihood times prior. Then divide by P(E) so the hypotheses you care about sum to 1.

P(E) for two hypotheses (down vs up): P(E|down)P(down) + P(E|up)P(up). That is the law of total probability: weigh each world by how often it happens, then add.

Odds form: prior odds times likelihood ratio P(E|H)/P(E|not H) gives posterior odds. A test that is 18 times more likely under H than under not-H is strong, but a rare H can still lose.

A tiny example

Prior: vendor down 10% of the time. If down, P(timeout)=0.9. If up, P(timeout)=0.05 (blips). You see a timeout. What is P(down | timeout)?

P(timeout) = 0.90.10 + 0.050.90 = 0.09 + 0.045 = 0.135.

P(down | timeout) = (0.9*0.10) / 0.135 = 0.09/0.135 = 2/3 ≈ 0.667.

The posterior is much larger than 10%, but it is not 90%. Base rates matter. A second independent timeout multiplies another 0.9 vs 0.05 into the masses and pushes further. If both timeouts are the same hung socket, do not multiply.

Moving parts

Posterior is proportional to likelihood times prior. Divide by P(E) so the hypotheses you care about sum to 1.

A second walkthrough (rare fraud)

Fraud is rare: prior P(fraud) = 0.01. Detector: P(alert|fraud) = 0.90, P(alert|ok) = 0.05.

P(alert) = 0.900.01 + 0.050.99 = 0.009 + 0.0495 = 0.0585.

P(fraud|alert) = 0.009 / 0.0585 ≈ 0.154.

A “90% detector” on a 1% base rate still leaves about 15% posterior. Most alerts are still not fraud. That is alert fatigue as a number, not a mood. Raise the prior (this tenant is already flagged) or lower the false-alert rate if you want a posterior you would act on.

Odds form, same story: prior odds 1:99. Likelihood ratio 0.90/0.05 = 18. Posterior odds 18/99 = 2/11, which is about 0.154 again.

Zero likelihood: if P(E|H) = 0 for every H you listed, P(E) = 0 and you cannot divide. You forgot a hypothesis (a third world: “bad schema,” not down vs up).

A Friday ticket

Friday retries. First timeout: P(down|timeout) ≈ 0.67, so one retry is rational (could be a blip). The same hung socket timed out again. The code multiplied a second 0.90 vs 0.05 as if the draws were independent. Posterior jumped to ~0.97. The agent hammered a live API that was slow, not down.

They counted copies as one observation and added a third hypothesis: “slow but up.” After one timeout the mass split three ways instead of two. The retry policy followed the posterior, not the slogan “always retry twice.”

A tool timeout

Run to execute this in your browser. Nothing is sent to a server.

P(timeout) prints 0.135. P(down | timeout) prints about 0.6667. The mass line is the same posterior, written as two weights that sum to 1: down about 0.667, up about 0.333. After two independent timeouts, P(down) prints about 0.973. That last jump is why people hammer retries — and why it is wrong if the two timeouts are copies.

If P(down | timeout) is only 0.67, one retry is rational (could be a blip). If it is 0.99, stop hammering and page a human. The number changes the policy.

Write the two masses even when you have more than two hypotheses (refund vs shipping vs auth). Each mass is likelihood times prior. Divide by the sum. That is all of Bayes for a finite list of buckets.

What goes wrong

• Swapping likelihood and posterior: “90% of down APIs time out” is not “90% of timeouts mean down.” Base rates matter. Write both numbers.
• Skipping P(E): unnormalized masses are not chances. You cannot compare them to a 0.5 cutoff until you divide.
• Double-counting copies: two timeouts on one socket are one observation. Independent multiply is illegal.
• Rare H plus noisy detector: posterior stays modest. Acting on every alert is a false-positive factory.
• Zero P(E): you omitted a hypothesis. Add a bucket or refuse to update.

Production logs: prior, likelihoods, P(E), posterior, and whether evidence was counted as independent. Assert posteriors sum to 1 over the listed H, all ≥ 0, and P(E) > 0. A line prior=0.10 post=0.67 E=timeout copies=1 is a policy. I am pretty sure the API is down is not.

How agents use this

A well-behaved agent changes its plan when observations arrive. Prior “search the docs first”; evidence “search returned nothing”; posterior “ask a clarifying question.” Writing actual numbers is useful when you set alert cutoffs and retry policy.

RAG is Bayesian too, loosely: prior over documents (uniform or recency-weighted), likelihood from an embedding score. You will rarely write the formula, but when a high cosine from a garbage chunk beats a slightly lower cosine from a trusted source, you forgot the prior over sources. Trust and recency are priors. Cosine is (a stand-in for) likelihood. Posterior ranking should mix them.

• Tokens: the model’s next-token chances are a prior over words given the prompt. New tool output is evidence. A stubborn agent that ignores the tool result is refusing to update.
• Ranking: multiply (or add in log space) a source-trust weight with cosine. That is prior times likelihood without the full sermon.
• Loss: Bayes is not a training loss. Calibration (does 0.7 mean 70%?) is the cousin you plot in evals.
• Sampling: do not “Bayesian update” by sampling harder. Update the plan. Then sample from the new policy (maybe fewer tools).

Do not update as if evidence were independent when the same tool is retried on the same bug. Count it as one observation.

Watch out:Do not update as if evidence were independent when the same tool is retried on the same bug. Count it as one observation.