Probability

Probability assigns a number in [0, 1] to an event — a subset of possible outcomes. The set of all outcomes is the sample space. For a fair six-sided die, the sample space is {1,2,3,4,5,6} and each outcome has chance 1/6. The event “even” has three outcomes, so chance 1/2.

Two events are mutually exclusive if they cannot both happen. Their chances add. If every outcome is equally likely, probability is counting: size of the event divided by size of the space. If outcomes are not equal (a bent coin, a language model), you cannot count; you weigh.

Independence means P(A and B) = P(A) P(B). Coin flips are the textbook case. Agent tool failures are not independent if they share a downed API. Multiplying “99% reliable” twelve times is only legal if the failures do not cluster. They cluster.

The complement is cheap: P(not A) = 1 - P(A). “Chance we never hit bad JSON in 8 calls” is 1 minus the chance of at least one bad call. If you cannot compute an event directly, compute the opposite and subtract.

A wrong picture

A wrong picture is: “if each tool is 90% reliable, a 12-step agent is basically fine.” If you assume independence, 0.9 12 ≈ 0.28. The chain is not fine. If failures share a cause**, the product is the wrong model entirely: one outage fails every call at once.

Another wrong picture is treating an eval pass rate as the true chance. Your eval set is a sample, not the whole space. 29/32 is a noisy frequency. Report counts. Small n cannot tell 0.80 from 0.88 (variance lesson).

A third wrong picture is: “mutually exclusive” and “independent” are the same. They are almost opposites. Exclusive events cannot both happen; their “and” chance is 0, which equals the product only if at least one chance is 0. Independent events can both happen; their “and” is the product.

The formula in words

• Chance is a weight in [0, 1]. All outcomes together weigh 1.
• Exclusive “or”: add the chances.
• Independent “and”: multiply the chances.
• Complement: subtract from 1.
• Equal outcomes: count / total.
• Unequal: sum the weights of the outcomes in the event.

Conditional P(A|B): restrict the space to B, then measure A. “Chance the ticket is fraud given the tool returned country mismatch.” Next lesson turns that into Bayes.

Simulation: draw many outcomes, measure frequencies. Frequencies get close to probabilities if draws are independent and from the same distribution. For agents, simulation is often the only honest tool: the space of “what the model might do” is too large to list.

A tiny example

Two dice, 6×6 = 36 equally likely pairs. Sum to 7: (1,6), (2,5), (3,4), (4,3), (5,2), (6,1) — six pairs. Chance 6/36 = 1/6.

Biased coin, P(heads)=0.3. You cannot count faces. Simulate: many random.random() < 0.3 checks, divide heads by n. Around 0.3 if n is large.

Five tools, each fails with 0.1, independent: chance any fails is 1 - 0.95 ≈ 0.41. Shared outage with chance 0.1 that kills all** of them: chance of “all fail via outage” is 0.1, not 0.41. Different sample spaces, different numbers.

Counting vs simulation

Run to execute this in your browser. Nothing is sent to a server.

Dice: space 36, sum7 is 6, P is 0.1666... which is 1/6. Simulated P(H) should sit near 0.3 (seed 0, 5000 trials: close, not exact). Independent any-of-5-fail should land near 0.41. Shared outage near 0.10. The two failure models do not give the same number. If your risk doc multiplies independent 10% chances, but production is a shared outage, you priced the wrong sample space.

random.random() < p is the Bernoulli trial you will name in the distributions lesson. Here it is just a biased coin.

When you write max_retries=3, you are asserting a model of how often independent retries help. Measure it. If errors are systematic (bad schema), retries sample the same failure. The complement “never succeeds” stays near 1.

Conditional probability

P(A|B) is chance of A given that B happened: restrict the sample space to B, then measure A. All timeouts, then how many are “vendor down.” That is not the same as “how many downs time out.” Bayes next.

Token sampling is a weighted space: the vocab is the outcomes, softmax supplies the weights. You do not count tokens; you weigh them. Same definition of probability.

How agents use this

Every decode step is a draw from a distribution over the vocabulary. Every tool call is an event: success, timeout, bad JSON, wrong side effect. Your eval set is a sample, not the whole space — so a 91% pass rate on 32 tickets is a noisy frequency. Report counts, not just percents: 29/32 is more honest than 0.91.

• Tokens: next token is a random event with a huge sample space (the vocab). Greedy decoding picks the mode; sampling draws. Both need the weights to sum to 1 (softmax).
• Ranking: retrieval is usually not random given the index. The query is random from users. Measure recall on a sample of queries, not on one lucky question.
• Loss: log-likelihood is the log of a product of token chances — independence along the sequence given the model. That is a modeling assumption, not a law of tickets.
• Retries: independent retries help blips. They do not help a wrong schema. Write down which sample space you think you are in.

When errors cluster, stop multiplying. Draw the events. Exclusive vs independent vs shared cause. That drawing is the risk doc.

Note:random.random() < p is the Bernoulli trial you will name in the distributions lesson. Here it is just a biased coin.