Distributions

A distribution is a full assignment of chance to outcomes (discrete) or a density (continuous). Named families show up constantly:

• Bernoulli(p): one toss, success with chance p. Tool call succeeds or not.
• Uniform on a set: every outcome equal. random.random() is uniform on [0, 1). Uniform over a vocabulary would be a maximally confused model.
• Gaussian (normal): bell-shaped, with mean mu and standard deviation sigma. Measurement noise, some embedding coordinates.

You describe a distribution by parameters, then either write a formula or draw samples and look at mean and spread. Sampling is how you check you coded the right family.

A categorical distribution is a Bernoulli generalized to more than two labels: a list of chances that sum to 1. Softmax produces one. Sampling a token is a categorical draw.

A wrong picture

A wrong picture is: “everything is Gaussian.” Production latency is often a bump plus a timeout spike. A Gaussian has unbounded tails and no hard cap. One family rarely describes traces by itself. Mix a body with a separate timeout event.

Another wrong picture is: “my confidence is 0.99, so I am Bernoulli(0.99) correct.” If you are wrong half the time at that confidence, you are miscalibrated. The named family is a claim. Plot predicted p vs empirical frequency (a reliability diagram).

A third: a Gaussian with sigma = 0 is a constant. It will sneak into tests. Always print sample mean and std after you write a sampler. Dead embedding dimensions (std near 0 across a corpus) waste cosine; exploding dimensions dominate it.

The formula in words

Every named family has a support (which values can appear) and parameters (which member of the family you picked).

• Bernoulli support is 0 and 1. Mean is p. Variance is p(1-p).
• Uniform on [0, 10] cannot produce 11. Mean is the midpoint 5.
• Gaussian support is all real numbers. Mean mu, spread sigma. About 95% of mass sits within 2 sigma of the mean in the ideal story — a slogan, not a law of tickets.
• Categorical: a list p_i >= 0 summing to 1. Mean is not “the average token id”; that number is usually meaningless. Entropy (next part) measures spread.

Bernoulli is a cutoff on a uniform: 1 if random.random() < p else 0. Uniform floats: a + (b-a) * random.random(). Gaussians can be built from two uniforms (Box–Muller): if U1, U2 are uniform, a formula gives a standard normal. Scale by sigma and add mu.

A tiny example

Draw 4000 Bernoulli(0.3) samples. Mean should sit near 0.3, std near sqrt(0.3*0.7) ≈ 0.458.

Uniform 0–10: mean near 5.

Gaussian N(5, 2): mean near 5, std near 2. Tail P(X > 9) is a few percent — rare, not impossible. That is why “3 sigma” thinking exists: do not treat every outlier as a new regime, and do not treat a 4-sigma wait as a blip.

Moving parts

Support is a contract. Uniform on [0, 10] cannot produce 11. Bernoulli cannot produce 2. A Gaussian can produce a negative wait — which is why it is a bad solo model of latency.

A second walkthrough (body plus spike)

Eight independent tools, each fails Bernoulli(0.10). Chance all succeed: 0.9 ** 8 ≈ 0.430. Chance at least one fails: 1 - 0.430 = 0.570. That is not a Gaussian story. It is a product of Bernoullis.

Now latency. 95% of waits are about N(200 ms, 40 ms). 5% are a hard timeout at 8000 ms.

Mean wait = 0.95200 + 0.058000 = 190 + 400 = 590 ms.

A Gaussian-only dashboard that reports “mean 200 ms” missed the spike. The spike is most of the mean. p95 vs mean will disagree. Model the body and the timeout as two families mixed, not as one bell.

sigma = 0: every Gaussian sample is mu. A dead embedding dimension (std ≈ 0 across a corpus) is this. It wastes cosine. An exploding dimension (std huge) dominates cosine. Print std per coordinate.

A Friday ticket

Friday 19:00. p95 latency “looked Gaussian” on a chart that never showed the timeout bucket. On-call kept treating 8-second waits as 3-sigma blips of a 200 ms bell. They were the 5% timeout event. After they histogrammed waits, the mix was obvious: a bump near 200 ms and a spike at the cutoff. They split the metric: body mean, timeout rate, timeout cap. The Gaussian was allowed to describe the bump only.

Draw three families

Run to execute this in your browser. Nothing is sent to a server.

Bernoulli mean, std should be near 0.3 and 0.46. Uniform mean near 5, std near 10/sqrt(12) ≈ 2.89. Gauss mean near 5, std near 2. P(gauss>9) a few percent (about 0.02–0.03). Seed 1, n=4000: close, not exact. If Bernoulli mean printed 0.01, your cutoff is wrong. If Gauss std printed 0, you passed sigma 0.

The Box–Muller line uses 1.0 - random.random() so u1 is never 0 (log would die). That is the same “do not log 0” rule as the logs lesson.

What goes wrong

• Wrong family: latency is not Gaussian. A cap plus a spike needs a mix. Support matters: if a formula cannot produce 0, it cannot model “zero retried calls.”
• sigma = 0: a constant sneaking into tests. Print sample std.
• p outside [0, 1]: Bernoulli(1.2) is not a chance. Clip or raise.
• Miscalibration: predicted 0.99, empirical 0.50. You are not Bernoulli(0.99) correct. Plot p vs frequency.
• Categorical that does not sum to 1: not a distribution. Softmax first.

Production logs: family name, parameters, sample mean, sample std, and (for latency) timeout rate separately. Assert 0 <= p <= 1, sigma >= 0, categorical sums to 1. After you write a sampler, print mean and std once.

Discrete vs categorical

Sampling a token is a categorical draw. The sampling lesson later implements that draw with a cumulative sum. Uniform over vocab is the maximum-entropy categorical (entropy lesson). Temperature moves you toward or away from that uniform.

Noise in embeddings is often treated as roughly Gaussian in each coordinate. That is a model, not a law. Still, mean and std of a coordinate across a corpus tell you whether a dimension is dead (std ≈ 0) or exploding.

How agents use this

Calibrated agents need distributional honesty. If your “confidence” is always 0.99, you are not Bernoulli(p_correct); you are miscalibrated.

• Tokens: softmax output is categorical. Greedy is the mode. Sampling is a draw. Uniform would be temperature infinite in the slogan limit.
• Ranking: cosine scores are not a named family. You can still histogram them. A blob of similar top-k scores is a high-entropy retrieval neighborhood.
• Loss: Bernoulli log-loss is cross-entropy for two classes. Categorical cross-entropy is the same idea with more labels.
• Latency / cost: model the body and the timeout spike separately. A single Gaussian will smear the spike into fake “typical” waits.

Always print sample mean and std after you write a sampler. A Gaussian with std 0 is a constant, and it will sneak into tests. Support matters: if a formula cannot produce 0, it cannot model “zero retried calls.” Histogram waits before you name the family. Timeout rate is its own Bernoulli, not a tail of N(200, 40).

Tip:Always print sample mean and std after you write a sampler. A Gaussian with std 0 is a constant, and it will sneak into tests.