Sums, Products, and Averages

Almost every formula in this track is a loop that adds. Math writes a capital sigma: the sum of x_i from i = 1 to n. In Python that is s = 0 then s += x, or sum(xs).

A product multiplies instead. Independent chances multiply. So do token chances: the chance of a whole sentence is the product of next-token chances (or a sum if you take logs — next lesson).

An average (the mean) is a sum divided by a count. Batch loss is an average. Cost per ticket is an average. “The agent usually takes 8 steps” is a mean hiding a spread.

A wrong picture

A wrong picture is: “sigma is advanced notation I can skip.” It is a for-loop. If you can add a list, you can read a paper’s loss. Another wrong picture is averaging percentages that came from different counts: 90% of 10 tickets and 50% of 200 tickets is not “70%.” You must go back to the raw sums: total passed over total tickets. A third wrong picture is treating a product of success chances as if retries and tools fail independently when they share one API. The formula is only as honest as the independence you assumed.

The formula in words

• Sum: start at 0. For each number, add it. That is sigma.
• Product: start at 1. For each number, multiply. That is the capital pi you will see next to likelihood.
• Mean: sum, then divide by how many items. That is the average.
• Weighted mean: multiply each item by a weight, add, and (if the weights do not already add to 1) divide by the sum of weights. Softmax output is a list of weights. Attention is a weighted average of value vectors. You do not need the transformer lesson yet. You need “sum of weight times vector.”

Mean squared error against a target: for each item, subtract the target, square, add all those squares, divide by the count. In words: “how wrong, on average, if we always predicted this target.” Training will change weights so that a similar sum gets smaller.

Indexes

If x is a list, x[0] is the first item in Python. Papers often write x_1 for the first item. Off-by-one between papers and code is a classic bug. Write the range down: i from 0 to n-1, or 1 to n. When a formula says “sum from i = 1 to n,” your Python is for i in range(n) if you stored items starting at index 0.

A moving average is the same idea over time: fold each new loss into a running mean so a dashboard does not twitch. Agents do this to token counts and tool wait times. You are still dividing a sum by a count. You just pick which window to include. A window of 1 is the last point (noisy). A window of all history is slow to show a regression.

A tiny example

Five numbers: [2, 5, 5, 8, 1]. Count n = 5. Sum is 2+5+5+8+1 = 21. Mean is 21 / 5 = 4.2. Product is 25581 = 400. If the target is 5, the squared errors are (2-5)^2=9, 0, 0, (8-5)^2=9, (1-5)^2=16. Sum of squares is 34. MSE is 34 / 5 = 6.8. That is “how wrong, on average, if we always predicted 5.”

The log of the product equals the sum of the logs: log(400) matches log(2)+log(5)+log(5)+log(8)+log(1). Next lesson uses that identity so long products of chances do not hit 0.0.

Run the operations

Run to execute this in your browser. Nothing is sent to a server.

You should see n 5, sum 21, mean 4.2, product 400, mse vs 5 6.8. The two log lines should match (tiny float noise is fine). math.log is natural log, base e. The identity does not care which base you pick, as long as you are consistent.

The sum is the fancy-looking sigma in the blog post. MSE is the shape of many training losses: add up how wrong each example is, then average so a bigger batch does not automatically look like a bigger loss.

Token budgets are sums

A 20-step loop that resends a growing transcript is a sum of tokens per step, not “20 times the first prompt.” If step t costs c_t tokens, total tokens are those c values added up. Dollars are that sum times price. If step 1 is 800 tokens and each later step adds 100 new tokens but resends the old ones, the costs look like 800, 900, 1000, … That is an arithmetic growth you can sum. People who budget “20 * 800” underprice the loop.

When you average eval scores, know what you averaged. Mean pass-rate over 50 tickets hides that 10 tickets are impossible. A product of per-step success chances (if you assume they do not share a cause) shows why long runs fail even when each tool is “usually fine”: 0.9 ** 12 is already about 0.28. Twelve “almost sure” steps are not almost sure as a chain.

Prefer sums of logs over giant products. Products of chances underflow to 0.0 in floats. Log-likelihood is just a sum. The next lesson is that move.

Weighted averages show up again in retrieval: if you average chunk embeddings to make one document vector, you add the lists slot by slot, then scale by 1/n. That is only legal if every chunk used the same embedder. Mixing models is adding lists that do not live in the same space.

How agents use this

Budgets, losses, and evals are sums and averages. Write them that way in logs.

• Cost: sum tokens per step, then multiply by price. Print the sum, not only the mean, when one ticket exploded.
• Batch loss: average of per-example losses. If you forget to divide by n, bigger batches look worse even when they are not.
• Success chains: product of per-step chances, or sum of logs. Long agents fail because products shrink, not because each tool is terrible.
• Dashboards: a moving average of latency hides spikes. Pair it with max, or with a high percentile later. Mean alone is a lie when variance is huge (expectation lesson).
• Attention / RAG mix: weighted sum of vectors. Weights from softmax. Same “sum of weight times vector” as here.

Off-by-one in a sum is a silent bug: skipping the last token chance, or adding an extra zero. Print n next to the sum. If n is not the length you expected, the formula in the comment is not the loop you wrote.

Tip:Prefer sums of logs over giant products. Products of chances underflow to 0.0 in floats. Log-likelihood is just a sum.