Why Agents Need Math

Agents look like English on the outside. A support bot answers a refund question. A coding helper picks a tool. A search step returns a paragraph. You read words. Inside the machine there are no words until the last layer turns numbers back into tokens. There are lists of numbers, scores, and chances.

This track teaches those objects in simple English, with Python lists of floats. No NumPy. No special array type. If you can loop over a list, you can run every formula an agent actually uses.

What is going on inside

When a support agent finds the right paragraph, it is not “vibes.” A question becomes a list of numbers. Each document becomes a list of numbers. A score ranks those lists. The highest score is the chunk stuffed into the prompt. If the wrong chunk wins, the model never sees the right paragraph. That failure is geometry, not personality.

When a model picks a slightly different next word, that is chance. The model first writes one number per possible token (a logit). Those numbers become chances. A random draw picks one token. Change temperature and you change the chances. The English changes because the draw changed.

When a dashboard says loss went from 2.4 to 1.1, that is a function of weights. Training nudges those weights so the function’s output gets smaller. If loss does not move, the slope is zero, huge, or pointed at the wrong thing.

Joeven teaches this math so you can debug. A search that “feels random” is often a geometry bug: bad chunks, unnormalized lists, or a score on the wrong axis. A model that repeats itself is often temperature or sampling. A fine-tune that does nothing is a slope that is zero, huge, or pointing at the wrong loss.

A wrong picture

A common wrong picture is: “the agent understands English, so math is optional.” The product is English. The machinery is numbers. If you skip the numbers you can still call APIs. You cannot explain why Tuesday’s agent is worse than Monday’s. You cannot fix a retriever that quietly ranks the wrong chunk first. You will argue with the model instead of measuring a score.

Another wrong picture is: “I need a PhD.” You do not. You need a working picture of six objects: functions, vectors, matrices, derivatives, probability, and entropy. We implement them with Python lists. The agent use sits next to the formula. Later lessons stay in this lane: tokens, ranking, loss, sampling. They do not teach transformer internals. One later page will say, in one line, that attention is a weighted average — and then stay on the average.

The map

Every expensive part of an agent has a name on the right.

If you can code the right-hand column with lists and loops, vendor docs stop looking like magic. You will see the same six objects in RAG, training, and evals.

Tiny numbers, same shape

Take a query as four numbers: [0.2, 0.8, 0.1, 0.0]. Pretend “refund policy” is [0.1, 0.9, 0.0, 0.1] and “password reset” is [0.0, 0.2, 0.1, 0.9]. Slot by slot, the query and the refund list both have a large second number. The password list has its large number in a different slot. A score that multiplies matching slots and adds them up will rank refund higher. That is ranking.

In production the list has 384, 768, or 1536 numbers. The shape is the same: close lists rank high, far lists rank low. You do not need to see 1536 arrows. You need to believe that “close” is a formula you can print.

Tokens are counts. A prompt of 800 tokens plus a reply of 200 tokens is 1000 tokens. Cost is tokens times price times steps. If a 20-step loop resends a growing transcript, you sum the tokens at each step. You do not multiply 20 by the first prompt.

Chance is a number between 0 and 1. If a tool is “usually fine” with chance 0.9 of success, twelve independent steps have chance 0.9 ** 12 of all succeeding — already ugly. If the failures share one downed API, you must not multiply. Probability later names that trap.

Entropy is “how spread out is this list of chances?” A model that puts 0.95 on search is sure. A model that puts about 0.33 on three tools is unsure. Unsure policies waste calls. Confident-and-wrong policies waste the whole ticket.

A ranking you can see

This box pretends a tiny 4-number embedding model exists. The shape is the same in 4 numbers as in 1536.

Run to execute this in your browser. Nothing is sent to a server.

Run it. You should see refund policy near 0.978, shipping times near 0.35, password reset near 0.221. The query is close to “refund policy” and far from “password reset.” That is ranking.

mag is length: square each slot, add, take the square root. cosine is “multiply matching slots, add, then divide by both lengths.” Later lessons name the formula (cosine) and the list (a vector). Change the query toward [0.0, 0.2, 0.1, 0.9] and watch password reset rise. The English did not change. The numbers did.

If you forget to divide by lengths, a longer list can win just because its numbers are bigger. That is a real RAG bug. Normalization (making length 1) is the fix you will code on the vectors page.

What this track will teach, in order

First: functions, sums, logs, min/max/clip. Those are the arithmetic of loss, cost, and cutoffs.

Then: vectors, dot products, matrices, linear maps. Those are embeddings, ranking, and the shape of a linear layer.

Then: slope, gradients, the chain rule, downhill steps. Those are training and “what happens if I nudge this knob.”

Then: chance, Bayes, named distributions, expectation and spread. Those are sampling, retries, and budgets.

Then: entropy, softmax, temperature, cross-entropy, a weighted average (attention in one picture), embedding geometry, and precision/recall. Those are the numbers on agent dashboards.

You will not train a giant model here. You will compute the tiny versions until the giant ones look like the same objects with more slots.

How agents use this

Every expensive part of an agent is a number you can measure:

• Context length is a count (tokens). Cost is tokens times price times steps. A loop that resends the whole transcript is a sum that grows, not a flat fee.
• Memory is a set of vectors you search. Retrieval is a score (often cosine) plus a sort plus a cutoff. If top-k is 4 and the right paragraph is 5th, the model never sees it.
• The policy is a chance over tokens or tools. Temperature reshapes those chances. Sampling draws one. Entropy says how flat the list was.
• Training and many evals are a function from weights or traces to one number (loss, pass rate, dollars). You shrink or grow that number on purpose.
• After a tool result, belief should update. That is Bayes, even when you write it as if timeout: retry once, then page.

When an agent fails, ask which number was wrong: a similarity, a probability, a count, or a cutoff. Write the number in the log next to the decision. “Skipped chunk cosine=0.22 threshold=0.35” is a fixable sentence. “The model was weird today” is not.

If you skip math, you can still ship a demo. You cannot tell ranking bugs from sampling bugs from loss bugs. Those three look the same in English and different in numbers.

Tip:When an agent fails, ask which number was wrong: a similarity, a probability, a count, or a cutoff. That question is this track.