The Chain Rule

Most interesting functions are pipelines. y = f(g(x)). The chain rule says the slope of the outer function is the product of slopes along the path:

dy/dx = df/du * du/dx

where u = g(x). If there are more nests, keep multiplying (and, with several paths, adding).

This is backpropagation. A deep net is a chain of maps: linear, ReLU, linear, softmax, loss. The derivative of loss with respect to an early weight is a product of many local derivatives. You compute it from the output backward because each local derivative is easy, and you reuse middle results. Forward: compute the numbers. Backward: multiply the slopes.

(Transformer internals: that chain is long. You do not need the blocks. You need the product.)

A wrong picture

A wrong picture is: “backprop is a special ML trick unrelated to calculus.” It is the chain rule, implemented carefully. Libraries save the forward values so the backward multiply is cheap.

Another wrong picture is: “if the last layer has a slope, every weight gets a useful signal.” If any local derivative is 0 (a dead ReLU, a hard cutoff), the product is 0 and that weight gets no signal. A tool that is never called has local derivative zero with respect to that tool’s prompt: changing the prompt cannot affect a loss that never saw the tool.

A third wrong picture is ignoring several paths. If x is used twice, say y = x * x, then two paths contribute and you add the two products. Forgetting to add (overwriting) is a bug that looks like “half the gradient.”

Exploding and vanishing: if each factor is 2 and you have 40 layers, the product explodes. If each factor is 0.5, it vanishes. Those names are this product getting huge or tiny.

The formula in words

Outer slope times inner slope. More boxes: more factors. Split and rejoin: add the incoming backward slopes.

Tiny numeric. Let g(x) = 3x + 1 and f(u) = u^2, so y = (3x + 1)^2. Then df/du = 2u and dg/dx = 3, hence dy/dx = 2(3x+1)3. At x = 2, u = 7, y = 49, slope 27*3 = 42.

A tiny nudge: h = 1e-5, central difference on y should land on 42.

If y is a loss and w is the 3 in g, then du/dw = x = 2, so dy/dw = 2u x = 14 2 = 28. Step w opposite that number to reduce y.

Check a nest two ways

Run to execute this in your browser. Nothing is sent to a server.

You should see u=7, y=49, chain rule 42, nudge about 42.0. Backprop prints dy/dx 42.0 and dy/dw 28.0. The last block is the spirit of autodiff: during the forward pass you remember u and x; going backward you multiply by the local slope. dy/dw tells you how to update the weight w if y were a loss (you would step opposite that number).

If the two dy/dx lines disagree, you mis-multiplied. If they agree, you are allowed to trust the chain on a longer pipeline — still check a numeric nudge on a small net when you write custom backward code.

Several paths and zeros

y = x * x is two uses of x. Each path contributes x (because d(uv)/dx along one factor holding the other is the other factor). Sum: 2x, which you already know. Frameworks sum gradients into the same tensor.

Hard cutoff: y = 1 if u >= 0.35 else 0. Local slope is 0 almost everywhere. A retrieval cutoff blocks gradient from the generator back to the embedder if you trained them as one pipe. In agents, that is often fine: you train retriever and generator separately, with their own scalars.

When people say “the loss doesn’t include latency, so the model will not optimize latency,” they are saying there is no path in the chain from latency to the scalar that training differentiates. Add a term, or don’t be surprised.

How agents use this

You rarely hand-write backprop for a transformer. You still use the chain rule in your head when an agent’s metric is nested: dollars depend on tokens, tokens depend on steps, steps depend on a retry policy. A change in retry chance ripples through the product of local slopes. If you want less spend, you must change something that actually has a nonzero path to spend — a tool that is never called has local derivative zero with respect to that tool’s prompt.

• Tokens: d(dollars)/d(max_steps) is d(dollars)/d(tokens) * d(tokens)/d(max_steps). Price is the first factor. The second is “does allowing another step actually add tokens on this ticket set?” If agents already stop early, the second factor is near 0.
• Ranking: retrieval cutoff is a jump. There is no useful classical slope through “did the gold chunk make top-k.” Eval the retriever with recall@k directly.
• Loss: training works because every weight has a path to the scalar. If you detach a tensor (stop the chain), those weights freeze. That is a switch, not a mystery.
• Sampling: a hard sample (one token id) is another jump. The chance of that token still has a slope (cross-entropy). That is why we train on log-chances, not on the random draw.

Draw the boxes. Arrows forward for values, arrows backward for slopes. If you cannot draw it, you cannot debug it. Nested agent costs are the same drawing with “retry,” “tokens,” and “dollars” as boxes.

Tip:Draw the boxes. Arrows forward for values, arrows backward for slopes. If you cannot draw it, you cannot debug it.