Gradients

When f depends on several inputs, there is one derivative per input: a partial derivative. Hold the other inputs still. The gradient is the vector of all partials.

The gradient at a point points uphill: the direction you should walk if you want f to increase as fast as possible, nearby. Steepest descent is the negative gradient. Training a network is: compute the gradient of loss, step the opposite way.

The length of the gradient is how steep that best direction is. Near a minimum it should be near zero. If it is huge, a fixed step size will overshoot. If it is tiny but loss is still bad, you may be on a flat.

A wrong picture

A wrong picture is: “the gradient is the weights” or “the gradient is the data.” The gradient is a vector of slopes, one per weight (or per input you differentiated). It has the same length as the thing you are moving. It is not the thing itself.

Another wrong picture is: “we always follow the gradient.” We follow it to maximize. We follow minus the gradient to minimize. Flipping that sign is a common training bug. Loss goes up. People blame the learning rate. Check the sign on a two-variable bowl first.

A third wrong picture is: “if the gradient is small, we are done and the model is good.” Small gradient means a flat of whatever scalar you differentiated. If that scalar is not the loss you care about, you found a flat of the wrong hill. If loss is still high on a flat, you may be stuck, saturated, or measuring a constant.

The formula in words

Partial of f with respect to x: treat y, z, ... as frozen numbers. Slope as in the last lesson.

Gradient: pack those partials into a list [df/dx, df/dy, ...].

Uphill step: point + step gradient. Downhill: point - step gradient.

Numeric partial: nudge one coordinate, central difference, put the others back.

A tiny example

Let f(x, y) = x^2 + y^2, a bowl touching zero at the origin. Then df/dx = 2x and df/dy = 2y, so the gradient is [2x, 2y]. At (1, 0) the gradient is [2, 0] — purely in the x direction, which matches the picture: you are on the x-axis wall.

At (1, 2), f = 1+4 = 5, gradient [2, 4]. Downhill with step 0.1: subtract [0.2, 0.4] to get (0.8, 1.6). New f = 0.64 + 2.56 = 3.2, which is less than 5. Uphill would go to (1.2, 2.4) and f would rise.

A model with a million weights has a million-dimensional gradient. You cannot plot it. You can still plot loss vs step, gradient length vs step, and a couple of coordinates. The definition did not change.

Partials by hand and by nudge

Run to execute this in your browser. Nothing is sent to a server.

Point (1, 2), f = 5. Formula grad [2.0, 4.0]. Nudge grad should match to many decimals. After downhill: (0.8, 1.6) with f = 3.2. After uphill: (1.2, 2.4) with f = 7.2. Loss should drop after a downhill step and rise after uphill. If your training loop does the opposite, you flipped a sign. That bug is common and visible with a two-variable bowl before you touch real models.

If analytic and numeric disagree, you differentiated the wrong expression, or h is bad, or f is not smooth (a cutoff jump). Check f on paper first.

Many dimensions

You cannot draw a million arrows. You can still use the vector as a direction. Gradient length near 0 plus loss still high: flat or dead units. Gradient length exploding: shrink the step, or clip the gradient vector’s magnitude (same clip idea: cap length, keep direction).

Agents that search over continuous knobs (cutoffs, temperatures) can use tiny nudges on a held-out eval. Discrete choices (which tool) do not have classical gradients; there you use scores, bandits, or just rules.

How agents use this

Two gradients show up in agent work. First, training: if you fine-tune or train a ranker, you are following -grad(loss). Second, sensitivity: which input feature, if nudged, changes the score most? That is a partial of the score with respect to features — a poor person’s explanation tool. If the partial with respect to “contains the word refund” dwarfs everything else, your classifier is a keyword detector in costume.

• Tokens: you rarely take a gradient through a whole LLM by hand. You still use partials on your scalar: dollars vs max tokens, pass rate vs cutoff.
• Ranking: a linear scorer s = w · embed has gradient embed with respect to w. Training the scorer is this lesson plus a loss.
• Loss: the gradient is steepest ascent of whatever scalar you differentiated. If that scalar is not the loss you care about, you will energetically optimize the wrong product. Latency not in the loss means latency will not be optimized.
• Sampling: discrete draws do not have a classical gradient. People use tricks (REINFORCE, straight-through) outside this track. For agents, log the chance of the chosen tool instead of pretending the draw was a slope.

The gradient is a vector. Treat it like the vectors lesson: print its length, check dimension matches the weights, do not mix two models’ gradients.

Note:The gradient is steepest ascent of whatever scalar you differentiated. If that scalar is not the loss you care about, you will energetically optimize the wrong product.