Functions and Graphs

A function is a rule that sends each allowed input to exactly one output. Write y = f(x). The set of legal x is the domain. The outputs that actually appear are the range.

That sounds like school. It is also the whole model.

• A language model is a function from a token list to a list of logits (one number per word in the vocab).
• A loss is a function from weights (and a batch of data) to one number that should not be negative.
• A policy is a function from what you saw to an action, or to chances over actions.
• Temperature is a function that reshapes logits before they become chances.

If two inputs can map to the same output, that is still a function. Many tickets can map to “call search.” If one input would need two outputs at once, that is not a function — and it is a bug in your API. A tool that sometimes returns a string and sometimes a dict for the same arguments is not a function you can test.

A wrong picture

A wrong picture is: “the model is random, so it is not a function.” Randomness is extra input. If you include the seed (and the draw) in the domain, the map is still a function: same transcript, same weights, same seed, same next token. When people say “the model is random,” they mean this function also depends on a draw you did not log. Still a function — the domain includes the random draw.

Another wrong picture is: “a graph is only a drawing.” A graph is the set of pairs (x, f(x)). On this page you get a drawn curve, then a table you can print. You will not plot a 7-billion-weight network. You will plot the tiny functions that network is made of.

A third wrong picture is: “functions must be smooth formulas.” A lookup table is a function. A 20-line def is a function. A transformer is a function. Same idea, different cost.

The formula in words

Pick an allowed input. Apply a rule. Get one output. That is all.

A table is a function you can read with your eyes: each row is an input, the last column is the output. A formula is a function you can compute: f(w) = (w - 3)^2. A program is a function you can run. Training is the sport of changing the program’s weights so the outputs on data get closer to what you wanted.

Composition means stacking: h(x) = f(g(x)). Agents stack constantly. softmax(logits / T) is three maps: scale by temperature, turn into chances, then (later) pick a token. If any stage is wrong, the outer function is wrong. Debugging is isolating which map failed.

A tiny example

Try a toy loss f(w) = (w - 3)^2. In words: take the weight, subtract 3, square it. At w = 3 you get 0. At w = 2 you get 1. At w = 5 you get 4. At w = 0 you get 9. That is a U shape with a lowest point at w = 3. Training is “walk downhill on this graph.” You do not need calculus yet to see the valley.

Change the function. abs(w - 3) is a V: same bottom, sharp corner. 2 ** (-abs(w - 3)) is a bump: high at 3, low far away. Same loop, different mapping. Loss bowls and policy bumps are the same object: a function you can print.

Tables are graphs

Print w and f(w) for a few points. Draw hashes so the eye sees height. That is a graph without a plotting library.

Run to execute this in your browser. Nothing is sent to a server.

As w gets near 3, the bar shrinks. At 3 you get 0 and no hashes. Past 3 it grows again: 4, 5, 6, 7 give 1, 4, 9, 16 hashes (16 is int(16)). Later lessons name the slope of this graph and a rule for picking the next w. For now, notice: left of 3, increasing w lowers loss; right of 3, increasing w raises loss. Downhill is toward 3 from both sides.

Swap loss for abs(w - 3) and rerun. The valley is a V. Swap for 2 (-abs(w - 3)) if you like a bump you would maximize** instead of minimize. Sign of the goal matters: training usually minimizes loss; some agent knobs maximize an eval score.

Discrete vs continuous

Token ids are discrete: 0, 1, 2, … A temperature slider is continuous: 0.2, 0.7, 1.0. A function can have either kind of input. A policy over tools is a function from a transcript to a list of chances (continuous numbers) and then a draw (a discrete pick). When you log p_tool_search=0.61, you are reading the continuous output. When you log tool=search, you are reading the discrete pick.

Step functions show up as cutoffs: 1 if cosine >= 0.35 else 0. That is still a function. Its graph is a jump. Derivatives later will complain about jumps. Agents use jumps anyway: refuse, retry, stop.

Stack functions

Agents stack functions. A tool-using agent is also a function: transcript in, next tool out. Retrieval is a function from a query vector to a ranked list. The generator is a function from (prompt + chunks) to tokens. The product is the composition. If retrieval is wrong, the generator never sees the right text. The outer function is then “wrong” even if the generator is fine. That is why you eval the retriever and the generator, not only the English at the end.

A function can ignore some of its input. A broken router that always returns search is still a function. It is a constant function of the ticket. Constants are easy to test and useless as policies. If your logs show one tool 100% of the time, you implemented a constant.

How agents use this

When you log loss=1.83 or p_tool_search=0.61, you are reading f(current_state). Plots of those numbers over steps show goal drift (the function you meant to shrink is not the one going down) and mode collapse (almost all chance sits on one action).

Treat every knob as a function:

• Temperature T maps logits to a new list of chances. Same logits, different T, different policy.
• A cutoff maps a cosine to {keep, skip}. Sweeping the cutoff traces a graph of precision and recall (last lesson in this track).
• max_steps maps a run to “stop or continue.” Dollars are a function of that choice.

You will not graph a 7-billion-weight network here. You will graph the tiny functions that network is made of, until “the model is a function” feels obvious. Then, when loss is flat, you will ask: is this function actually depending on the weights I am changing? A constant does not care about your learning rate.

If two stages are composed and the outer metric is bad, isolate the inner map. Print f(x) for a few hand-picked x. Functions become debuggable when you treat them as tables.

Note:A lookup table is a function. A 20-line def is a function. A transformer is a function. Same idea, different cost.