Scaling

Scaling laws say: if you grow parameters, data, and compute together, next-token loss tends to fall on a smooth curve. Skip one of the three and you bottleneck.

“Emergent” skills are often that curve crossing a threshold on a harsh metric (pass a coding test that was near 0%). The underlying loss was already sliding.

For agents, the practical law is different: a bigger model on a bad prompt, a stale index, or a fuzzy goal still calls the wrong tool. You are buying fluency, world knowledge, and sometimes better tool use — not a free skip of evals, packing, or greedy decode for actions.

This lesson is size. It is not “always buy the largest.” It is “know what size predicts, and what it does not.”

A wrong picture

A wrong picture is: “bigger always follows company policy.” Policy is prompts, tools, fine-tunes, and code. Scale does not install your handbook.

Another wrong picture is: “emergence means a phase change in the universe.” Often your metric was a cliff (pass/fail JSON, pass/fail code test) sitting on a smooth loss curve. The table looks jumpy. The loss did not jump.

A third wrong picture is: “a frontier model with no tools beats a small model with tools on arithmetic and ‘what is in this file.’” Hands beat scale on those jobs. Scale is not a substitute for tools.

Three knobs together

Parameters: width and depth (and vocab). More knobs can fit more patterns.

Data: more tokens, better filtered. Repeating a small crawl is not the same as new data.

Compute: how long you train, at what batch. You can waste a huge model on too little data, or starve a small model that could have trained longer.

Laws are trends, not promises for your JSON schema. They predict loss on the pretraining game. Your product metric (valid tool calls, grounded answers) is a different curve. Measure that curve on a frozen eval.

Mixture-of-Experts scales parameters faster than compute: a router picks one or two expert MLPs per token. Failure mode: the router ignores an expert, or all traffic hits one expert. Your tool router can fail the same way. You met this in the block lesson. Here it is a scaling trick.

A tiny example in words

Cartoon: loss ~ 3 * N^(-0.1) with N in millions of parameters. Loss slides slowly. If your skill is “pass tool JSON” when loss drops below 1.7, the table looks like a jump: no, no, no, yes. The loss did not jump. Your metric was a cliff.

Do not plan a company around the jump. Plan around the eval you will actually run.

Smooth loss, jumpy pass/fail

Print the cartoon. Lists of numbers. No extra libraries.

Run to execute this in your browser. Nothing is sent to a server.

The table looks like a jump. The loss did not jump. Your metric was a cliff.

Change the threshold to 2.0 and the jump moves. That is how brittle product metrics make “emergence” stories. Report the smooth thing when you can (calibrated error, token-level schema errors), not only pass/fail.

Pick the smallest that meets the eval

Pick the smallest model that meets a frozen eval at the latency and price you can pay. Then spend the leftover money on evals, retrieval, and tools. A 10× larger model that still cannot emit valid JSON is a failed product, not an incomplete scale-up.

Small models plus tools often beat a frontier model with no tools on arithmetic, search, and “what is in this file.” Scale is not a substitute for hands.

Report quality per dollar and per second, not only per billion parameters. This track will not teach hosted price tables. It will teach you that decode policy, packing, and adapters still matter at every size.

LoRA and SFT are how you move a small model toward your schema without buying a 10× base. Try that before you assume you are bottlenecked on parameters.

What size does not buy

Obedience to a schema you never showed. A 70B model will still invent keys if the prefix never named them and decode is sloppy. Grammar plus greedy is cheaper than a size bump for that bug.

A cure for cutoff. Bigger weights freeze on the same day unless the new model’s crawl is newer. Check the calendar, not the parameter count.

Even quality across a 100k window. Larger models can use long context better sometimes. They still dilute. They still fade in the middle. Pack.

Immunity to bad demos. SFT on junk at 70B is a more expensive religion than SFT on junk at 7B. Curate first.

A substitute for evals. The cliff metric will still jump on a smooth loss curve. Report pass rate, calibration, and dollars. Then pick a size.

When someone proposes “just use the biggest,” ask which frozen eval failed and which cheaper lever was tried: packing, greedy tools, a missing calculator, a LoRA on clean traces. Size last among those, not first.

How agents use this

Pick the smallest model that meets a frozen eval at the latency you can pay. Then spend leftover budget on evals, retrieval, and tools. A 10× larger model that still cannot emit valid JSON is a failed product, not an incomplete scale-up.

When someone says “we need the bigger one,” ask which frozen eval failed, and whether packing, greedy tools, or a missing tool would have passed it. Size is a lever. It is not the only lever.

• Laws: loss vs params/data/compute, together.
• Emergence: often a cliff metric on a smooth curve.
• MoE: more params than compute; routers can fail.
• Hands: tools beat scale on arithmetic and files.
• Pick small: that still passes the eval.

Tip:Report quality per dollar and per second, not only per billion parameters.