The Assembler Has a Budget

The assembler is a budget officer. It decides what the model is allowed to see. Everything else is noise you are paying for, and worse: noise the model will imitate.

A language model does not “remember the week.” It reads the blob you assemble this call. If that blob is the full transcript, forty tool schemas, yesterday’s stack traces, and a Slack dump, two things happen. Cost grows with every step. Attention to the actual goal dies. The model starts copying old errors because they are sitting in the window looking like examples.

Treat assembly as a function with a token budget, not as string concatenation. Concatenation has no policy. A budget has a policy: what must be present, what may be truncated, what must never appear.

What belongs in context

The assembler typically packs:

• The goal, in one place, not restated five ways
• A checkable success predicate if you have one (“finish only after an observation that contains final”)
• A window of recent actions and observations — last N, not the entire run
• Tool schemas, but only the tools legal in this state
• Retrieved notes, user profile, or policy snippets that this step needs

That list is already too much if you paste it raw. The goal should be short. The window should be recent and truncated. Tool docs should be the schema the dispatcher will actually run, not the company’s entire catalog. Retrieved notes should be retrieved, not “everything we ever embedded.”

The goal is a contract, not a vibe

“Help the user” is not a goal. “Add 2 and 3, then finish” is a goal. “Refund order 99 if it is inside the 30-day window, else deny” is a goal. Put the goal in a field the model cannot miss. Do not bury it under three paragraphs of persona.

If you have a success check in code, say so in the assembled context: what “done” looks like. The model still might ignore it. Stop will not. The assembler and stop should agree on the predicate. If the prompt says “always answer” and stop says “budget then refuse,” you trained a conflict.

Windows, not autobiographies

Recent events are the loop’s RAM. Last three to six turns is a starting point, not a religion. Each observation should already be short (executor’s job). If an observation is a 40k HTML dump, the assembler is too late — truncate at execute time, then maybe truncate again at assemble time.

When the window is full, summarize into memory. Do not grow the prompt forever. A rolling summary is lossy on purpose. Lossy and bounded beats lossless and bankrupt. The memory lesson later names scratchpad vs summary vs retrieve. The assembler is the reader of those stores. It must not dump all three in full every time.

A common failure: dumping the full transcript plus 40 tool docs into every call. The model then treats an early failed call as a few-shot example of how to fail. You will swear the model “cannot learn.” It is imitating you.

Only legal tools

Tool schemas are expensive and dangerous. Every advertised name is a name the model may emit. If refund is in the prompt during gather, the parser and executor must still refuse it — but you already spent tokens tempting the model.

The assembler should advertise only the tools the dispatcher will run in this state. Gather sees search and get_ticket. Apply sees refund. Done sees finish. That is typed state, used here as a filter. The tools track owns the registry. The assembler owns the subset in the prompt.

Forty tools is a fuzzy manual of the internet. Eight tools with tight schemas is an agent. If a tool is illegal this phase, it is not “documented for later.” It is absent.

Truncation is a feature

A hard character or token cap on the assembled blob is not rude. It is how you stay alive. If the blob is over budget, drop the oldest events first, then shrink observations, then drop optional retrieval. Never drop the goal. Never drop the legal tool list until you have a smaller legal list.

Mark truncation in the context (truncated: true) so the model and the trace know the window is incomplete. Hidden truncation is how you get “it forgot the ticket id” tickets. Visible truncation is a signal to retrieve or summarize.

In the live box we use a tiny character limit so you can see the policy without a tokenizer. Production uses token counts. The policy is the same: measure, then cut with a rule, then tell the trace you cut.

Retrieved notes are not the scratchpad

Profile facts (“always wants C not F”) and policy snippets belong in a retrieve step or a small pinned block, not mixed into every observation. If you paste the user profile into the scratchpad, it will roll off. If you paste every policy doc, you will crowd out the last tool JSON.

Retrieve on demand when the goal needs it. Pin a short policy when every step needs it. Do not pin a wiki.

Assembly is testable

Because assembly is a function, you can unit-test it:

• Gather never lists refund.
• A fat history sets truncated.
• The goal string is always present.
• Observation text is capped.

No tokens required. If you only test the model, you will never catch “we advertised delete in the FAQ specialist.”

Common mistakes

• String-concat the week, then wonder why cost exploded.
• Advertise every tool “so the model has options.”
• Leave 50-page observations in the window.
• Hide truncation.
• Restate the goal as a poem every step.
• Treat the assembler as “the prompt file” instead of a function of state.

Run to execute this in your browser. Nothing is sent to a server.

Gather never sees refund. A fat history gets truncated. Apply sees refund and finish only. TOOLS_ALL exists in the file so you can see what was not advertised. That is the assembler doing its job.

Lower char_limit further and the window shrinks to one event. Raise it and truncation flips off. The goal stays. That is the budget: cut history, keep the contract.

How agents use this

Only advertise tools the dispatcher will run (tools track). Cap observation size before it reaches the assembler (same as packing RAG). If the window is full, summarize into memory — do not grow the prompt forever.

Log what was advertised, the window size, and whether you truncated. When a run copies an old mistake, read the assembler row first. Most “attention” problems are budget problems with a marketing name.