Context Engineering

Context engineering is packing the window on purpose. Prompt engineering was a slogan for the same job when the window only held a poem. Now you assemble a working set: just enough for this step. The Transformers track called the window a hard length. This lesson is which bytes you spend it on.

If attention is a spotlight, you choose the stage. You do not re-derive attention math here. You decide what the spotlight is allowed to see: the goal, the spec, the enabled tools, the latest observation. “Might help” is how you drown the model and the bill.

The cost lessons showed why fat tails hurt: every step resends the prompt. Prompt cache wants a stable prefix. This lesson is the packer that satisfies both: spec and tools first, volatile evidence last, drop the rest by priority.

The working set (priority)

1. Goal — what done means
2. Pinned spec — safety and schema
3. Tool docs — only tools that are enabled
4. Current observation — last tool result, latest user correction
5. Retrieved evidence — top chunks with ids
6. Compressed history — decisions, not every thought
7. Scratch — optional plan

Everything else is a candidate for deletion. The handbook paragraph loses to the observation. Evidence beats encyclopedia.

Patterns that keep agents small:

• Search after the model names a need, not 20 chunks up front.
• Keep job_id in a Python dict, not reread from a 12k essay.
• The writer does not need the SQL schema; the SQL tool caller does. Pass ticket_id, fetch in the tool.
• Disabled tools disappear from the docs and from examples. Leftover few-shots that call shell after you removed shell are how you get hallucinated tools.
• When you compress history, keep decisions and ids, drop chain-of-thought. A summary that says “looked up the job” without job_id=17 forces a guess. Prefer a template: step n: tool args=... result=....

Token estimates belong inside build_context. If packing exceeds the budget, drop by priority in code, then send. Asking the model to “be brief” after you already overflowed is pleading. If the vendor truncates the prompt, the spec at the front may survive (good) or, depending on side, the tail may survive and the spec dies (bad). Know which side they truncate. Prefer not to overflow.

Run to execute this in your browser. Nothing is sent to a server.

WINDOW is 40 words on purpose. Goal, spec, tools, and observation fit. The handbook and the thought-stream do not. dropped should include rag and history. That is a success: the model still has what it needs to quote timeout. If you reversed priority, you would keep the handbook and drop the observation, then hallucinate a status. Run it. Then lower WINDOW to 15 and watch even tools fall off — that is your cue to shorten the spec, not to beg the model.

Anti-patterns: pasting the entire repo; repeating the spec in every user message and the system; leaving failed JSON novels in history so the model imitates them (keep a one-line validator error); dumping another agent’s full window into this one; putting datetime.now() first (cache lesson).

Per-step packs

A classifier step needs the ticket text and a label enum. It does not need last week’s traces. A SQL tool-caller needs the schema of that database, not the poetry writer’s style guide. Build build_context(state, step_kind), not one mega prompt for every node. Routing models (next lessons) is cheaper when each node sees a small working set.

Failed JSON belongs in history as one line: validator: missing job_id. That error is the only extra the model needs on retry. Pasting the whole invalid object plus your anger paragraph teaches the model to imitate the invalid object. The structured-output lesson retries with the schema error; packing should keep that error and drop the rest.

Retrieved chunks need ids and a cap: top 3, not top 20. If the packer has to drop RAG to keep the observation, that is the correct drop. A handbook that never fits is a retrieval problem, not a reason to enlarge the window and hope. Lost-in-the-middle (Transformers track) is worse when you stuff twenty similar paragraphs. Packing fewer, better chunks is the API-level fix.

Code state is the other half of packing. state["job_id"] = 17 survives every trim. Inject job_id=17 as a one-line system or tool view each step. If the id only lives in a paragraph from step 2, step 8 will guess. Context engineering is as much what you refuse to put in the window as what you include.

Prompt cache wants spec + tool docs first, unchanged. The packer should concatenate in a stable order: spec, tools, then goal/user, then observation, then RAG, then history summary. Sorting tools alphabetically every time is packing and caching at once.

What goes wrong

• Might-help RAG on every call.
• Disabled tools still in few-shots.
• Summaries without ids.
• Packing in the prompt (“please ignore extra docs”) instead of in code.
• Overflow and blaming attention’s “lost in the middle.” Lost-in-the-middle is real (Transformers track). Packing still comes first: fewer tokens, better seats.

How agents use this

Write build_context(state) -> messages and unit-test it: spec always present, last observation present, token estimate under budget, no disabled tools, prefix bytes stable across tickets that share a spec. This function is the product. Models will come and go; packing policy stays.

If a piece of data must be true, keep it in code state and inject a one-line view. Code state does not get lost in the middle. A 12k essay does.

Unit tests for the packer are not optional: spec bytes present, last obs present, disabled tools absent, estimate <= budget, prefix of spec+tools identical across two tickets. When a developer adds “just one more paragraph” to the spec, the estimate test fails on the same night, not after the invoice.

Tip:Working set = decide and act now. Archives go to retrieval or summaries. If it must be true, it lives in code, not in hope.