Working Memory

Working memory is short, authoritative, structured. Not a novel. Not a vector index of every “ok.” It is the live task: what we are doing now, how much budget remains, what the last observation was.

Put goal, budget_remaining, last_error (or last_obs) in a JSON object you rewrite. Rewrite means the new state replaces the old object (or you update fields in place). You do not append a 400-page transcript to the object and call it state. Transcripts belong in episodic storage if you need them.

When the transcript grows, people summarize. Summaries are lossy. Keep the raw trace in episodic storage with a pointer from working memory (trace_id). Never let a summary become the only copy of a refund receipt, an id, or a policy exception. Those are fields or events, not vibes.

What does not belong

Do not embed every “ok” and “thanks.” That is how vector memory fills with noise and then retrieves “ok” as policy. Chit-chat is not a goal. Tool results are observations; user thanks are not.

Do not put the whole wiki in working memory. That is semantic retrieve + pack. Working memory may hold ids of packed chunks (evidence_ids: ["runbook.md#oom"]), not the 8,000-token dump, unless the dump already fits your cap — and it should not.

Cap working memory like you cap observations. If json.dumps(state) exceeds a limit, you have a bug: you stuffed a novel into a field. Fail or trim fields you declared trimmable, not the goal and not money ids.

Run to execute this in your browser. Nothing is sent to a server.

“ok thanks” did not overwrite the goal. The first user line is the goal; later chatter is ignored for that field. Last observation is the tool result, not the chit-chat. Episodic log holds tool events (two of them). Working state stays small. If you lower max_chars far enough, you get working_too_big — that is the cap firing. Raise it for the demo; keep a cap in production.

This is not the agent loop. There is no “think, act, observe” driver here. There is a compress function that other code will call after each observation. Learn the state shape first.

Rewrite after every observation

After retrieve, set last_obs to a handle (packed evidence ids + scores), not necessarily the full block if the assembler already has the block for this turn. After a tool (later track), set last_obs to the tool JSON. Decrement budget in the same rewrite. If you only append to a list named memory, you do not have working memory. You have a pile.

Working memory should be serializable (JSON) so you can store it with a job id and resume. Vectors of the chat are not a resume format.

Fields, caps, and what summaries may lose

Required fields: goal, a budget (steps or tokens remaining), last_obs (a short handle), maybe evidence_ids. Optional: last_error. Forbidden: the company wiki, every user thanks, a second copy of the packed DATA block unless it already fits a cap you enforce in code.

Rewrite after every observation: retrieve returns, you set evidence ids and last_obs handle; you do not concatenate the chunk texts into state. The assembler already has the packed block for this turn. Next turn, retrieve again or get_doc if you still need the parent.

Summaries: if the transcript is huge, summarize episodic text into a short field and keep trace_id. A summary may drop the exact cents. Cents belong in a field or an event, not in a paragraph the model wrote about the paragraph.

Cap: if json.dumps(state) exceeds N characters, fail or trim only fields marked trimmable. Never trim goal. Never trim money ids. “Working too big” is a product bug, not a reason to embed the overflow.

Chit-chat does not overwrite goal. First user task line wins until a new task is declared in a structured way, not until someone says thanks.

This is a store. The loop that calls compress is later. If the store is a novel, the loop cannot see the goal.

Common mistakes

• Goal overwritten by “thanks.”
• Budget only in the prompt prose, not in a field you decrement in code.
• Summary as the only copy of an id.
• Embedding working state “for long term” every turn.

How agents use this

Rewrite working state after every retrieve (and later, after every tool call). Cap its size like you cap observations. If you must summarize, store the summary and a pointer to the full trace.

The loop later reads goal and budget_remaining from this object. If you stuffed the wiki into last_obs, packing was skipped and the bill is the state. Keep last_obs short. Keep evidence in the packed data block for that turn, and ids in state if you need them next turn.

Working memory is a store. RAG is not it.

Serialize the object with the job id so a retry resumes the same goal and budget, not a new empty dict. Do not embed “ok thanks.” Do not let a summary be the only copy of an invoice id. Decrement budget in code, not in prompt prose. If dumps of state exceed the cap, that is a bug in what you stored, not a reason to start a vector index of working memory. The later loop will fail in readable ways if this JSON is small and true. First user task line stays the goal until a structured new task arrives. Tool results update last_obs; chatter does not. Working memory is short JSON you rewrite, not a novel you append, and not a vector soup of thanks. Cap it. Point at the full trace when you summarize. Resume from the job id with the same object.