Working Memory in the Loop

The RAG track named four stores. This lesson is how the loop uses them. Memory is not a vibe and it is not one cosine search over everything the agent ever said.

The assembler reads memory. Memory is not the context window. The window is a view over stores. If you dump the whole week into every call, you skipped the assembler budget. If you retrieve a thought from last Tuesday as if it were a fact, you skipped the distinction between stores.

Do not dump the whole week into every call. Do not treat “memory” as one vector soup.

Scratchpad is RAM

The scratchpad is the recent act/obs pairs. It is the ReAct scratchpad with a budget. Last N turns, always assembled. This is how step 4 sees step 3’s observation. Observe-before-finish depends on this store being honest.

When the window is full, older turns roll into a summary. They are not deleted forever (you may retrieve later). They are not pasted into every future prompt. They are not written into the system prompt as law. RAM overflows into a summary. The assembler stays on a budget.

N is a product choice (2, 4, 6). The live box uses 2 so you can see a roll. Production might use token weight, not turn count. The policy is the same: bound the RAM.

Summary is lossy on purpose

A rolling summary is a compress: “searched docs, mentioned Paris.” You lose detail. That is the trade. If you need the detail again, retrieve (if you stored notes) or look at the trace (operators). Do not keep an unbounded summary either. Summaries that grow forever are a second furnace.

Summarize with code when you can (join old acts). Summarize with a model call when the window is large. Cap that call. Do not summarize every step if N is small.

The last observation should still sit on the scratchpad, not only in the summary. Truncate old, keep last. The assembler lesson said this. Memory is how you implement it.

Retrieve is RAG, not reminiscence

Notes you remember() are durable facts or user preferences. Retrieve them by query when the goal needs them. “User wants C not F” is a profile note. It should not ride the scratchpad until it rolls off and dies.

Do not index every thought as a note. Thoughts are not facts. If you only have one vector index, you will retrieve a thought from last Tuesday as if it were a ticket status. Split stores even if the split is “list of strings” vs “last N dicts.” Vectors can wait. The loop needs the split now.

Profile is stable

Profile is user or org facts that change rarely. Do not mix them into the scratchpad. Pin a short profile or retrieve on demand. If the user changes units, update the profile store, do not append a contradictory scratch line and hope.

What the assembler should receive

for_assembler(query) returns a small object: summary string, scratch list, retrieved notes. That is the memory API. The assembler then applies a token budget. Memory should not return a novel. If notes are many, retrieve already ranked a few.

Typed state (next lesson) is not the same as these stores. Phase and ticket_id are a contract object. Scratchpad is a log. Do not encode phase only as a sentence in the summary. You will parse it wrong.

Persistence

Long-running jobs later serialize memory into a store. Scratchpad plus summary plus notes plus profile. JSON is enough. Do not pickle a 4k “memory” string and call it architecture. Checkpoints should round-trip these fields.

What not to remember, and when to retrieve

Not every string the model emits is a note. Thoughts are working scratch; they die with the window or live in the operator trace. Observations that are huge HTML should be truncated before they become scratch, then maybe stored as a pointer (doc id) you can retrieve. If you remember() every observation, retrieve becomes a junk drawer and the assembler will pack last Tuesday’s rain into a refund prompt.

Retrieve when the goal needs it, with a query you can log: “user units,” “ticket T1,” “policy refund window.” Vague queries (“context”) return noise. Retrieving on every step with the whole user message as the query is how you blow the budget and still miss the profile line. Once per phase is a reasonable default: gather may retrieve policy, apply may retrieve the ticket note, done retrieves nothing.

Profile updates are explicit. The user said “always Celsius” — that is remember, not an obs on the scratchpad. If you only append it as a turn, window 2 will roll it into a summary that might drop the word Celsius. Stable facts get a store that does not roll.

Amnesia in traces looks like thrash: same search, same q. Before you blame the model, print for_assembler for that step. Empty scratch is a missing add_turn. Empty notes with a profile that should have hit is a bad query. Fat scratch with an old failed call is the assembler imitating errors — memory did its job too well and the window was wrong.

Four stores is the design even if two of them are lists of strings in a dict. You can add vectors later. You cannot add a split after you indexed thoughts as facts.

Common mistakes

• One vector index for thoughts, obs, and profile.
• Unbounded scratchpad.
• Deleting old turns forever with no summary and no trace.
• Putting profile in the system prompt as a page of wiki.
• Retrieving on every step with a vague query.
• Encoding ticket_id only in prose.

Run to execute this in your browser. Nothing is sent to a server.

Window 2: the first search rolled into summary. Scratch length is 2 (rain, finish). The assembler still sees recent turns plus a retrieved profile note because the query was "user". Change the query to "zzz" in your head: notes would be empty. Retrieve is not a dump.

The summary is a string join, not a second model. That is enough to learn overflow. A fancier summarizer still writes to self.summary, not into the system prompt.

How agents use this

Scratchpad is the loop’s RAM. Summary is a cheap compress. Retrieve is RAG. If you only have one vector index, you will retrieve a thought from last Tuesday as if it were a fact.

The assembler budget and this lesson are one design: stores first, then a window. Typed state next is the third piece: a contract object the assembler also reads, so legal tools do not live only in prose.