Memory Types
Working, episodic, semantic, and procedural memory are different stores — not one magic vector soup.
People say “give the agent memory” and then dump the whole chat into a vector database. That is one store pretending to be four. The operations differ. If you only have cosine, you cannot answer “did we already refund INV-17?” except by accident.
Four types, four jobs:
| Type | What it holds | Typical store | Failure if you skip it |
|---|---|---|---|
| Working | Current goal, last observations, budget | Context window + a JSON state | Goal drift, lost tool results |
| Episodic | What happened: traces, “last Tuesday” | Logs, time-indexed events | Cannot see what you already tried |
| Semantic | Facts: runbooks, prefs as statements | RAG corpus, profile rows | Hallucinated policies |
| Procedural | How to act: tools, playbooks | Code, schemas, git | A different procedure every Thursday |
Working memory is the live task. It is small and structured. The next lesson is only that store.
Episodic memory is a log of events with time. “We already replayed the DLQ at 10:04” is an event. You append. You filter by time and kind. You do not cosine for “yesterday” as your only clock.
Semantic memory is curated facts: the runbook, “Acme never cash.” This track’s RAG corpus is mostly semantic. Profile rows (never_cash: true) are semantic too, and often better than an embedded sentence.
Procedural memory is how to act: the replay_dlq function, the JSON schema, the git-pinned playbook. It should be deterministic. Embedding a wiki of “how we usually refund” as the only procedure is how Thursday’s bot differs from Wednesday’s.
Four stores, four jobs. One vector soup is how all four fail.
Four memory boxesWorking is JSON. Episodic is a log. Semantic is the handbook. Procedural is code.
Four stores, not one soupDo not vectorize everything
Vectors are an index for semantic prose. Using them as working memory loses order (sets of vibes, not a state machine). Using them as procedure loses determinism. Using them as the only episodic store makes “yesterday” a cosine accident: you retrieve a similar Tuesday, not this Tuesday.
A practical system has four APIs:
- mutate working state (rewrite JSON)
- append an event (episodic)
- upsert a fact (semantic, with trust rules — write-back lesson)
- call a procedure (code)
memory.add(text) feels unified, then you cannot answer “did we already refund INV-17?” because that is a row or an event, not a similar paragraph.
RAG in this track is the semantic library: chunks, search, citations. Working memory is the live JSON. Episodic is the trace. Procedural is tools. The agent loop that walks those APIs is later. Learn the stores first or the loop has nowhere honest to write.
Run to execute this in your browser. Nothing is sent to a server.
Four dicts, four jobs. act reads a semantic fact, writes working state, appends episodic events, calls a procedure. Production is the same with Postgres, an object store, and a tool registry. If you only have a vector DB, you have a hammer, and every memory looks like a chunk.
The lambda in procedural is a stand-in for a real function. Procedures live in code review, not in a cosine neighbor of “how to replay.”
Where RAG sits
Ingest + retrieve is how semantic prose gets into a prompt. It is not how you store “steps_left: 3.” It is not how you store “refund INV-17 ran at 10:04” unless you like incidents. You may index episodic summaries for search (“what did we try last outage”) — that is semantic-over-episodes, a second corpus with time filters, not a replacement for the event log.
Four APIs, four failure modes
Mutate working state: if you only append chat lines, the goal drifts and “thanks” becomes the task. Rewrite JSON fields.
Append an event: if you only cosine for “did we replay the DLQ,” you retrieve a similar Tuesday. Filter episodic by time, kind, and id.
Upsert a fact: if untrusted pages can write, next week’s retrieve is poisoned. Write-back is the next-plus-one lesson.
Call a procedure: if “how to replay” lives only as a wiki chunk, Thursday’s wording differs from Wednesday’s. Compile playbooks into functions and schemas. RAG can explain the procedure; code does it.
Semantic-over-episodes is allowed as a second index: summaries of incidents, tagged with time, retrieved with a time filter. The event log remains the source of truth. Do not delete the log because search exists.
Profile rows (never_cash: true) are semantic and better than an embedded sentence. RAG still holds the prose runbook. Both can be true: a boolean for the runtime, a paragraph for the citation.
When someone says “long-term memory,” make them pick a type. Then pick a store. One vector soup is how all four fail at once.
Common mistakes
- One vector index named memory.
- Embedding “ok thanks” as policy (next lesson).
- Letting procedures live only in wiki chunks so they drift.
- Asking cosine “did this id already refund.”
How agents use this
When someone asks for “long-term memory,” ask which of the four they mean. Then pick a store that supports that API. RAG in this track is mostly semantic. Working memory is the next lesson. Write-back is how semantic upserts stay honest. The Agents track will loop working memory. If you skip the split, the loop will memory.add a novel every turn.
Citations apply to semantic prose you retrieved. They do not apply to a procedure that is a function call: the trace is the citation.
Draw the four boxes on the incident doc: JSON state, event log, handbook plus profile rows, tool code. If the ask was “remember this refund,” that is either an event (we paid) or a trusted profile field (never cash), not a vector of the chat. If the ask was “what does the runbook say,” that is retrieve on semantic chunks. If the ask was “replay the DLQ,” that is a procedure. Mixing them is how cosine answers “did INV-17 already refund” with a similar paragraph from last quarter.
Check your understanding