Chat Transcripts

A transcript is an ordered list of messages with roles. The model does not remember yesterday. It reads this list. If the list is wrong, the policy is wrong. Most “the model ignored the tool” bugs are omitted or mis-ordered messages, not mystical attention.

The previous part taught the HTTP POST. This part is the document inside the POST: who spoke, in what order, and what you must never invent.

Roles

Some vendors use developer instead of or besides system. Some put tool calls as structured fields on the assistant message, not as text. Your wrapper should normalize to one internal schema. The loop should not branch on vendor role names.

The last message you send is usually a user or tool observation you want a reaction to. If the list already ends on assistant, you already have an answer — calling again is asking for a sequel, which is how loops ramble.

Hygiene

• Do not invent an assistant message the model did not produce (except a controlled stub after a filter, and log that it is a stub).
• Do not drop tool results. The next call cannot “know” a job failed if you never appended it.
• Do not leave secrets in logs if you cannot store them. Redact when you serialize for debug.
• Do not concatenate two user goals without a boundary. Two asks in one user blob is how one of them is skipped.
• Do not promote tool text into system “so it is official.” Next lessons.

Users edit an old message in a chat UI. Agents should append a correction so the policy sees the change: “user correction: job id is 18, not 17.” Rewriting history in place makes traces unreproducible.

Parallel tool calls: one assistant turn, several tool messages with ids. Results may arrive out of order. Match by id, not by “the last one.”

Run to execute this in your browser. Nothing is sent to a server.

good prints ok (or an empty error list). bad lists missing pinned system, a tool result with no assistant before it, and a transcript that ends on assistant. The cheerful “All good!” is the kind of final line people screenshot. The validator would have refused to send that document as a next prompt, and it flags that the tool result appeared from nowhere.

You can disagree with a rule (some APIs allow system later). The point is to have rules and run them before POST. A validator is cheaper than a mystery.

The transcript is state

A ReAct agent’s state is largely this list plus a bit of code state (ids, budgets). Serialize it. Version the schema (transcript_v3). When a run is bad, replay the same messages into a fake client and assert the next tool. Name messages in logs: msg_12 tool get_job.

Render transcripts for humans (a pretty thread). Store the machine form separately (roles, ids, tool names, raw arguments). Hide chain-of-thought if the vendor forbids showing it. Pretty and machine are two views of one object. If you only store pretty HTML, you cannot replay.

Version the schema: role, content, optional tool_id, optional tool_calls list. When a vendor uses function instead of tool, map it in the adapter so the rest of the code sees tool. Do not let vendor names leak into the packer.

Trim from the middle of history, not from the spec, and not from the latest observation. Keep a summary of dropped steps as one user or system-adjacent line if you must, but prefer a compact step n: ... list you built in code. Users editing old UI messages: append a correction; do not rewrite msg_3 in place or your replay diverges from what the model saw at the time.

A walkthrough: the missing tool row

Support says the bot ignored the outage. The pretty UI shows a tool card. The stored messages never included the tool role — a frontend component faked the card from the assistant’s prose (“I called get_job”). Replay fails: the fake client never sees a failure observation, so the next answer stays sunny. The fix is not a better model. The fix is: UI cards are driven from tool messages, not from assistant claims. Hallucinated actions are a later lesson; the hygiene starts here.

What goes wrong

• System prompt duplicated in user “for emphasis,” then truncated, so only the user copy remains and it is weaker.
• Dropping failed JSON from history so the context is “clean,” then the model repeats the same invalid shape with no error to learn from. (Keep a short validator error, not a novel.)
• Merging two tickets into one transcript. Goals collide.
• Using user role for tool results because “the model trusts the user more.” It also lets a webpage look like a human.
• Ending on assistant and calling again without a new observation. Sequels.

How agents use this

The transcript is the state of the loop. build_context(state) -> messages should be unit-tested: first role system, tool rows only after assistant tool calls, last role is what you intend to react to, token estimate under budget.

When you trim, drop old thoughts, keep decisions and ids. A summary that says “looked up the job” without job_id=17 forces a guess. Prefer a template: step n: tool args=... result=....

Validate before POST. A 20-line validate_transcript saves a day of “the model ignored the tool.” Tests: good list ok; tool-first list errors; missing system errors; ends-on-assistant errors if you were about to call again without a new user/tool row.

Tip:Render transcripts for humans. Store the machine form separately. Hide chain-of-thought if the vendor forbids showing it. Most “memory bugs” are omitted messages.