JJoeven

Reference/LLM APIs

Chat messages and roles

system / user / assistant / tool roles, ordering, and what not to put in a system prompt.

Vendor chat APIs take a list of messages. Joeven uses the same shape even with fake models.

Roles

RoleWho writes itContains
systemyoupolicy, JSON schema, tool docs
userhuman or supervisorgoal
assistantmodelthoughts, tool calls, answers
toolyour loopobservation JSON

Some APIs use function instead of tool. Same idea.

Ordering

  1. One system (or zero)
  2. Then alternating user/assistant, with tool messages after the assistant that called them
  3. Trim old tool bodies first when over budget

Do not put in system

  • API keys
  • Raw PII
  • Entire handbooks (use RAG)
  • "You are GPT-5" roleplay that fights the schema

Minimal payload

python
[
  {class="tok-s">"role": class="tok-s">"system", class="tok-s">"content": class="tok-s">"Reply with JSON {tool, args} only."},
  {class="tok-s">"role": class="tok-s">"user", class="tok-s">"content": class="tok-s">"Weather in Paris?"},
]

After a tool:

{"role": "tool", "name": "geocode", "content": "{\"lat\": 48.86}"}

Keep content a string on the wire even if you store dicts internally.

Watch out:Concatenating all roles into one string ("Transcript:") is fine for a fake model. Live APIs want the list.