Parse JSON from a Model

Language models often wrap JSON in a markdown fence: three backticks, the word json, the object, then three backticks again. They also add chat around it: “Sure!” before, “Hope this helps” after.

Your runtime must still get a dict like {"tool": "search", "args": {"q": "rain"}}. The parser is a gate. If the gate is eval, the model can run Python. If the gate is json.loads, the model can only send data. Data can still be a wrong tool name. That is the allowlist’s job. Running code is the parser’s job to refuse.

Models wrap JSON in fences

A clean action looks like this:

action = {class="tok-s">"tool": class="tok-s">"search", class="tok-s">"args": {class="tok-s">"q": class="tok-s">"rain"}}
print(action[class="tok-s">"tool"])

A messy model reply looks like: extra words, a fence line, the same object, a closing fence, more words. You must strip the wrapper. Then parse. You do not need a full markdown parser. You need: trim, drop a first fence line, drop a last fence line, find the outermost object.

Fences might say json or say nothing after the backticks. startswith on the first line is enough. Do not require the word json. Require the backtick prefix.

Strip, then json.loads

json.loads turns JSON text into a Python dict. If the text is not JSON, it raises json.JSONDecodeError. Catch that. Return an error dict. The loop can retry the model or stop. Do not crash the agent.

Never use eval() on model text. eval() runs Python. A model can write dangerous calls. json.loads only reads data. That is the point. exec is the same class of mistake. Do not use it on model text either. This site’s lessons will not show a working eval on purpose.

Building JSON with glued quotes also breaks when a value has quotes. Use json.dumps to write JSON. Use json.loads to read it.

Extra text before and after

A useful trick: find the first { and the last }. Slice that piece. Then json.loads. Chat before and after drops away. Nested args still work, because the last } closes the outer object — if the model printed one object. If it printed two objects, last } still closes the second one and you may parse the wrong span. Prefer one object per reply.

If there is no {, you have no object. Return an error dict. If } comes before {, also fail.

str.find returns -1 when missing. str.rfind finds from the right. Together they are the brace slice. Regex is optional here. Brace find is easier to test.

A robust parse_action

parse_action(text) should return {"tool": name, "args": dict} on success. On failure it returns the same keys plus "error". The caller always gets a dict. No crash.

Accept tool or name. Accept args or arguments. Models drift. Normalize to one shape: tool and args. Then the loop has one reader.

If args is missing, use {}. Then the tool might TypeError on a required field. You can treat missing args as an error instead. Either way, never pass a list as **args. Check isinstance(args, dict).

If the loaded value is a list, it is not an action object. Return not_object. Do not take [0] and hope.

Strip before you extract braces, or extract then parse — both can work. The tryit strips fences first so a fence line of backticks is not part of the JSON. Then it slices from first { to last }. If the model put JSON in a fence and chatted after the closing fence, the last } still belongs to the object if there is only one object. If it also put a second {...} example in the chatter, last } is the wrong end. Prompt the model for one object. The parser is a seatbelt, not a mind reader.

Keep detail on parse errors for you. The model can see bad_json without a stack trace. A huge exception string in the next prompt wastes tokens and can leak paths.

Common mistakes

• eval / exec on model text.
• Crashing the loop on JSONDecodeError.
• Assuming fences always exist, or never exist.
• ** unpacking a non-dict.
• Parsing without logging the raw text on failure.

Run to execute this in your browser. Nothing is sent to a server.

You should see three successful tool / args dicts, then an error dict. No eval. If parse fails in a real loop, you send the error back as an observation — or you retry the model. You do not run the text as Python.

How agents use this

Every tool-using agent parses model text into a tool name and an args dict. Models will wrap, chatter, and typo. Your parser is the gate. If the gate uses eval(), the model can run anything. If the gate returns an error dict, the loop can recover. Later lessons reuse this parse_action.

Keep the raw model text in the transcript even when parse fails. Then you can see the fence you failed to strip. A parser that only stores bad_json without the raw line cannot be improved. Redact secrets first. Then store.

Tests should include: clean JSON, fenced JSON, chat wrapping, not JSON, a list, missing tool, args as a string. That set is this lesson’s try-it plus two more cases. The mini-agent lesson will parse clean JSON only, on purpose, so the loop stays readable. Production should use the robust parser.

Never feed parse errors into eval as a “fallback.” There is no fallback that runs model text. The only fallback is retry the model, or stop, or ask a human. json.loads plus an error dict is the whole parser contract. Everything else is stripping wrappers so loads can see the object.