JJoeven

Curriculum/Python

Comprehensions

Build lists, dicts, and sets in one expression. Filter a trace of dicts. Use a for-loop when the body is more than one step.

intermediate20 min24 / 37

A comprehension is a short line that builds a new list, dict, or set from an old one. It is a loop packed into one expression. The result is a new collection. The old one is not changed.

Agents store a trace: a list of dicts, one dict per step. You often want names, failed steps, or unique tools. A comprehension does that in one line. They are not magic. Keep them short. If the line is hard to read, use a for-loop.

You already know the long form: make an empty list, for, append. The short form is for when the body is one value and maybe one test. That is most “filter this trace” work.

List in, list out
tracekeep failednew list

A comprehension builds a new list. The old trace stays. If the body is more than one step, use a for-loop.

List in, list out

List comprehensions

A list comprehension makes a new list.

FormMeaning
[x["name"] for x in trace]Take one field from each row
[x for x in trace if not x["ok"]]Keep only some rows
{x["name"]: x["ms"] for x in trace}Build a dict
{x["name"] for x in trace}Build a set of unique names

Read it as: “make a list of this, for each item in the old list, if this test.”

python
trace = [{class="tok-s">"name": class="tok-s">"search"}, {class="tok-s">"name": class="tok-s">"read"}]
names = [row[class="tok-s">"name"] for row in trace]
print(names)

That is the same idea as:

python
names = []
for row in trace:
    names.append(row[class="tok-s">"name"])
print(names)

Use the short form when the body is one simple value. If you need a comment in the middle, you needed a loop.

You can walk any iterable: a list, a tuple, a set, range, dict keys. [n * 2 for n in range(4)] is [0, 2, 4, 6]. Prefer naming the source: for row in trace, not for x in t.

Filter a list

Add if at the end to keep only some items. There is no else in a simple filter comprehension. You either keep the item or you skip it.

python
failed = [row for row in trace if not row[class="tok-s">"ok"]]
slow = [row[class="tok-s">"name"] for row in trace if row[class="tok-s">"ms"] > 100]

The test uses truth. if row["ok"] drops rows where ok is False. If ok might be missing, use if not row.get("ok") or a loop with .get. A missing key in row["ok"] is still KeyError inside a comprehension.

You can have more than one for (nested). Nested comprehensions get unreadable quickly. Nested loops are allowed to stay loops.

WantComprehensionLoop instead when
Field from each row[row["name"] for row in trace]you also print
Some rows[row for row in trace if ...]missing keys, try
Last value per name{row["name"]: row["ms"] for row in trace}you wanted all times
Unique names{row["name"] for row in trace}you needed order

Dict and set comprehensions

A dict comprehension builds a mapping. If a key appears twice, the last value wins. That is useful for “last latency per tool name.”

A set comprehension keeps unique values. Order is not the point. Uniqueness is. {row["name"] for row in trace} is the set of tools that ran.

python
last_ms = {row[class="tok-s">"name"]: row[class="tok-s">"ms"] for row in trace}
tools = {row[class="tok-s">"name"] for row in trace}
print(last_ms)
print(tools)

Do not use a set comprehension as the transcript. You would drop duplicates and order. Same rule as the sets lesson.

Filter a trace of dicts

A trace is a list of dicts like {"name": "search", "ok": True, "ms": 120}.

Common jobs:

  • keep only tool rows
  • drop failed steps, or keep only failed steps
  • make a smaller dict for the next prompt (fewer keys, fewer tokens)

Name the result. failed is better than a long line inside print. Named results are testable: assert len(failed) == 1.

Compacting a row is often a loop because you also print:

python
compact = []
for row in trace:
    if row[class="tok-s">"ok"]:
        compact.append({class="tok-s">"name": row[class="tok-s">"name"], class="tok-s">"ms": row[class="tok-s">"ms"]})

That is two actions (test and build a smaller dict) plus maybe a print. A comprehension can build compact in one line. The print cannot live inside it cleanly. When you need the print, keep the loop.

Token budgets often start with a comprehension: rows = [r for r in trace if r.get("role") != "debug"] then maybe rows[-8:]. That is filter then slice. Two clear steps. Do not pack both into an unreadable line.

Walkthrough: last-wins and .get

python
trace = [
    {class="tok-s">"name": class="tok-s">"search", class="tok-s">"ok": True, class="tok-s">"ms": 120},
    {class="tok-s">"name": class="tok-s">"search", class="tok-s">"ok": True, class="tok-s">"ms": 80},
]
last = {row[class="tok-s">"name"]: row[class="tok-s">"ms"] for row in trace}
print(last)  class="tok-c"># {'search': 80} — last wins

If you needed both times, you wanted a list: [row["ms"] for row in trace if row["name"] == "search"]. Dict keys are unique. Last write wins. That is not a bug in Python. It is the wrong collection if you needed history.

Optional fields:

python
safe = [row for row in trace if not row.get(class="tok-s">"ok")]

.get("ok") is None when missing, and not None is True, so missing ok counts as failed. Square brackets would crash the whole comprehension on the first bad row. A loop can skip one row and keep going. That is why messy traces prefer loops.

When a for-loop is clearer

Use a for-loop when you need more than one action:

  • append and print
  • handle a missing key
  • update two lists
  • nest a lot of tests
  • try/except per item

A comprehension should not hide a whole program. If the line does not fit on the screen, it is not a good comprehension. Write a loop.

There is also a generator expression: (row["name"] for row in trace). It is lazy. You do not need it to filter a normal agent trace. list(...) around it makes a list. Stick to [...] until you have a huge file.

Side effects inside a comprehension (print, append to another list) are legal Python and a bad habit. Readers expect a new collection, not a second mutation. Put mutations in a loop.

What goes wrong

  • Side effects inside a comprehension (print, append to another list).
  • KeyError because you used [] on optional fields.
  • Set comprehension as a log.
  • Nested comprehensions nobody can read.
  • Forgetting that dict comprehensions overwrite duplicate keys — sometimes you wanted a list of all times, not the last.
  • Packing filter plus slice plus a nested dict into one line.
Live PythonOpen full playgroundpython
Output
Run to execute this in your browser. Nothing is sent to a server.

Add a fifth row with a high ms and watch slow take the last value for that name. That last-wins rule is the dict comprehension. safe uses .get so a missing ok would not crash.

How agents use this

An agent trace is a list of dicts. You filter it before you send it back to the model. Keep failed tools. Drop huge fields. Take unique tool names with a set. If you also need to print or catch errors, use a for-loop. Short lines are a tool, not a rule.

Eval reports use the same idea: failed = [c for c in cases if not c["pass"]]. Then len(failed) is the scoreboard. You will write tiny test runners later. They are loops. The filter of results can be a comprehension. Mix them on purpose, not by habit.

Compacting for tokens is the daily job: keep role and a short content, drop debug rows, then slice the tail. Three named results beat one clever line. Tests can assert len(compact) <= 8 and "html" not in str(compact).

Allowlist diffs are a set comprehension plus a set: {row["name"] for row in trace} - ALLOWED. Tools the model used that you never allowed. That is a policy report in one expression. Keep ALLOWED as a set. Keep trace as a list.

When a comprehension crashes halfway, you get no list at all. A loop can append the good rows and record the bad index. At the trust edge (model JSON, files), prefer the loop. On traces you built with a stable shape, comprehensions are fine.

Check your understanding

When is a for-loop clearer than a comprehension?