Tests

An agent with no tests is a demo. It has not failed in front of you yet. You cannot unit-test “the model will be wise.” You can test everything around it: tools, parsers, and “is the goal done?”

assert is the smallest testing tool. If the check is false, Python raises AssertionError. On a laptop, pytest finds functions named test_*. Here we write a tiny runner that works in the browser. The idea is the same: known input, expected output, a report of pass and fail.

If you only run the happy path by hand, you will ship a parser that crashes on bad JSON. That crash is a missed test, not a mean model.

lambda is a tiny unnamed function

A lambda is a one-line function with no def name. You will see it in tests: lambda: add(2, 3) == 5. That means “run this check.” lambda can only hold one expression. No if block inside.

Prefer a normal def when the body is more than one expression. Lambda is only a shortcut for a single value. The runner stores (name, fn) pairs. fn is the lambda. Calling fn() runs the check.

lambda: add(2, 3) == 5 returns True or False. The runner then assert fn() is True. Using is True rejects a sloppy non-empty string that would pass if result. Done checks must be real booleans.

assert

assert 2 + 3 == 5
assert True is True

Write is True when the function must return a real True. assert some_text would also pass for a non-empty string. That is a sloppy “done” check. assert goal_satisfied(state) is True is the contract.

If you run Python with optimizations that skip asserts, tests would vanish. Beginners do not do that. In Joeven, assert works. On a laptop, pytest uses assert too.

Test a tool

A tool is a function. Call it with known inputs. Check the output. Do not call the live web. Do not call a paid model. Stub those.

def add(a, b):
    return a + b

assert add(2, 3) == 5

If add breaks, the test fails before any agent loop runs. Test edge cases: negatives, zeros. For search, test a known query against a fake index dict, like the mini-agent will.

A tool that hits the network is not a unit test. Inject a fake: pass a function that returns a fixed dict. The loop must run without a credit card.

Test goal_satisfied and parse_action

goal_satisfied(state) returns True when the job is done. If this function is wrong, the agent stops early or never stops. Test the happy path. Test missing fields. Test a bad URL. Test empty answer. Empty "answer": "" must not count as done if you require text.

parse_action(text) must accept good JSON and reject junk. Test both. A crash on bad JSON is a bug in your parser. Return a dict with ok: False. Then the test asserts ["ok"] is False.

A tiny test runner

The runner calls each check, prints PASS or FAIL, and counts. That count is the report. Catch AssertionError as FAIL. Catch other exceptions as ERROR — a crash in the test is not a clean fail. You want to see TypeError if a helper is broken.

On a real machine:

python -m pip install pytest
pytest -q

Joeven’s runner is the same idea without files. Name tests so a FAIL line is readable: goal_empty, not test3.

Keep tools boring in tests. If search hits the web, inject a fake. The loop must run without a credit card.

A table of cases is easier to extend than copy-paste asserts. The runner already is a table: name plus lambda. Add a row for “args is a list” and expect ok is False. Add a row for done missing. If you cannot name the row, you do not know the contract.

Tests that import your module must not start the agent. That is the __name__ == "__main__" lesson. If run_tests() is called at import time, a later pytest collection would run it twice. In this box, calling run_tests() at the bottom is the program. On a laptop, put that call in the main block.

Common mistakes

• Testing the model’s wisdom instead of your functions.
• assert result on a string.
• No test for bad JSON.
• Live network in unit tests.
• Asserting on print output only, never on return values.

Run to execute this in your browser. Nothing is sent to a server.

Break goal_satisfied so an empty answer counts as done. Watch goal_empty go red. That is the loop you want: tests spell the contract.

How agents use this

goal_satisfied is how “done” becomes checkable. The model can ramble. The function cannot. Test parsers so bad JSON becomes an observation, not a crash. Test tools so add(2, 3) stays 5. Later eval lessons test models. You start with assert. Agents without tests are demos.

The budget should be tested too: run the loop with max_steps=1 and a script that wants two tools. Expect stop. That test lives in the next lesson’s second run. Promote it to assert when you extract run_agent.

Do not replace tests with a bigger model. A smarter model still emits fences. Your parser still has to strip them. Tests are cheaper than hope.

A failing test is a gift if the name is good. goal_empty FAIL tells you the contract. test7 FAIL tells you nothing. When you change goal_satisfied, run the table. When you change parse_action, run the table. The mini-agent is allowed to use those same functions. Then a parser bug fails in 0.1 seconds instead of at step 6 of a live run.