When Not to Use an LLM

The first lesson said: if you can replace the LLM with a table or a workflow, do that. This lesson is the checklist you run before you open complete(messages). Fashion is not a reason. Latency, cost, and new failure modes are reasons to keep the workflow you already have.

Skip the LLM when:

• The answer is in a database and the query is known
• The action is a fixed form (refund with id + amount already validated)
• A regex or classifier already hits 99% on this intent
• You need a guarantee (checksums, totals, authz, “this hash matches”)
• The user needs the same deterministic paragraph every time (legal, medical dosing you are not allowed to generate)
• The state machine is short and you can draw it on one page

Use the LLM when:

• Language is messy and the schema is small
• You must propose the next tool from a documented list
• You must draft prose the user will edit
• The branch list is too long to hard-code, and you have evals

Hybrids win: rules first, LLM on the leftover, tools for facts, humans for money. A tiny router (even without a model) that says “this ticket has job_id and the word status → get_job” should not wait on a poem.

The test

Write the if-statement. If you can, ship it. If you cannot, write the schema, the cap, and the eval, then add an LLM step. Do not start with the model and hope the if-statement appears later. It will not. You will grow a prompt instead.

Replacing a working workflow with an LLM because it is fashionable is how you buy latency, cost, and hallucinations on a path that used to be a SELECT. Keep the workflow. Add the model on the messy branch.

Run to execute this in your browser. Nothing is sent to a server.

Status with an id does not wait on a poem. A structured refund does not wait on temperature. “How do I” can use a draft the human edits. The weird mixed ticket is why you still have a model — after a classifier, not instead of tools. Four prints, four policies. The last line is the leftover language this whole track exists to wrap: schema, cap, eval, then complete.

A classifier can itself be a small model. That still counts as “use an LLM,” but it is a bounded use: labels in an enum, temperature 0, no tools that spend money. Do not confuse that with sending the invoice to a frontier chat to “figure it out.”

Draw the branches you can draw

Sit with the last 50 tickets. Mark each: known id + known intent; known form; messy language; needs a draft. If 40 of 50 are the first two buckets, those 40 should never call a chat model. The remaining 10 get a classifier, then either a tool or a draft. That arithmetic is the product. A demo that sends all 50 to a frontier model will look smarter in a meeting and worse in a month.

Regex and keyword routers feel unfashionable. They are also inspectable. When they hit 99% on an intent, keep them. Use the model for the 1% leftover — with a schema — not as a replacement for the 99%. If the 99% path is a refund form with amount already validated, the workflow is: authz, cap, idempotency key, POST payment. The model does not sit in that path.

Guarantees

Authz is code. Totals are code. Checksums are code. The model may propose a refund. The executor checks the allowlist, the amount cap, and the idempotency key. If those fail, the poem does not matter. The first lesson called the LLM a guessing policy. This lesson is when you should not ask it to guess.

Deterministic legal paragraphs belong in templates. If legal needs the same sentence every time, a model that paraphrases is a bug, not a feature.

The same rule applies to “explain this error code.” If the mapping from code to paragraph is a table, use the table. If the user pasted a stack and you need a draft for a human, that is leftover language — LLM, then human. Do not auto-send the draft to the customer without a template check.

What goes wrong

• LLM for known SQL.
• LLM for checksums.
• LLM to replace a 4-branch form because a demo impressed a meeting.
• No leftover path, so messy tickets get a regex that is 60% and everyone pretends.
• Drafting legal copy without a template and a lawyer.

How agents use this

Every new skill starts as: can this be a tool + if-statement? If yes, ship that. If no, add an LLM step with a schema, a cap, and an eval. The next tracks (prompts, tools, RAG, agents) assume you already know when not to open this box.

Joeven’s fake clients exist so you can test the wrap without a model. If the test passes with a scripted assistant, you still need an eval when you turn the model on. If the test is the if-statement, you may not need the model at all.

Write the skip rule next to the skill: if ticket.job_id and "status" in text: return get_job(ticket.job_id). Review it like any other API. The LLM track ends here on purpose. Prompting, tools, RAG, and agent loops are for the leftover. They are not a license to delete the if.

Tip:Keep the workflow. Add the model on the messy branch. Known query + known tool is a workflow. Save the LLM for leftover language.