Agents vs Chatbots vs Workflows

Three designs get mixed up in product meetings. They all might contain an LLM. They are not the same product. If you pick the wrong one, you will spend months “debugging the agent” when you actually needed a flowchart.

A chatbot takes a user message and returns an assistant message. It may use RAG (fetch a document, then answer). It has no side effects unless the user copies something and does the work. Good for Q&A, drafting, tutoring.

A workflow (also called a graph, pipeline, or DAG) is a path you already know. A DAG is a directed acyclic graph: steps with arrows, no loops that run forever by design. Example: extract, retrieve, generate, validate, save. An LLM may sit inside a step. Control flow is your code. Good for invoice processing, deterministic ETL with an LLM extract, most business automation. ETL means extract, transform, load — move data from one place to another in known stages.

An agent is the design where the model chooses the next step. You cannot write the DAG in advance because the path depends on observations. Good for research, messy debugging, computer use, open-ended ops, games, coding until tests pass.

People confuse them because demos look the same: a text box, a spinner, a paragraph. The difference is who owns the branch. If your code always classifies, then retrieves, then answers, you have a workflow even if a vendor stamped “agent” on the repo.

Side by side

Chatbot. Cheap to eval as “did the answer match the doc.” Hard to mistake for an agent unless you add tools and forget to say so.

Workflow. You can test step 2 without calling a 70B model. You can log which step failed. You can put schema validation after extract. This is the default for money-moving systems.

Agent. You need a trace, a budget, and a small tool list. You need evals that score trajectories, not just final prose. You need a human gate for irreversible actions.

The expensive mistake

Teams wrap a two-step workflow in an agent framework, then spend months debugging why the model skipped step 2.

Rule: if you can draw the flowchart without a diamond that says “LLM decides,” it is a workflow. Use a workflow.

Even when you need an agent, constrain the action space. An agent with 8 tools beats an agent with 80 tools. Unknown number of searches, unknown files, unknown whether the task is possible — those are agent-shaped. “Always extract, then write to the database” is not.

Autonomy without evals is just randomized production incidents.

A billing ticket that should not be an agent

A ticket says “My invoice is wrong.” Another says “The runner crashed.” A third mixes both. Product wants “an agent that handles support.”

Look at the path you already know: classify the ticket, fetch two canned docs for that kind, return the first doc as the answer plus sources. That is three steps. You can write them in Python. In production, classify might be an LLM call with a schema: billing or tech. Retrieve might be a database. Answer might be a template. The LLM never chooses to skip retrieve. That is a feature.

The box below is that workflow. Control flow is in Python. You can test it without mocking a giant model. Watch what happens on a mixed ticket that contains both “invoice” and “runner.” A brittle if "invoice" in ticket will pick billing and ignore the crash. That is a workflow bug you can see. An agent would hide the same bug inside a trace.

Run to execute this in your browser. Nothing is sent to a server.

The first print is billing, answer about refunds, both billing docs as sources. The second is tech, restart the runner. The third is also billing, because the string contains invoice. The crash is ignored. That output is the lesson: a workflow makes the branch visible. If you need both topics, you change your classify step (for example, allow two kinds). You do not give a model eight unrelated tools and hope.

When the LLM should choose

Use an agent when the branching factor is high. Branching factor means how many next steps are plausible.

• Unknown number of searches
• Unknown files to open
• Unknown tools (calendar? browser? shell?)
• Unknown whether the task is even possible

Still constrain tools. Still write a goal check. Still prefer a workflow for the parts you do know. Hybrid systems are normal: a workflow that calls a small agent inside one diamond, then returns to your code.

Watch out:Autonomy without evals is just randomized production incidents.

What goes wrong

• Agent-washing a workflow. Two known steps, wrapped in ReAct, now skip step 2 at 3 a.m.
• Chatbot with a refund button in prose. It says it refunded. No function ran. The customer waits. Finance is confused.
• Eighty tools. The model picks a nearby wrong one. Narrow APIs beat a Swiss army knife.
• No schema after extract. The workflow writes junk rows to the database. The agent version writes junk rows and wanders.
• Mixed tickets with one keyword. Like the Try it box. Keyword classify is a start, not a product.
• Swarm first. Multiple agents for a three-step invoice pipe. You added coordination cost and no new information.
• User-facing “agent” that cannot act. It is a chatbot. Call it a chatbot so people do not expect side effects.

Where this goes next

Tools will show how to wrap one function so either a workflow or an agent can call it. Prompting is how a classify step returns JSON instead of a poem. Evals are how you catch the mixed-ticket failure with a test. Multi-agent is for when roles truly split, not for invoice extract. Production is queues and approval gates for the side effects. This lesson only teaches you to name the three designs.

How agents use this

Default to a workflow. Promote a step to an agent only when you can say which observation would change the path.

• Code: write the flowchart as functions in order. If a step needs an LLM, that function returns data, not “whatever the model felt.” Keep an allow-list of kinds (billing, tech) rather than free text.
• Logs: log kind, retrieved ids, and whether a side effect ran. For an agent, log each chosen tool. For a workflow, log each step name. Same idea, shorter traces for workflows.
• Tests: invoice wording must classify billing. Runner wording must classify tech. Mixed tickets must match the rule you document — two labels, or a priority — not a surprise.
• Stop conditions: workflows stop at the end of the graph, plus validation failure. Agents stop on goal, budget, or handoff. Do not run an agent loop around a graph that already has an end.

You must extract fields from PDFs, then write rows to a database? That sentence already is a workflow. Put the model inside extract. Validate the schema. Then write. The next lesson is the map of Joeven so you know where classify, tools, and evals will be taught.