The short answer
AI agent costs spiral when one business task expands into many model calls, repeated context, tool use, retries and verification steps. A cheaper model does not fix an unbounded workflow. Measure the cost per completed outcome, cap the work each run may consume, and route only the hard decisions to expensive models.
Why can an AI agent cost more even when token prices fall?
A chatbot usually answers one request. An agent may interpret the request, retrieve records, plan several steps, call tools, inspect the result, correct an error and produce a final response. Each step can send part of the conversation and operating instructions back to a model. The buyer sees one task. The provider bills a chain of calls.
That difference matters more than the headline price per million tokens. Gartner calls the effect the "Inference Paradox": improving unit economics can increase total AI spend because businesses use more capable systems for longer, more complex workflows. In August 2026, Gartner forecast that inference cost per agentic workflow will rise more than fivefold through 2028. It also said routing a task to an agentic reasoning model can cost at least five times a basic chatbot interaction, and often more as complexity grows.
The useful definition is simple: agent cost is the total compute consumed to produce an accepted business outcome, including failed attempts. Token price is only one input. If a workflow retries three times, reads an oversized context on every step and sends easy classification work to a frontier model, a lower token price can hide a badly designed operation.
This is separate from [what an AI agent installation costs](/en/notes/how-much-does-an-ai-agent-installation-cost/). Installation cost pays for discovery, connections, permissions, tests and release. Runtime cost begins when the system starts doing work repeatedly. Owners need to budget and control both, but they are not the same bill.
What actually drives the recurring bill?
The first driver is repeated context. Agents need enough information to make the next decision, but more context is not automatically better. Anthropic describes context as a finite resource with an attention budget. Long-running agents continuously create new observations, tool results and intermediate decisions. If every turn carries everything forward, the input grows even when the useful signal does not.
The second driver is model mismatch. A workflow may contain document classification, extraction, drafting, policy judgment and final approval. Those steps do not need the same model. Sending every step to the most expensive reasoning model is the AI equivalent of hiring senior counsel to rename files. The [business case for model routing](/en/notes/why-ai-model-routing-matters-for-business/) is not chasing benchmark winners. It is paying for judgment only where judgment changes the result.
The third driver is failure amplification. A tool timeout can trigger a retry. A weak completion check can make the agent repeat work that already happened. A vague objective can keep a loop alive because the system cannot decide when the task is complete. These are not rare edge cases once the agent touches real business systems. A five-cent attempt repeated twenty times is still a one-dollar failure, before anyone checks whether the result was usable.
The fourth driver is output and verification. Agent workflows often produce far more tokens than a short chat answer because they write plans, tool arguments, reports and corrections. Verification also costs money, but removing it blindly is the wrong fix. The goal is not the fewest calls. The goal is the cheapest workflow that reaches an acceptance standard without creating hidden rework or unsafe actions.
The cost model buyers should use
Do not budget an agent from the provider's input rate alone. Budget one completed task. For each workflow, record the total input tokens, output tokens, cached tokens, tool calls, retries and final status. Then divide total spend by accepted outcomes, not attempted runs.
A practical equation is: cost per accepted outcome = total model cost plus tool cost plus recovery cost, divided by accepted outcomes. Recovery cost includes failed runs that had to be repeated. Human review time belongs in the business case too, even when it does not appear on the provider invoice.
Suppose two agents each process 1,000 requests. Agent A looks cheap because its average model call costs less. But if only 700 results pass review and 200 requests need a second run, its true cost per accepted result may exceed Agent B, which uses a stronger model once and passes 940 results. Cost control without acceptance data rewards the wrong system.
Track at least five fields per workflow: cost per attempt, cost per accepted outcome, retry rate, percentage of calls sent to the expensive model, and the business event produced. That last field prevents a dashboard full of cheap activity from looking like value. A completed research memo, qualified record or approved follow-up is an outcome. A successful API response is not.
Which controls reduce spend without making the agent worse?
Start with a hard task boundary. Define what the agent may do, what proves completion and when it must stop. A maximum turn count is useful, but the better limit is tied to the workflow: one record, one decision, one approved action. If the system cannot complete inside that boundary, it should return a specific failure state rather than improvise indefinitely.
Route by decision difficulty. Use a lower-cost model for stable extraction, formatting and monitoring. Escalate ambiguous judgment, sensitive decisions or final review. OpenAI and Anthropic both publish different prices across model tiers, which makes routing an economic control, not just a technical preference. The exact rates change. The design principle does not.
Shrink context deliberately. Keep durable business records outside the prompt, retrieve only the evidence needed for the current decision, and summarize older working context when the task runs long. Anthropic's context-engineering guidance recommends compaction for long-horizon work and describes subagents that consume large working contexts but return condensed results. The saving comes from moving less irrelevant history through every later call.
Use caching when the provider supports it and the prefix is genuinely reusable. Official OpenAI and Anthropic pricing both distinguish cached input from ordinary input. Caching is most useful for stable instructions or reference material repeated across many calls. It does not rescue a workflow that changes its entire prompt every run, and it should not become an excuse to send unnecessary data.
Add per-workflow observability before optimizing. Tools such as Langfuse document token and cost tracking by model, use case, tags and users. You can build the same principle into your own records: every model call needs a workflow ID, task ID, model, token usage, cost and outcome. Without that attribution, the monthly invoice tells you that spending increased but not which business process caused it.
Finally, treat retries as a design signal. Separate transient failures from bad instructions and bad inputs. A transient network error may justify one bounded retry. A repeated validation failure needs a different prompt, model or workflow rule. Automatic retries without classification turn defects into recurring spend.
When is a more expensive model actually cheaper?
A higher-priced model is cheaper when it reduces the total number of attempts or prevents expensive downstream mistakes. Use it where the decision has real ambiguity, where evidence must be reconciled, or where a bad answer triggers human cleanup. Do not use it because the model is new or sits at the top of a benchmark.
The test is mechanical. Run the same representative task set through two designs. Compare accepted outcomes, retries, review time and total cost. If the cheaper route saves 40 percent per call but doubles the failure rate, it may lose. If a small model handles 90 percent of deterministic steps and escalates only the uncertain cases, it may beat both single-model designs.
This is why a [managed AI agent service](/en/notes/managed-ai-agent-service-monthly-what-is-included/) should report operation, incidents and controlled changes, not merely access to a model. Runtime management is the work of keeping cost, quality and risk inside an agreed boundary as the workflow and providers change.
A seven-step cost audit for an agent already in production
1. Choose one workflow and one week of real runs. Do not start with the entire AI budget.
2. Group all model calls and tool calls by task ID. Include failures and abandoned runs.
3. Mark which tasks produced an accepted business outcome. Define acceptance before reading the results.
4. Calculate cost per attempt and cost per accepted outcome. Add retry and human-review time.
5. Find the largest multiplier: repeated context, an expensive model on easy steps, excessive output, uncontrolled retries or unnecessary verification.
6. Change one lever and replay the same representative task set. Keep quality and safety checks fixed.
7. Set a warning threshold and a hard stop. Review the threshold when the workflow, model or pricing changes.
The order matters. Cutting prompts before you can connect spend to outcomes is guesswork. The expensive line item may be justified while a smaller, repeated call quietly creates most of the waste.
Limitations
Cost per accepted outcome does not prove business value by itself. A perfectly controlled agent can still automate a process that should not exist. Start with a workflow that has repeatable value, usable inputs and a measurable finish. [Choosing the right business process](/en/notes/how-to-choose-a-business-process-for-an-ai-agent/) comes before optimizing its inference bill.
Provider prices, cache rules and model behavior change. Recalculate from current invoices and official pricing rather than preserving a spreadsheet indefinitely. Cost controls can also reduce quality if they remove context the decision genuinely needs or force complex work onto a model that cannot handle it. The safe target is bounded, observable execution, not the lowest possible token count.
Some workflows carry risk that justifies extra verification. Financial actions, external messages, permissions and irreversible changes should not lose approval steps to save a small amount of inference cost. A cheaper agent that creates an untracked external mistake is not cheaper.
Turn the invoice into an operating decision
If your agent bill is growing and nobody can explain which workflow produced the spend, the first fix is not another model. Map one workflow, define its accepted outcome, trace every call and put a boundary around retries and escalation. That gives you a cost you can defend, improve or kill.