AI Agent Cost Per Success: The Hub That Replaces Cost Per Task
Cost-per-task is the metric everyone publishes; cost-per-success is the metric that actually breaks cheap-first routing myths. How Tacavar measures agent economics.
Every AI agent vendor publishes cost-per-task. It is the number that fits on a slide, the metric that makes a cheap model look like a bargain, and the benchmark footnote that never mentions the failures. Cost-per-task is easy to compute: take total inference spend, divide by tasks attempted. But it is the wrong metric. It measures what you paid to run, not what you paid to complete. Cost-per-success — the fully loaded price of an outcome that actually satisfies the business requirement — is the metric that breaks cheap-first routing myths and exposes where agent economics truly leak.
Across deployments on Tacavar's production stack, we have watched the same pattern repeat: a team optimizes cost-per-task, celebrates a 40% reduction, and then discovers that cost-per-success went up because failures now require expensive re-runs and human cleanup. This article is the hub that ties those threads together: how to measure cost-per-success, why it defeats cheap-first routing, what to instrument, and how to run the loop that replaces cost-per-task as your operating metric.
The two ways to measure an agent's cost
Cost-per-task is the arithmetic average of every inference dollar spent divided by the number of tasks the agent attempted. It includes no judgment about outcomes. A task that produced a hallucinated answer, which a support agent later had to manually unwind, still counts as one task. A retry loop that fired three times before producing a valid schema still counts as one task. Cost-per-task is a volume-adjusted meter reading, not a unit-economics number. It is the metric your vendor dashboard defaults to because it always trends downward when you swap in a cheaper model.
Cost-per-success is the fully loaded cost per task that meets a predefined success criterion — and the criteria differ per task class. It includes every inference call across all retries, the cost of the verifier that confirmed the outcome, downstream remediation when a failure propagated, and human review minutes when a human had to confirm or fix the output. When you compute cost-per-success, cheap models stop looking cheap. A model that fails 30% of the time and costs one-fifth the token price can easily produce a higher cost-per-success than a mid-tier model that fails 5% of the time. The math is brutal because failure is never free, and the failure cost of an agent is the cost of the most expensive recovery step, not the cost of the failed call.
For a deeper look at how published benchmark numbers mislead, see our analysis of why cost-per-task benchmarks overstate the value of cheap models.
Why cheap-first routing can triple cost-per-success
Cheap-first routing is the most common pattern we see in agent production stacks: send the request to the smallest, cheapest model, and escalate only when it fails. It sounds prudent — fail fast, escalate only when needed. In practice, it triples cost-per-success on a meaningful class of workloads. Here is the mechanism from our production telemetry.
On a technical-support triage agent, one pod routed roughly 80% of tickets to a small fast model with a mid-tier fallback. The small model produced a classification that looked plausible, passed a light confidence check, and was wrong. The wrong classification sent the ticket down an orchestration path: API calls fired, a mid-tier model drafted a response from the wrong context, and a human had to unwind the entire thread. The recovery cost was higher than the cost of simply using the mid-tier model up front. The small model delivered a 0.35x cost-per-task versus the mid-tier, but a 2.7x cost-per-success.
Three failure modes drive that pattern:
- Silent wrongness: the cheap model fails confidently, and the workflow only discovers the failure after downstream work has been committed.
- Verifier distrust: if you know a model is unreliable, you must verify every output. Verification often costs more than the model call it checks, making the cheap model the most expensive component in the pipeline.
- Retry cascades: cheap models fail erratically, so escalation paths are hit non-deterministically. You cannot predict the cost distribution, and tail costs climb.
Once the telemetry was in place, the fix was boring: route the task class directly to the mid-tier model and delete the cheap-first path. Total spend dropped by more than half. We documented the full case in cost-per-success as the LLM routing metric.
Defining success for deterministic vs probabilistic tasks
You cannot compute cost-per-success until you define success, and success is not a single definition. Split your agent's tasks into two classes and give each an explicit success contract.
Deterministic tasks
Deterministic tasks have ground truth that a machine can check. JSON extraction against a schema, API orchestration, ticket routing to a fixed taxonomy, SQL generation against a known database, code generation with a test suite — each has a verifiable outcome. Success is binary: the output passes the validator, or it does not. The verifier is cheap and fast, so cost-per-success is straightforward to compute. The only nuance is that you must decide before the run whether a retry that eventually passes still counts as a success — we recommend yes, but the retry cost belongs in the numerator.
Probabilistic tasks
Probabilistic tasks require judgment. Summarization, triage prioritization, root-cause hypotheses, draft generation — there is no single correct answer. Success must be defined by an explicit rubric and validated by a human judge or an LLM-as-judge. The rubric must distinguish between a completed outcome, an acceptable outcome, and a failure. A summary that a human would rewrite entirely is a failure even if it is well-formed prose. A triage note that misses the critical issue is a failure even if the customer rated the interaction positively. For probabilistic tasks, the verifier's cost belongs in the cost-per-success calculation — and if every cheap-model output requires an expensive judge to validate, the cheap model's published per-token price is irrelevant.
The deterministic side is where founders should cut first, because deterministic workflows are where you can ruthlessly control spend with validation and guardrails. See the founder's cost-control playbook for deterministic AI.
Instrumentation checklist: what to log on every run
Most teams do not have a cost-per-success number because they never logged the ingredients. You cannot compute what you never observed. Every agent run needs a structured log that captures, at minimum:
- Model identity: model ID, provider, and version. Models change weekly, and version drift changes success rates.
- Prompt and template version: a hash of the prompt template, so a prompt change shows up as a cost signal rather than a mystery.
- Call count per task: an agent run is rarely one call. Log every LLM call with input and output token counts.
- Retry count and retry reason: timeout, validation failure, guardrail rejection, or low confidence.
- Routing decision and confidence: which router selected which path, and the confidence score at decision time.
- Verification method and result: deterministic validator, LLM judge, human review, or no verification — plus the outcome.
- Success label and labeler: who marked the run successful. A validator-labeled success and a self-reported success are different signals.
- Latency percentiles: cost and time correlate with failure in the tail.
- Downstream cost: if a failure propagated, what did the recovery cost in tokens, API calls, and human minutes?
Without this log, you are guessing. With it, cost-per-success becomes a weekly metric you can segment by task class, trend over time, alert on, and optimize. This is the instrumentation foundation for every routing change you will make next.
Model tier comparison: illustrative cost-per-success ranges
The table below shows illustrative relative ranges drawn from Tacavar's production instrumentation — not vendor list prices, which change monthly and are volume-dependent. What matters is the shape: the ordering flips when you switch from cost-per-task to cost-per-success. These ranges are typical for triage and extraction workloads; your exact multiples depend on your success criteria and recovery paths.
| Model tier | Relative cost per task (attempted) | Relative cost per success | Typical failure profile |
|---|---|---|---|
| Small / cheap | 1x | 2.5x–6x | High silent-wrongness rate; fails validation erratically |
| Mid-tier / fast | 3x–4x | 1.5x–2.5x | Occasional rubric misses; needs light verification |
| Frontier / slow-reasoning | 6x–10x | 1x–1.8x | Rare silent failures; overkill for many deterministic tasks |
Read that table carefully. The cheap tier is the most expensive tier by cost-per-success in most real workloads, despite being the cheapest by cost-per-task. The frontier tier is the cheapest per success on hard probabilistic tasks — and the most expensive mistake on easy deterministic ones. The winning move is not "always cheap" or "always frontier." It is routing per task class with cost-per-success as the objective function.
The hub: cost-per-success as the routing objective
The hub that replaces cost-per-task is an operating loop: instrument every run, compute cost-per-success per task class, adjust routing thresholds, and repeat. You are not choosing a single model for your agent. You are continuously choosing a model per task class, and the only defensible driver is success-weighted economics. Cost-per-success is the only LLM routing cost metric that reflects business outcomes rather than token meter readings.
Concretely, the loop looks like this:
- Compute cost-per-success weekly, segmented by task class — never in aggregate.
- Set a target success rate per task class, for example 92% for deterministic extraction and 85% for probabilistic triage.
- Route by task class first, and let historical cost-per-success override the model tier hierarchy.
- When a cheaper model dips below the target success rate, shift its traffic to the next tier — then measure whether cost-per-success actually drops.
To reduce AI agent inference cost, the first lever is always routing. Once you have extracted everything routing can give you, the next lever is architectural: owning inference through vertical integration. We break that down in inference vertical integration as the agent cost lever.
If your agent cost curve is flat or climbing, your routing logic is the first place to audit. You likely do not have a cost problem — you have a measurement problem and a routing problem wearing the same jacket. Start by instrumenting the checklist above, compute cost-per-success per task class, and let the data fire your cheap model.
Ready to run this loop inside your stack? Talk to Tacavar Growth and we will audit your routing against success-weighted economics within a week.