AI API Cost per Useful Result: A Reproducible…

The useful denominator is an accepted result

A low price per million input tokens does not guarantee a low application cost. One model may need longer prompts, more retries, more output, more retrieval, or more human correction. Compare candidates on the same representative workload and report cost per accepted result alongside quality, latency, and risk.

Record the provider, exact model identifier, endpoint, region, service tier, price-page retrieval time, tokenizer or API token counts, caching state, tool use, and tax/currency assumptions. A comparison without these fields cannot be reproduced.

Build a billing model from the current price sheet

For each request i, a general API-cost model is:

request_cost_i =
    uncached_input_tokens_i / 1_000_000 * input_rate
  + cached_input_tokens_i   / 1_000_000 * cached_input_rate
  + output_tokens_i         / 1_000_000 * output_rate
  + reasoning_tokens_i      / 1_000_000 * reasoning_rate
  + tool_calls_i            * tool_rate
  + storage_i
  + other_provider_charges_i

Not every provider exposes or charges each category in the same way. Use the provider’s live documentation and invoice semantics. Do not assume that a cached token, reasoning token, search query, image, audio second, or batch request is priced like ordinary text.

Calculate cost per accepted outcome

effective_cost_per_accepted_result =
  (API charges
   + retrieval and data charges
   + retries and failed-request charges
   + orchestration and observability
   + human review and correction
   + allocated engineering and security cost)
  / accepted_results

Define “accepted” before testing. It might mean a response that passes a factuality and citation rubric, a classified case confirmed by a reviewer, or code that passes a test suite. Report severe-error rate separately; inexpensive unsafe failures should not be averaged away.

Use a controlled comparison

Freeze a representative test set. Include common, long-context, multilingual, adversarial, ambiguous, and failure cases.
Define the system configuration. Record prompts, retrieval corpus, tools, temperature, output limits, reasoning effort, and retry policy.
Set acceptance criteria. Use blinded human review or executable checks where practical. Resolve reviewer disagreements.
Run repeated trials. Capture request-level usage, latency, errors, cache hits, tool calls, and outputs.
Measure quality and uncertainty. Report denominators, confidence intervals where appropriate, and error categories—not only a mean score.
Price from actual usage. Join API usage to the dated price sheet and reconcile against the bill.
Stress the system. Test rate limits, provider errors, timeouts, fallbacks, and budget guards.

Do not compare tokens as if they were standardized work units

Tokenization varies across model families and languages. Output length, hidden reasoning accounting, context caching, retrieval behavior, and tool use can also differ. Compare the same task at the application boundary. Preserve both provider-reported usage and workload-level measures such as documents processed, cases resolved, or tests passed.

Include latency and capacity

Measure time to first output, generation rate, end-to-end latency, concurrency, throttling, timeout rate, and recovery behavior at the intended service tier. Peak hardware FLOPS or TOPS cannot substitute for these measurements. They describe theoretical operations under specified numerical formats and conditions, not application throughput.

Model caching and batching explicitly

Caching can reduce billed repeated-prefix work, but eligibility, retention, storage charges, minimum sizes, privacy implications, and invalidation behavior vary. Batch services may reduce price in exchange for a longer completion window. Test the real cache-hit rate and scheduling constraints instead of applying the maximum advertised discount to all traffic.

Compare self-hosting on the same basis

Open weights do not make inference free. For self-hosting, calculate:

self_hosted_cost_per_accepted_result =
  (hardware amortization
   + power and cooling
   + networking and storage
   + hosting or colocation
   + engineering and on-call labor
   + evaluation, security, and observability
   + downtime and spare capacity)
  / accepted_results

First verify memory feasibility. A model with P parameters stored at b bits per parameter needs an idealized minimum of P × b / 8 bytes for weights alone. Add runtime buffers, activations, key-value cache, framework overhead, and fragmentation. Quantization format, context length, batch size, parallelism, interconnect, and software determine whether the workload fits and how fast it runs.

Multiple computers do not automatically form one shared-memory accelerator. Any distributed benchmark must identify the sharding method, network, software, model artifact, quantization, context, batch, decoding parameters, warm-up, measurement window, wall power, and output-validation method.

Track quality, cost, and risk together

Dimension	Minimum evidence
Quality	Accepted-result rate, severe errors, rubric or executable tests, subgroup/scenario results
Cost	Request-level usage joined to dated rates, review labor, infrastructure, and retries
Latency	p50/p95 end-to-end latency, time to first output, timeout and throttling rates
Reliability	Repeated-run variance, provider errors, fallback and recovery tests
Data controls	Retention, training use, regions, logging, subprocessors, deletion, and contract scope
Change risk	Version pinning, deprecation notice, regression tests, migration and exit plan

A small auditable calculator

from dataclasses import dataclass

@dataclass(frozen=True)
class Usage:
    uncached_input: int
    cached_input: int
    output: int
    tool_cost: float = 0.0

@dataclass(frozen=True)
class Rates:
    input_per_million: float
    cached_input_per_million: float
    output_per_million: float

def api_cost(usage: Usage, rates: Rates) -> float:
    if min(usage.uncached_input, usage.cached_input, usage.output) < 0:
        raise ValueError("Token counts must be non-negative")
    return (
        usage.uncached_input / 1_000_000 * rates.input_per_million
        + usage.cached_input / 1_000_000 * rates.cached_input_per_million
        + usage.output / 1_000_000 * rates.output_per_million
        + usage.tool_cost
    )

def cost_per_accepted(total_cost: float, accepted: int) -> float:
    if accepted <= 0:
        raise ValueError("accepted must be positive")
    return total_cost / accepted

Store rate-sheet URLs and retrieval timestamps beside the inputs. Extend the calculator only for billing categories verified in the provider documentation.

There is no permanent price winner

A defensible choice is conditional: candidate A was less expensive for workload W, at quality threshold Q, under configuration C, using prices retrieved on date D. Re-run the evaluation when a model, prompt, traffic mix, service tier, price, or requirement changes.