Model Inference Pricing: Examples & Companies

What is it

Model inference pricing is pricing for AI model inference services — APIs and platforms that run trained models on user inputs, typically billed per token, per request, or per GPU-hour. “Inference” is the forward pass through an already-trained model: you send an input, the model produces an output — text, an image, a transcript, an embedding vector — and you pay for the compute that output consumed.

Model inference is the largest single use-case category in the UsagePricing corpus: 114 companies tag it as a primary use case. They span the full stack, from frontier API labs like OpenAI and Anthropic, to discounted open-model platforms like DeepInfra and Together AI, to raw GPU clouds like RunPod and Lambda, to modality specialists in voice, image, and embeddings.

The defining feature of this category is its price spread — and its speed of deflation. Discounted open-weight models start near DeepInfra’s Llama-3.1-8B at $0.02/1M input tokens; frontier models like Claude Opus 4.8 sit at $5/$25 and OpenAI’s GPT-5.5 at $5/$30 per 1M input/output. Between those poles are dozens of hosted-model platforms competing on the same open weights, which pushes the floor lower with every model generation. Because the underlying rates change quarterly, inference is the one category where a cost model built six months ago is almost always wrong.

The other defining feature is transparency. Inference is the most publicly-priced segment in the corpus — nearly every vendor here ships a public per-token or per-GPU-hour rate card, because their buyers are engineers who compare rates before they ever talk to sales. Gated pricing in inference almost always signals dedicated-capacity or enterprise-commitment tiers, not the standard self-serve API.

How it works

Inference pricing uses three primary billing units, each mapping to a different underlying cost driver.

The token model carries a structural asymmetry: output tokens cost 4–10× more than input tokens on the same model, because generation is far more compute-intensive than encoding the prompt. On Cohere Command A that ratio is exactly 4× ($2.50 in / $10 out); on Claude Opus 4.8 it is 5× ($5 / $25). This is why a chat workload heavy on long completions costs far more per call than a classification workload that emits a single token.

A worked example. Suppose a RAG assistant processes 2,000 input tokens (a retrieved context block) and emits 500 output tokens per query, running 1M queries a month. On Claude Sonnet 4.6 at $3/1M in and $15/1M out, that is (2,000 × 1M ÷ 1M × $3) + (500 × 1M ÷ 1M × $15) = $6,000 input + $7,500 output = $13,500/month. Route the same workload to DeepInfra’s DeepSeek-V3.1 at $0.21/$0.79 and it drops to $420 + $395 = $815/month — a 16× swing driven purely by model choice, before any batch or cache discount applies. Model those tradeoffs on a DeepSeek pricing calculator or an OpenAI pricing calculator against your real prompt shape.

Two structural discounts sit on top of the base rate on most platforms:

• Cached-input pricing (50–80% off): when a prompt reuses a processed prefix — a fixed system prompt, a repeated document, a stable RAG context — the reused tokens bill at a cache-hit rate. Fireworks AI bills cached input at 50% of list; Baseten quotes 50–80% off; DeepSeek’s V4-Flash cache-hit input drops to $0.0028/1M, one-tenth of its miss rate.
• Batch processing (~50% off): asynchronous, latency-tolerant jobs run at roughly half the synchronous rate at Fireworks and Together, and at 33% off on Voyage AI’s embedding batch API. Fireworks stacks the two — cached input at 50%, then another 50% off for batch.

Companies using this

The 114 companies below all tag model inference as a primary use case. The list runs from frontier LLM labs and multi-model routing platforms through GPU clouds, vector databases, and voice, image, and embedding specialists — use the table’s filters to narrow by pricing model, billing unit, or free-tier availability.

Patterns observed

Output-premium token pricing is nearly universal. Every LLM-API vendor in this set prices input and output tokens separately, with output 4–10× higher. Cohere Command A is 4× ($2.50/$10), Claude Sonnet 4.6 and Opus 4.8 are 5×, and OpenAI’s GPT-5.5 is 6× ($5/$30). The asymmetry is the single most important thing to model, because it means your bill is dominated by how much the model writes, not how much you send.

Hosted platforms cluster tightly around the open-weight floor, and it keeps dropping. Together AI, Fireworks AI, DeepInfra, Groq, SambaNova, and Cerebras all serve overlapping open models (Llama, Qwen, GPT-OSS, DeepSeek, Kimi) at nearly indistinguishable rates — GPT-OSS-120B runs $0.15/$0.60 at Groq and Together, $0.22/$0.59 at SambaNova, and $0.35/$0.75 at Cerebras. When the model is a commodity, competition moves to throughput (Groq’s LPU silicon, Cerebras’s wafer-scale) and to structural discounts rather than the headline rate.

GPU clouds meter time, at ever-finer granularity. RunPod (H100 $2.89/hr), Lambda (H100 SXM $3.99/hr), Replicate (H100 $0.001525/sec), and Baseten (H100 $0.10833/min, ~$6.50/hr) all bill raw GPU time rather than tokens — and increasingly quote per-second or per-minute, which makes scale-to-zero economics visible. Baseten explicitly markets per-minute pricing so a 4-minute warm burst costs a legible $0.43, granularity that Bedrock and Vertex AI hide behind hourly abstractions.

Free tiers are the onboarding default, and they’re mostly credit grants. Free access is more common in inference than in any other corpus category. Most vendors — DeepSeek, Fireworks AI, Groq, Cerebras — offer either a free web tier or a starter credit grant. Voyage AI gives the first 200M embedding tokens free; SambaNova hands you $5 of credits that expire in 30 days — a deliberate nudge from trial to pay-as-you-go. The pattern reflects a self-serve, developer-led sales motion where the goal is to get an API key generating tokens as fast as possible.

Pass-through and marketplace models are emerging. OpenRouter routes to 400+ models at each provider’s exact per-token rate with no markup, monetizing instead through a 5.5% credit-purchase fee — and by May 2026 processed ~100 trillion tokens/month. Hugging Face Inference Providers does the same, charging the underlying provider’s rate with no HF markup. This layer competes on breadth and routing, not price, since price is by definition identical to going direct.

Counterexamples & variants

Per-request and per-pixel billing where tokens don’t fit. Not everything reduces to a token. Fal bills generative media per output — Seedream V4 at $0.03/image, Veo 3 video at $0.40/second — and per-megapixel for models like Qwen ($0.02/MP). Cohere’s Rerank API bills per query ($2/1,000 queries), not per token, because reranking cost tracks the number of documents scored. Voyage AI’s multimodal models bill on two meters simultaneously — $0.12/1M text tokens and $0.60/1B pixels — with each image clamped between $0.00003 and $0.0012. Any cost model that assumes “inference = tokens” will misprice these entirely.

GPU-time vendors invert the free-tier norm. The near-universal free-tier pattern breaks on dedicated compute. Lambda has no free GPU tier at all — you pay per-minute for any instance you launch, and persistent storage keeps billing (~$0.20/GiB/month) even on a stopped instance, so orphaned volumes are a real hidden cost. Hugging Face Inference Endpoints require an active subscription and a card on file with no free dedicated GPU. When the marginal cost of serving is a reserved GPU rather than a shared token pool, “free” stops being economical.

Hosted middlemen that price near first-party, and those that don’t. The usual expectation is that a hosted-inference reseller adds a 30–50% markup. Baseten is a notable counterexample — its Model APIs price DeepSeek V3.1 at $0.50/$1.50, within 5–10% of DeepSeek’s own first-party rates. Meanwhile OpenRouter and Hugging Face formalize zero-markup pass-through as the product itself. The variant to watch is the opposite: dedicated-deployment tiers where the token rate disappears entirely and pricing becomes GPU-utilization negotiation, as on Baseten’s per-minute dedicated deployments or Lambda’s committed 1-Click Clusters.

GPU list prices can rise, against the deflation narrative. Token prices fall each generation, but raw GPU rates don’t always follow. Lambda raised its on-demand H100 SXM rate from $2.99 to $3.99/GPU/hr between 2025 and 2026 as hyperscaler and superintelligence demand outstripped capacity — a reminder that the “everything gets cheaper” assumption applies to model efficiency, not to the underlying silicon supply.

What this means for buyers vs vendors

For buyers

Never compare list prices at the model level — compare cost per real workload. The output-token premium means a chatty, long-completion workload and a terse classification workload with the same “price” can differ 10× in practice, so plug your actual input/output token shape into a DeepSeek pricing calculator or OpenAI pricing calculator before committing. Then ask three procurement questions of any inference vendor: (1) does cached-input pricing apply to my prompt structure, and at what discount — a stable system prompt or large RAG context can cut input cost 50–80% at Fireworks AI or Baseten; (2) can any portion of my workload run async for the ~50% batch discount; and (3) if I outgrow the shared API, what does a dedicated deployment cost in GPU-hours, and is that cheaper than tokens at my volume? Re-baseline the whole model at least twice a year — DeepSeek and open-weight platforms routinely reset the floor, and a cheaper new model may already meet your quality bar. Grounding these tradeoffs in the fundamentals of usage-based pricing models and in how to choose the right usage metric makes the comparison far cleaner.

For vendors

Transparency is the price of entry here — engineers will not adopt an API whose rate card they can’t read, so publish per-token or per-GPU-hour rates and let the self-serve motion do the qualifying. Differentiate on structure, not headline: because open-weight models are a commodity served at near-identical rates by Together AI, Groq, and SambaNova alike, your leverage is throughput, cached-input and batch discounts, and billing granularity (Baseten’s per-minute meter is a genuine differentiator). Free credits or a free web tier are effectively table stakes for onboarding — but structure them as an expiring grant, as SambaNova does with its 30-day $5 credit, to convert trial into pay-as-you-go rather than subsidize idle experimentation. If you resell hosted models, decide deliberately whether you compete on markup or on pass-through: OpenRouter’s zero-markup model with a 5.5% credit fee is a distinct strategy from a marked-up rebrand, and buyers can tell the difference. Whichever you choose, prepaid credits are the dominant funding mechanism in this category — the prepaid credits model guide covers how to structure grants, top-ups, and expiry.

Explore this theme in the knowledge graph →