AI Inference Platforms

Executive Summary

This comparison covers 15 platforms, checked September 15, 2026. It retains the previous thirteen and adds Cloudflare Workers AI and Tembo. Workers AI supplies managed inference; Tembo qualifies through its external model gateway as well as its coding-agent platform. The scope already includes gateways, so a provider does not need to operate its own model-serving hardware to qualify.^[1][2]

The first decision is what the team needs to own: an API request, dedicated capacity, custom serving code, or a gateway policy. A shared model API removes infrastructure work. A custom deployment gives more control over weights and serving behavior. A gateway centralizes access across upstream services. Several platforms span these layers, so their product names are less useful than the deployment and billing unit selected.

Disclosure: Ry Walker is Tembo's founder and CEO. Tembo is evaluated here as a gateway and agent-workflow platform, with its current access and documentation limitations stated alongside competitors.

---

Scope and Comparison Matrix

Focused update, September 16: The SpaceXAI context below is newly checked; the 15-member comparison and other providers retain their September 15 review baseline.

Included products provide externally usable model APIs, custom inference deployments, or multi-model gateways. Raw GPU rental without a serving layer, local inference engines, and individual model laboratories are outside this report's focus. This is a decision-oriented selection, not an exhaustive directory. Agent frameworks and agent sandboxes are related application and execution layers.

SpaceXAI / Grok API is relevant upstream context, outside the matrix under the same model-laboratory exclusion. Buying access to Grok 4.6 is a model/API decision; it does not establish a general custom-model hosting or multi-provider gateway contract. Its model page specifies capabilities and rates separately, so API-format compatibility should not substitute for checking the exact endpoint.^[3]

The regional distinction is particularly useful when comparing direct model access with a gateway. Grok's US endpoint currently serves only grok-4.6 and charges a 10% token premium. Its location guarantee covers request handling, inference, moderation, and retained request data, but excludes Files, Collections, server-side tools, and the network path to the service. Those features can therefore have different data paths within one request.^[4] In a gateway evaluation, check which upstream endpoint is selected and how fallbacks affect that choice. A compatible request schema or a model name alone does not prove the required location or retention policy.

---

Deployment Decisions

Shared APIs versus dedicated capacity

Shared APIs are the straightforward starting point when a catalog model and the account's limits fit. They avoid selecting replica counts or serving images, but the application still needs an overload strategy, a compatible schema, and a decision about model version changes.

Dedicated capacity changes the cost equation. DeepInfra documents custom model deployments that can autoscale to zero, while running GPUs are billed regardless of request traffic. Nebius bills replicas in the ready state; a zero-replica endpoint stops its compute bill but gives capacity back to the shared pool. Those are concrete operating choices, not a promise that every dedicated endpoint starts instantly at any load.^[14][25]

Modal now spans both sides. Its managed endpoint product offers shared token-priced models and dedicated compute-priced deployments, including custom weights. For a more general application, Modal also exposes Python functions through HTTP, FastAPI, ASGI, and WSGI serving paths. The correct comparison depends on whether the team wants a model endpoint or programmable infrastructure.^[10][29]

Custom weights, adapters, and training are different capabilities

A single “fine-tuning” checkmark obscures three separate questions: where training happens, which resulting artifact can be deployed, and who operates inference.

• Baseten's deployment guide pins a serving image and model-weight revision, configures health and prediction endpoints, and selects an accelerator. This is a concrete custom-serving workflow rather than merely choosing a hosted model ID.^[5]
• Together and Fireworks publish separate training and inference products. Training charges do not include every subsequent serving cost; compare the resulting endpoint's billing independently.^[7][9]
• Groq's LoRA path is enterprise-only during gradual rollout. Its guide lists Llama 3.1 8B Instant as supported, requires an exact base-model match, and expects externally produced adapter files. That is narrower than arbitrary custom weights.^[13]
• Cerebras dedicated endpoints document custom/fine-tuned weights and management controls beyond the public catalog. SambaNova lists custom weights as an enterprise add-on. Custom hardware does not justify a blanket “no custom models” label.^[16][17]
• Workers AI's LoRA beta accepts adapters for selected models, with file and account limits. Confirm supported base models before training an artifact for that deployment target.^[26]

Generative media has different units

fal's Model APIs can charge per image, megapixel, generated video second, video, or compute unit. Its documentation says queue wait and server-error responses do not incur successful-output charges. Custom serverless app billing is separate.^[20]

Replicate likewise separates public-model execution/output pricing from private-model compute. Its private pricing includes setup and idle periods, with a documented exception for fast-boot fine-tunes. A “pay only when a prediction completes” assumption is therefore unsafe across the whole product.^[11]

For custom fal applications, fal deploy creates a persistent endpoint with private authentication by default; the development fal run workflow defaults to public access. The application owner pays for public endpoint compute, so preview authentication and production authentication deserve an explicit check.^[21]

---

Pricing: Compare the Same Workload

All prices below are published USD rates checked September 15, 2026. They are examples, not negotiated quotes or a measured quality ranking.

One model, six published price points

GPT-OSS-120B provides a useful common model label. The illustrative monthly workload is 100 million uncached input tokens and 20 million output tokens. It excludes discounts, taxes, minimums, retries, and other services, and does not assert identical quantization, latency, or task quality.

Calculation: 100 × input rate + 20 × output rate. This arithmetic is reproducible; the actual cost per successful task still depends on how much inference the application needs.

Different tiers can change the result. Fireworks lists GPT-OSS-120B priority pricing at $0.18 input and $0.72 output per million, while batch processing is advertised at a 50% discount. Cached-input rates differ again. Select the service tier before treating the model's name as a price.^[8]

DeepInfra's current price table includes models such as DeepSeek V4 Flash 0731 at $0.06 input and $0.18 output per million, and Kimi K3 at $2.85/$14.25. These are different models from the example above, so the lower sticker price does not establish a better answer or cheaper completed task.^[31]

Compute-priced deployment examples

A lower GPU-hour rate is not a throughput benchmark. Estimate replica utilization, batch behavior, warm capacity, and operational effort before comparing this table with a token bill.

Free access and gateway fees

---

Gateways, Routing, and Agent Workflows

Vercel AI Gateway can be used from infrastructure outside Vercel. It centralizes model calls and credentials, with multiple API surfaces rather than requiring one application hosting product.^[19] Its routing rules now support team-wide model rewrites and denials. Rewrites preserve supplied options, but provider-specific options are not translated to a different provider automatically. Model substitution therefore needs application testing, even when it requires no application-code change.^[38]

OpenRouter's role is multi-provider access and billing. Its FAQ distinguishes token-price pass-through from platform fees, and its current pricing distinguishes ordinary platform-funded requests from BYOK usage. Compare the effective purchase and routing arrangement, not just the displayed per-token rate.^[23][22]

Tembo is relevant when model access is connected to coding-agent execution and review workflows. Its public Max page advertises an external proxy.tembo.io gateway for CLIs and SDKs, OpenAI/Anthropic-compatible access, and provider failover. The linked Max setup documentation required authenticated access during this review, so those advertised capabilities were not validated through an executed external-client setup.^[39][2]

Tembo's model documentation separates managed models from BYOK-only entries and distinguishes free-plan access from paid access. A model appearing in a price table should not be treated as proof that every account can call it with managed credentials. Its published GLM 5.3 rate, for example, is $1.61 input and $5.06 output per million; standard bring-your-own-key usage has different billing.^[28][37]

For all gateways, trace the failure path: which provider receives the retry, which key pays, whether the model changes, and whether the destination still meets the application's data policy. A fallback can improve availability while changing cost or behavior.

---

Security, Privacy, and Compatibility

A platform-wide compliance badge does not answer whether a particular model, region, account tier, or upstream provider meets the application's requirements. The useful review follows the request: gateway logs, provider processing, caching, application storage, credentials, and any contractual restrictions.

Vercel documents a concrete ZDR routing policy. With zero-data-retention enabled, ordinary BYOK keys are excluded unless the customer marks them compliant under its own agreement. Fallbacks remain within the ZDR routing set, and requests fail if no qualifying provider exists. Provider-side caching still depends on the provider's policy.^[40] Team-wide and per-request controls can have different fees; a no-markup token policy does not imply every governance feature is free.^[36]

OpenRouter says it does not log prompts and completions by default, while retaining request metadata; upstream providers have their own policies. Workers AI says it does not use customer content to train or improve models without consent. Neither statement alone answers every retention, geography, or application-storage question.^[23][41]

API compatibility also has layers. A successful chat-completion request does not verify streaming event types, tool-call arguments, reasoning fields, structured-output schemas, embeddings, or the Responses protocol. Cloudflare's current guide includes chat, embedding, and a Responses example, but each application's interactions still need testing.^[42]

What developers report

On April 12, Hacker News user mrnoname, author of a Codex–Workers AI proxy, reported protocol friction while trying to use existing Cloudflare startup credits. They described a day of successful Kimi use after adding a translation layer. The account is useful evidence of integration work, not proof of a current platform-wide defect or representative reliability.^[43] The current official Responses example further limits what can be inferred from that older report.^[42]

No independent comparative benchmark or hands-on load test was conducted for this refresh. Vendor performance figures were not converted into a universal speed ranking.

---

Company and Product Changes That Matter

Replicate announced that it had joined Cloudflare on December 1, 2025. The retained Replicate API and Workers AI are separately documented products; common ownership does not establish shared catalogs or interchangeable billing.^[44][11][1]

Groq announced on August 12, 2026 that it had become an NVIDIA Cloud Partner and planned to bring NVIDIA GPU capacity online. Its present LPU model service and a future GPU expansion should not be collapsed into a claim that every workload already runs on both.^[45]

SambaNova's July 8 announcement reported the first close of a $1 billion Series F at an $11 billion valuation. That is a material update to the earlier article's distressed-company framing. It is an attributed financing announcement, not proof of model quality, profitability, or guaranteed service continuity.^[46]

Financial headlines can inform counterparty review, but they do not replace a supported-model list, capacity terms, or a tested migration path. This comparison omits unverified revenue run rates and fundraising rumors.

---

How to Choose

These are editorial starting points based on the documented mechanisms above, not rankings from an unperformed benchmark. A useful evaluation has four stages:

1. Pin the workload. Select representative prompts, model revisions, tool schemas, expected output sizes, and acceptance criteria. Include difficult failures, not just a successful demo.
2. Measure the entire interaction. Record task success, time to first token, total completion time, p95 latency, and behavior at expected concurrency. For agents, include retries and follow-up model calls.
3. Reconstruct the bill. Separate uncached input, cache reads/writes, output, gateway fees, idle capacity, and application compute. Run both a quiet period and a burst.
4. Exercise the failure path. Check rate limits, unavailable capacity, cancellation, credential failure, fallbacks, and recovery. Confirm that the replacement path preserves required model behavior and data policy.

The resulting choice should be explainable in operational terms: which deployment owns the workload, why its constraints fit, and what evidence supports its cost and reliability. Headline token prices and hardware labels are inputs to that decision, not the decision itself.

---

Research by Ry Walker • methodology