AI Inference Platforms

Executive Summary

AI inference platforms have become critical infrastructure as organizations move from prototype to production with large language models and generative AI. These platforms solve the problem of "how do I serve AI models at scale?" — handling GPU orchestration, optimization, autoscaling, and cost management. The past two quarters reshaped the category: an IPO, a ~$20B technology license, an acquisition, and revenue run rates that tripled in months.

13 Platforms Compared: Baseten, Together AI, Fireworks AI, Modal, Replicate, Groq, DeepInfra, Cerebras, SambaNova, Vercel AI Gateway, Fal, OpenRouter, Nebius

Key Findings:

• Cerebras went public on May 14, 2026 (Nasdaq: CBRS) in the biggest tech IPO of the year — $5.5B raised at $185/share, closing day one at $311 (~$66B market cap) — with an OpenAI supply relationship behind it
• Groq transformed: Nvidia licensed its LPU technology for ~$20B in December 2025 and hired founder Jonathan Ross; Groq remains independent and is pivoting from chipmaker to inference cloud
• Replicate was acquired by Cloudflare (announced November 2025, closed early 2026) — the brand and API continue, with its 50,000+ model catalog feeding Workers AI
• ARR exploded across GPU platforms: Baseten ~$600M (reportedly raising at $11B), Fireworks AI ~$800M ($15B talks), Together AI ~$1B ($7.5B talks), Modal ~$300M ($4.65B Series C)
• DeepInfra shed its underdog status with a $107M Series B (Nvidia, Samsung Next) and ~5 trillion tokens/week; SambaNova survived a collapsed Intel deal with a $350M down-round Series E at ~$2.2B
• New entrants broaden the category: Fal owns generative-media inference ($4.5B, ~$400M reported ARR), OpenRouter is the standalone gateway leader ($1.3B, 25T tokens/week), and Nebius layers Token Factory on public-company neocloud economics ($1.9B AI ARR)
• The market now has three clear layers: custom silicon and GPU platforms serving tokens, API-first platforms simplifying access, and gateways/routers (OpenRouter, Vercel AI Gateway) consolidating on top

---

Comparison Matrix

Pricing Comparison

Hardware Comparison

---

Product Profiles

Baseten

Enterprise AI inference platform at ~$600M ARR, reportedly raising $1B at an $11B valuation.^[1] Deploys custom and open-source models with the Truss open-source framework.

• $585M raised ($300M Series E Jan 2026 at $5B — CapitalG, IVP, NVIDIA); ARR ~3x in Q1 2026 to ~$600M
• GPU selection from T4 to B200, SOC 2 Type II and HIPAA compliant
• Per-minute billing, forward deployed engineers; expanding into training (Loops) and model monetization (Frontier Gateway)
• ⚠️ More complex setup than pure API platforms; per-minute billing carries a premium at sustained utilization

Best for: Enterprise teams deploying custom models at scale with compliance requirements.

---

Together AI

"The AI Native Cloud" at roughly $1B annualized revenue, reportedly raising $1B at a $7.5B pre-money valuation.^[2] Research-driven: FlashAttention-4, RedPajama, ATLAS, Mamba-3.

• ~$1B annualized revenue (early 2026), up 3x+ from mid-2025; customers include Cursor, Decagon, and ElevenLabs
• On-demand NVIDIA B200 clusters, GB200/GB300 NVL72 for scale; ISO 27001 certified (June 2026)
• Full stack: inference, fine-tuning, pre-training, GPU clusters, and Together Code Sandbox (CodeSandbox acquisition)
• ⚠️ Premium pricing; HIPAA not advertised

Best for: Teams wanting the full ML lifecycle on research-grade infrastructure with the latest hardware.

---

Fireworks AI

Speed-optimized inference from ex-PyTorch engineers, at an estimated ~$800M annualized revenue and reportedly in talks at a $15B valuation.^[3] 10T+ tokens/day.

• $327M+ raised ($250M Series C Oct 2025 at $4B); $15B funding talks reported May 2026
• 10K+ customers including Cursor, Notion, Uber, Vercel, and Quora
• Serverless 2.0 and a Microsoft Foundry (Azure) integration shipped in 2026; fine-tuning up to 1T+ parameter models
• ⚠️ No ongoing free tier ($1 signup credit); weak on image/video relative to LLMs

Best for: Production workloads requiring speed, reliability, and model lifecycle management.

---

Modal

Serverless Python infrastructure with elastic GPU scaling — $355M Series C at $4.65B (May 2026) with ~$300M annualized revenue.^[4] Full profile: Modal.

• Define infrastructure in Python code (no YAML/Docker); sub-second cold starts
• GPU access without quotas (T4 through B200); revenue up ~5x to ~$300M annualized
• Sandboxes at scale: 1B+ run, only GPU-accelerated OpenAI Agents SDK provider
• ⚠️ Python-centric; pricing premium over raw GPU providers

Best for: Python-native ML teams, GPU workloads, training and inference.

---

Replicate

Developer-friendly platform to run open-source models via API — now part of Cloudflare (announced November 2025, closed early 2026).^[5][6] One line of code to run any model.

• 50,000+ community and official models; API unchanged post-acquisition
• Pay per use — hardware by the second, official models per token/image/video-second
• Catalog being integrated into Cloudflare Workers AI and the broader developer platform
• ⚠️ Less infrastructure control; fal contests its generative-media niche; long-term roadmap independence unproven

Best for: Developers wanting quick model access without infrastructure overhead, especially on Cloudflare's platform.

---

Groq

Custom LPU hardware delivering the fastest LLM inference speeds — transformed in December 2025 when Nvidia licensed its LPU technology for ~$20B and hired founder Jonathan Ross.^[7][8]

• Remains independent; GroqCloud continues uninterrupted, pivoting from chipmaker to inference cloud
• 3M+ developers and teams; deterministic latency, no batching delays
• Free tier, OpenAI-compatible API, LoRA adapter inference; $750M raised at $6.9B (Sep 2025) plus Nvidia licensing cash
• ⚠️ Founder and key technical leadership departed to Nvidia; limited to supported models, no custom model deployment

Best for: Applications where latency is the #1 priority — with eyes open on post-Ross execution risk.

---

DeepInfra

Cost-efficient inference APIs for 190+ open-source models, now backed by a $107M Series B (May 2026) with Nvidia, Samsung Next, and Supermicro participating.^[9]

• ~5 trillion tokens/week — 25x growth since April 2025 — from eight US data centers on its own hardware
• Among the cheapest per-token rates; OpenAI-compatible API
• Now offers custom model deployment, dedicated B300 clusters ($1.98/GPU-hr), SOC 2 and ISO 27001
• ⚠️ Community reports FP4 quantization where rivals serve FP8 — cheapest tokens may not be equivalent tokens

Best for: Cost-conscious teams running open-source models via API, including high-volume agent workloads.

---

Cerebras

Wafer-Scale Engine — the largest chip ever built — now public on Nasdaq (CBRS) after the biggest tech IPO of 2026.^[10][11]

• IPO'd May 14, 2026: $5.5B raised at $185/share, opened +108%, closed day one at $311 (~$66B market cap)
• $510M 2025 revenue (+76% YoY) with a swing to $237.8M net income; OpenAI supply relationship, AWS availability
• 2,000+ tokens/sec on supported models; ~1,000 tok/s even on trillion-parameter models
• ⚠️ Limited model ecosystem, no fine-tuning API; customer concentration and circular OpenAI economics flagged

Best for: Teams where inference speed is the product — real-time agents, voice, fast coding loops.

---

SambaNova

Custom RDU chip with enterprise-focused AI platform, refocused on inference after a turbulent stretch.^[12] On-premise, cloud, and sovereign deployments.

• April 2025: laid off 77 (~15%) and pivoted from training to inference; Intel's ~$1.6B acquisition offer collapsed
• $350M Series E (Feb 2026) led by Vista with Intel as ~9% strategic investor — ~$2.2B valuation, down from $5.1B in 2021
• Fifth-generation SN50 RDU for agentic inference; four sovereign-AI data center partners live across Australia, Europe, and the UK
• ⚠️ Prove-it mode: thinner ecosystem, smaller developer community, and company risk after the down round

Best for: Enterprise, government, and sovereign deployments requiring on-premise AI infrastructure.

---

Vercel AI Gateway

Unified gateway over 100+ models — text, image, video, embeddings — at provider list prices with 0% token markup.^[13] Moved into this comparison from our AI Agent Sandboxes report (June 2026): it routes inference, it doesn't execute code.

• 200K+ teams, tens of trillions of tokens routed (Vercel Production Index, June 2026)
• Automatic retries/fallbacks across providers, BYOK support, built-in observability
• OpenAI- and Anthropic-compatible endpoints; AI SDK v6 native
• ⚠️ No conditional routing/A-B testing; 5-minute Pro timeouts; ZDR fee $0.10/1k requests; community-reported latency spikes

Best for: Teams (especially on Vercel) wanting one key over many providers without managing routing infrastructure — an OpenRouter alternative.

---

Fal

Serverless inference cloud for generative media — 1,000+ curated image, video, audio, and 3D models behind one API on a proprietary inference engine.^[14]

• $140M Series D led by Sequoia (Dec 2025) at $4.5B — its third raise of 2025; reported talks at ~$8B with ~$400M reported annualized revenue (unconfirmed)
• 2.5M developers; customers include Adobe, Canva, and Amazon MGM Studios; AWS named preferred cloud (May 2026)
• Transparent per-output pricing (e.g., ~$0.03/image, $0.05–0.07/sec of video) plus serverless GPU (H100, RTX PRO 6000, B300)
• ⚠️ Closed platform, single-cloud concentration; recurring complaints about CDN/delivery speed; headline revenue is reported, not confirmed

Best for: Teams shipping image/video/audio generation features who want frontier media models behind one usage-billed API.

---

OpenRouter

The standalone gateway leader — 400+ models from 60+ providers behind one OpenAI-compatible endpoint, routing on price, latency, and uptime.^[15]

• $113M Series B led by CapitalG (May 2026) at ~$1.3B post — with NVIDIA, ServiceNow, MongoDB, Snowflake, and Databricks venture arms
• 25 trillion tokens/week (5x in six months) across 8M+ developers; ~$50M annualized revenue on a ~5% take
• No token markup: 5.5% fee on credit purchases, 5% BYOK fee after 1M free requests/month; per-key spend caps
• ⚠️ Owns no GPUs — routed-provider quality (quantization) varies; fees compound at single-model production scale

Best for: Multi-model apps and agent builders who want every model on release day with consolidated billing.

---

Nebius

Public neocloud (NASDAQ: NBIS) whose Token Factory serves 60+ open models per-token on infrastructure it owns end to end.^[16]

• $1.9B AI ARR (Q1 2026, revenue +684% YoY); $2B NVIDIA strategic investment; $27B five-year Meta contract
• Token Factory is management's fastest-growing segment, reinforced by the $643M Eigen AI acquisition plus Clarifai and Tavily
• Fast/Base serving tiers, dedicated endpoints with 99.9% SLA; SOC 2 Type II, HIPAA, ISO 27001, EU data residency
• ⚠️ Inference is the side bet next to hyperscale leases; open models only; Yandex heritage surfaces in some procurement reviews

Best for: Enterprises serving open-weight models at volume that want an audited public-company counterparty and EU residency.

---

Decision Guide

By Use Case

By User Profile

Enterprise ML team with custom models: → Baseten (Truss framework, compliance, GPU breadth) or Fireworks AI (speed, lifecycle management)

Startup shipping fast: → Replicate (simplicity) or DeepInfra (cost) or Groq (speed + free tier)

Generative media product: → Fal (curated gallery, per-output pricing) or Replicate (community model breadth)

Research team: → Together AI (latest hardware, research culture) or Cerebras (raw performance)

Latency-sensitive application: → Groq (custom LPU, deterministic) or Cerebras (WSE, 2,000+ tok/s)

Budget-conscious team: → DeepInfra (cheapest per-token) or Groq (generous free tier)

Multi-model app or agent harness: → OpenRouter (every model, one key) or Vercel AI Gateway (if already on Vercel)

European enterprise / public-company counterparty required: → Nebius (EU residency, audited financials, NVIDIA-backed)

---

Market Outlook

Near-Term (2026)

• Consolidation arrived early: Nvidia licensed Groq's LPU (~$20B), Cloudflare absorbed Replicate, Cerebras went public, and Nebius bought Eigen AI, Clarifai, and Tavily
• Mega-rounds (Baseten $11B, Fireworks $15B, Together $7.5B talks) price in continued ARR compounding
• Gateways (OpenRouter, Vercel AI Gateway) becoming the default on-ramp for multi-model and agent workloads

Medium-Term (2027)

• Remaining independents face acquire-or-IPO pressure as hyperscalers and neoclouds compress margins
• Custom silicon platforms expanding model support to match GPU versatility
• Gateway take rates tested as providers discount direct and platforms bundle equivalent routing free

Long-Term (2028+)

• 2-3 dominant platforms per segment (custom silicon, GPU cloud, developer API, gateway, generative media)
• On-premise and sovereign inference growing as models commoditize
• Integration with agent orchestration platforms (Tembo, etc.)

---

Bottom Line

13 platforms serve the AI inference market with distinct strategies:

The market has stratified into layers. Custom silicon proved its worth — Nvidia paid ~$20B to license Groq's LPU, Cerebras IPO'd at a ~$66B day-one market cap, and even SambaNova found a Vista/Intel lifeline. GPU platforms (Baseten, Together AI, Fireworks, Modal) turned ARR explosions into mega-round valuations. API-first platforms consolidated — Replicate into Cloudflare, DeepInfra into a funded cost leader — while Fal carved generative media into its own category. And gateways (OpenRouter, Vercel AI Gateway) now sit on top of all of it, routing trillions of tokens a week. The winners will be determined by which layer captures margin as inference demand keeps compounding.

---

Research by Ry Walker Research

Disclosure: Author is CEO of Tembo, which may integrate with inference platforms for agent execution.