DeepInfra

Company Overview

DeepInfra is a cost-efficient inference platform focused on making open-source AI models accessible via simple APIs.^[1] The value proposition is straightforward: run popular open-source models at the lowest possible per-token cost with minimal setup.

The "scrappy budget player" framing no longer fits. In May 2026 DeepInfra closed a $107M Series B co-led by 500 Global and Georges Harik, with participation from Nvidia, Samsung Next, Supermicro, A.Capital Ventures, Crescent Cove, Felicis, Peak6, and Upper90.^[2] That follows an $18M Series A in April 2025; the company says token volume has grown 25x since then, to nearly five trillion tokens per week.^[3] It runs eight US data centers on its own hardware and claims up to 20x inference cost efficiency using Nvidia's Dynamo platform on Blackwell and Vera Rubin GPUs.^[4]

What It Does

• Inference APIs — OpenAI-compatible endpoints for LLMs, vision/OCR, embeddings, reranking, image, video, and speech models^[5]
• Wide model selection — 190+ open-source models including DeepSeek, Qwen, Llama, Gemma, and Nemotron^[4]
• Custom model deployment — Dedicated instances on A100/H100/H200/B200/B300 with autoscaling and private endpoints^[5]
• DeepCluster — Dedicated Nvidia B300 GPU clusters at $1.98/GPU-hour, versus roughly $6.50 on public clouds^[1]
• Serverless — No infrastructure management, pay only for usage

How It Works

Call OpenAI-compatible API endpoints with your DeepInfra API key — point your existing OpenAI SDK at https://api.deepinfra.com/v1/openai and your code works without changes.^[5] Models run on DeepInfra's own GPU fleet in secure US data centers; it handles scaling, optimization, and hardware management.

Pricing

• Per-token pricing — transparent, published rates per model. As of June 2026: DeepSeek-V4-Flash at $0.10/M input and $0.20/M output, Gemma-4-26B at $0.07/$0.34, Nemotron-3-Ultra-550B at $0.50/$2.50^[1]
• No minimums or long-term contracts — pay only for what you use
• Self-serve — published rates, no enterprise negotiation required
• Dedicated GPU pricing — $1.98/GPU-hour for B300 clusters via DeepCluster^[1]

DeepInfra consistently offers some of the lowest per-token rates in the market for open-source models.

Adoption

As of June 2026, DeepInfra processes nearly five trillion tokens per week — 25x growth since its April 2025 Series A — and reports that over 30% of platform token volume is driven by autonomous agents.^[4] CEO Nikola Borisov frames the bet plainly: "Inference is no longer a thin layer – it's the system constraint that will define the majority of workloads."^[4]

Strengths

• Cost leader — Among cheapest per-token pricing for open-source models
• Simplicity — Minimal setup, OpenAI-compatible, self-serve
• Owns its stack — Eight US data centers and its own hardware, so per-token price carries no aggregator margin^[4]
• Compliance arrived — SOC 2 and ISO 27001 certified, with a zero-data-retention policy^[1]
• Well capitalized — $125M raised across Series A and B, with strategic backing from Nvidia, Samsung Next, and Supermicro^[2]
• Wide model selection — Quick to add new popular models; 190+ live

Weaknesses / Risks

• No custom silicon — Can't compete with Groq/Cerebras on raw speed
• Quantization opacity — Community observers note DeepInfra serving models at FP4 where other providers run FP8; cheapest tokens may not be equivalent tokens^[6]
• Thin moat — Commodity GPU inference remains a race to the bottom, even with owned data centers
• Limited fine-tuning — Still basic compared to Together AI or Fireworks
• Race-to-bottom margins — Competing on price against subsidized Chinese providers pressures unit economics

What Developers Say

Hacker News commenters treat DeepInfra as a pricing reference point for what open-model inference "really" costs. In a June 2026 thread on AI lab margins, wyrdcurt wrote: "even a US provider like DeepInfra can serve DeepSeek 4 Pro at $1.30/M in and $2.60/M out. Unless American labs are doing something wildly inefficient, it feels safe to assume Anthropic has some profit margin on inference at API prices."^[7] Another commenter, wongarsu, cited "providers like DeepInfra with no obvious reason to subsidize their API costs" as evidence that published open-model prices reflect true costs.^[7]

The critical note concerns quality: in a May 2026 thread, pants2 observed, "Interesting then that OpenRouter tags many providers as FP8 and DeepInfra as FP4," flagging that DeepInfra's price edge may partly come from more aggressive quantization.^[6]

Competitive Landscape

vs. Together AI: Together AI offers more features (training, clusters). DeepInfra wins on simplicity and often price, and now matches on dedicated GPU offerings.

vs. Groq: Groq is faster on custom silicon. DeepInfra is cheaper for most models and supports far more model types.

vs. Replicate: Replicate has a community marketplace and image/video focus. DeepInfra is more LLM-focused and often cheaper.

vs. OpenRouter: OpenRouter routes to providers (including DeepInfra); DeepInfra runs its own infrastructure, so there's no aggregator margin.

Ideal User

• Budget-conscious developers running open-source LLMs
• Startups and scaleups that need cheap inference without enterprise overhead
• Teams migrating from OpenAI to open-source models for cost savings
• Agent workloads where token volume, not latency, dominates the bill

Bottom Line

Recommended for: Teams that want the cheapest reliable per-token inference for popular open-source models, agent builders with high-volume background workloads, and anyone migrating off OpenAI with a one-line base-URL swap.

Not recommended for: Latency-critical applications (Groq/Cerebras win on speed), teams needing deep fine-tuning pipelines, or workloads where quantization precision matters and you can't verify what you're getting.

Outlook: The $107M Series B with Nvidia and Samsung Next aboard removes the "underfunded" risk and funds real infrastructure — eight US data centers, Blackwell/Vera Rubin GPUs, SOC 2/ISO 27001.^[2] Five trillion tokens a week says the cost-leader strategy is working. The open question is margin durability in a race-to-the-bottom market, and whether FP4 quantization shortcuts erode the trust that cheap tokens depend on.

---