Baseten

Company Overview

Baseten is an AI inference platform that lets teams deploy custom and open-source models to production with enterprise-grade reliability and compliance.^[1] Founded to solve the gap between training a model and serving it at scale, Baseten provides the infrastructure layer between model development and production serving.

The company raised $585M total, including a $300M Series E in January 2026 led by CapitalG (Google's investment arm) and IVP, with NVIDIA participating, valuing the company at $5B.^[2] As of May 2026, Baseten was reportedly in talks to raise another $1B at an $11B valuation — with some investors offering around $15B — after annualized revenue reached roughly $600M at the end of Q1 2026, up from ~$200M at the start of the quarter.^[3] Customers include Cursor, Notion, OpenEvidence, Abridge, Clay, Zed, ClickUp, and Writer.^[2][1]

What It Does

Baseten provides a managed platform for deploying ML models as API endpoints. Key capabilities:

• Dedicated inference — Deploy any model (custom PyTorch, TensorFlow, or open-source) as a scalable API^[1]
• Truss framework — Open-source model packaging tool that standardizes deployment^[4]
• Model APIs — Pre-optimized OpenAI-compatible endpoints for frontier open-source models (Kimi K2.6, DeepSeek V4, GLM 5.1 as of June 2026)^[1]
• Training — Managed fine-tuning and training infrastructure, plus Baseten Loops, a training SDK for frontier RL launched in 2026^[1]
• Frontier Gateway — A monetization API for model owners, new in 2026^[1]
• Autoscaling — Automatic scale-to-zero and scale-up based on traffic
• Multi-cloud & self-hosted — Deploy on Baseten's cloud or in your own environment

How It Works

1. Package your model using Truss (open-source) or bring a container
2. Deploy to Baseten's GPU fleet — choose from T4, L4, A10G, A100, H100, or B200^[5]
3. Optimize with TensorRT-LLM and Baseten's inference engine
4. Scale automatically based on request volume, including scale-to-zero
5. Monitor with built-in observability and logging

Baseten handles GPU orchestration, load balancing, and infrastructure management. The Truss framework provides a standardized way to package models with their dependencies, pre/post-processing logic, and configuration. Truss remains actively maintained — v0.18.9 shipped June 11, 2026.^[4]

Pricing

• Per-minute GPU billing — pay only for compute time used, no idle fees^[5]
• Plan tiers — Basic ($0/month, pay-as-you-go), Pro and Enterprise with volume discounts, priority GPU access, and custom SLAs^[5]
• Scale-to-zero — no charges when models aren't receiving traffic
• Forward deployed engineers — dedicated engineering support for enterprise accounts

Per-minute rates from Baseten's pricing page, converted to hourly. Volume discounts on Pro and Enterprise plans.^[5]

Strengths

• GPU breadth — Wide GPU selection among inference platforms (T4 through B200)^[5]
• Truss open-source — No vendor lock-in for model packaging^[4]
• Compliance — SOC 2 Type II and HIPAA set it apart from most competitors^[1]
• Enterprise support — Forward deployed engineers, not just tickets
• NVIDIA backing — Strategic investor brings hardware access advantages^[2]
• Revenue momentum — ~3x ARR growth in a single quarter (Q1 2026)^[3]
• Scale-to-zero — Cost-efficient for bursty workloads
• Self-hosted option — Deploy in your own VPC for maximum control

Weaknesses / Risks

• Complexity — More setup required than API-first platforms like Replicate or DeepInfra
• Not the cheapest — Premium pricing reflects enterprise features; per-minute billing can cost more than hourly rentals for sustained workloads^[6]
• Less developer-friendly — Steeper learning curve for simple use cases
• Competition — Together AI and Fireworks AI competing aggressively on similar turf
• GPU dependency — No custom silicon differentiation vs Groq/Cerebras
• Valuation risk — A reported $11B price on ~$600M ARR assumes inference demand keeps compounding^[3]

What Developers Say

Developer sentiment on Hacker News through mid-2026 is largely practical and positive — Baseten comes up as a default choice for serving open-weight models — with the sharpest criticism aimed at its billing model.

"Baseten and Fireworks have been my goto. Currently Baseten has ~610ms TTFT and ~82 tk/s for Kimi K2.6, which is roughly 2x the throughput of GPT-5.4 (per their openrouter stats)." — HN commenter spmurrayzzz, April 2026^[7]

"To me, the play is: open weight on a provider like BaseTen (solid performance, low price point), or pay up for Gemini3.1 Pro if you need it." — HN commenter sjt-at-rev, April 2026^[8]

"Baseten bills by the minute so it can be really useful if you need to handle small bursts of compute, but on the flip side they charge you a x5 premium if you end up being billed for complete hours" — HN commenter littlestymaar, on Zed choosing Baseten^[6]

The pattern: developers treat Baseten as a performance-credible home for open-weight frontier models, while cost-sensitive users flag that per-minute convenience pricing carries a premium at sustained utilization.

Competitive Landscape

vs. Together AI: Together AI offers a broader platform (pre-training, research) with newer hardware (GB200/GB300). Baseten wins on compliance and GPU breadth.

vs. Fireworks AI: Fireworks focuses on speed optimization. Baseten offers broader GPU options and stronger compliance posture.

vs. Modal: Modal is more general-purpose (any Python workload). Baseten is purpose-built for model serving with better model-specific tooling.

vs. Replicate: Replicate is simpler but offers less control. Baseten targets teams with custom models and enterprise requirements.

Ideal User

• Enterprise ML teams deploying custom models with compliance requirements
• Companies needing broad GPU selection for different model sizes
• Teams wanting open-source tooling (Truss) without vendor lock-in
• Organizations requiring HIPAA-compliant AI inference

Bottom Line

Baseten is the enterprise play in AI inference — strong compliance, broad GPU support, and hands-on engineering support, now backed by ~$600M ARR and a reported $11B valuation in the making.^[3] The expansion into training (Loops) and model monetization (Frontier Gateway) signals ambition beyond serving.^[1]

Recommended for: Teams past API wrappers that need production-grade infrastructure for custom or open-weight models, especially with HIPAA/SOC 2 requirements or enterprise support needs.

Not recommended for: Simple hosted-model use cases better served by Replicate or DeepInfra, sustained 24/7 workloads where per-minute billing premiums outweigh convenience, or teams betting on custom-silicon speed (Groq/Cerebras).

Outlook: If the $1B round closes near $11B, Baseten cements inference as its own infrastructure category; watch whether revenue growth (~3x in Q1 2026) holds as Together and Fireworks compete on price.

---

Research by Ry Walker Research • methodology