Every time a model gets a new capability, someone on Twitter declares that a thousand startups just died. That framing is lazy. But there is a real version of the concern, and it is about where you put your engineering effort inside the agent harness.
A year ago, building a serious agent meant building a retrieval pipeline. You stood up a vector database, you chunked your data, you tuned your embeddings, and you wired all of it into the prompt. That was table stakes. Today, a capable agent can often just navigate the codebase or the dataset directly, find what it needs, and skip the entire apparatus. The infrastructure you built to compensate for the model's weakness became weight you now have to carry.
This is the actual risk, and it is not that the LLM ate your product. It is that you invested in the wrong layer of the harness. The model is a box of parts, and the parts keep getting better and cheaper. If you brute-force everything with the most expensive model on every call, you will lose on cost. If you over-build infrastructure the model is about to make unnecessary, you will lose on maintenance. The discipline is figuring out which parts of the work belong to the model, which belong to a deterministic software function, and which belong to durable infrastructure that will still be load-bearing two model generations from now.
The answer is not the same for everyone. If you are Cursor or Notion or a search-native company at massive scale, a purpose-built retrieval layer is worth every dollar, and the companies that built those are crushing it. But if you are building an internal agent for a knowledge-work team, you almost certainly do not need that. The agent can wrap the existing systems and it will be fine.
The right move is to prototype with the most capable setup first, prove the workflow, then peel back. Replace LLM subroutines with software functions where logic is deterministic. Drop infrastructure the model no longer needs. Agents are software, and good software earns its complexity rather than assuming it. Invest in the layer that stays durable, not the layer the next model release deletes for you.
Key takeaways
- The model is improving so fast that infrastructure built to compensate for its weaknesses becomes dead weight.
- Vector RAG pipelines made sense when agents could not navigate a codebase or dataset on their own - now many can.
- Heavy retrieval infrastructure is justified at frontier scale but wasteful for most internal agents.
FAQ
Does this mean vector databases are obsolete?
No. For products operating at the scale of Cursor, Notion, or Linear, dedicated retrieval infrastructure absolutely matters. The point is that most internal agents do not operate at that scale and do not need it.
How do I know which part of the harness to invest in?
Start by assuming the model can do more than you think, then add infrastructure only where it measurably fails. Build with the expensive model first, then replace what you can with deterministic software functions.
Sources
Related Essays
The Agent Harness Problem
Enterprise agents need layered interfaces, real software skills, and flexible platforms. The harness around the model matters more than the model.
Software Factories Run on Pre-Prod
Autonomous agent loops are real, but almost every impressive self-driving software factory is running on code without customers. Production is a different game.
A Real VM Is the Primitive for Cloud Coding Agents
Containers cannot replicate a developer's local environment. Moving coding agents to the cloud requires a full virtual machine, and that is harder than it looks.
Drowning in pull requests that need your review? Try Tembo Review, a beautiful AI-assisted PR review tool unlike anything you’ve used.