There is a temptation to see human review as a temporary constraint — something we will automate away once the models get good enough. This is wrong, and it is dangerous.
Human review is not the bottleneck to be eliminated. It is the quality gate that prevents AI-generated slop from compounding into technical debt that takes years to unwind. The organizations currently shipping AI-built products without engineering review are building on sand. They do not know it yet because the failures have not cascaded.
The pattern that works is: context in, background execution, reviewable output, human approval. The agent does the work. The human exercises judgment. Code does not merge without a human saying yes.
The alternative — letting AI-generated code flow into production without review — is how you end up with 18,000 water bottles ordered at a Taco Bell. The error compounds silently because nobody is watching the seam where machine output meets the real world.
I've argued that the bottleneck has moved from generation to review. Design the system around that need for review. Invest in review tooling, review skills, review at scale. Teams that treat human judgment as a valuable input will be better equipped to keep their AI deployments working over time.
Key takeaways
- Treating human review as a temporary constraint to be automated away is wrong and dangerous.
- The pattern that scales is context in, background execution, reviewable output, human approval — code does not merge without a human saying yes.
- Organizations shipping AI-built products without engineering review are building on sand and do not know it yet.
FAQ
Should we try to automate human review out of the loop?
No. Review is the quality gate that prevents AI-generated slop from compounding into technical debt that takes years to unwind. Removing it is how you end up with 18,000 water bottles ordered at a Taco Bell — a system that looked fine until it cascaded.
What is the right operating pattern for background agents?
Context in, background execution, reviewable output, human approval. The agent does the work. The human exercises judgment. Code does not merge without a human saying yes. This is not a conservative position — it is the only position that scales without compounding errors.
Related Essays
Merging Code Nobody Understands Is the Real Risk
Autonomous merge sounds like speed until nobody in the org knows what the product is anymore. Human review is how a company keeps a mental model of its own software.
Review Needs a Computer, Not a Diff
Agents have made code generation cheap and review the bottleneck. The fix is giving every reviewer a live machine with the change already running, not a diff in a browser tab.
Agents Start Broken and That Is the Point
New agents should deploy in supervised mode by default because no agent is production-ready on day zero.