When you build an agent platform, the first instinct is to treat agent definitions as application data. Users log in, edit a text field describing what the agent should do, hit save, and the new behavior is live. It feels clean. It is also a trap.
We went the other direction. Agent specs live in a git repo, not in the app's database. Every agent is a YAML file. The studio reads that repo as its source of truth. Change the model from Opus to Sonnet in the file, commit it, and the running system reflects the change. The repo is the contract, and the app is just an execution surface over it.
This matters for one reason that database rows can never give you: review. A repo with PR-only merges means no one can accidentally destroy a working agent. A non-technical operator can spin up a draft and iterate fast, but the moment an agent is marked production, changes go through a pull request. Someone has to say yes. That is not bureaucracy. That is the difference between a demo and a system you can run with fifteen people touching it.
The part people miss is that the gatekeeper does not have to be an engineer. A YAML spec is not code. It is a readable description of intent, tools, and steps. A customer success lead can look at a diff and decide whether the change is sane. That decouples approval authority from engineering capacity, which is exactly what you need when agents start breaking out of engineering and into the rest of the org.
The deeper principle: if it is not in version control, it is not operationalized. The same discipline that made software deployable makes agents deployable. This is why agents are software, not prompts: the prompt is the cheap part. The factory around it, the review, the history, the source of truth, is the actual product.
If you are building an internal agent platform, start by asking where the definitions live. If the answer is a text box, you have a demo. Put them in a repo and you have something you can trust at scale.
Key takeaways
- Agent definitions belong in a version-controlled repo, not as editable rows in an application database.
- PR-only review lets non-technical people contribute agents without the ability to silently break production.
- A YAML spec is reviewable by non-developers, which decouples the gatekeeper role from engineering.
FAQ
Why not let users edit agent definitions directly in the app?
Direct database edits give you no history, no review gate, and no rollback. When agents are specs in a repo, every change is a reviewable diff and a non-technical operator can still read and approve it.
Doesn't requiring PRs slow down iteration?
Only for production agents. A new agent in draft can be iterated fast in a YOLO mode. The PR gate kicks in once an agent is marked prod, which is exactly when you want a second set of eyes.
Related Essays
Agents Are Software, Not Prompts
The industry treats agents as a new category. They are not. Agents are software, and the same engineering principles that have always mattered still apply.
Prototype on MCP, Productionize in Code
Pure LLM execution is a prototyping medium, not a production architecture. The $15 agent run becomes a $1 run when repeated reasoning gets hardened into code.
Kill the MCP, Ship the CLI
MCP servers dump their whole surface into your context and cost more per call. A wrapped CLI with good help output is cheaper, local, and easier to test.
Drowning in pull requests that need your review? Try Tembo Review, a beautiful AI-assisted PR review tool unlike anything you’ve used.