I recently built an internal tool that decides whether someone who signs up for our product gets routed as a sales lead. It is a mix of deterministic code and a small LLM classification step. It works. The problem is that no one knows how it works except me.
The scoring formula is logic in a repo. There is no place where it is atomically described, inspectable, or auditable. That is barely good enough for me, and not good enough for an enterprise. We are building agents at a rapid pace, and the work they do is the tip of the iceberg. The harder question is whether they are doing the right work, and whether they are doing it correctly.
Imagine what it would mean to have a directory of AI agents the way you have a directory of employees. Every agent has a description of what it does, how often it runs, what its recent outputs look like. You can inspect logic. You can judge performance. When the CEO says "that's wrong, it should work this way," there is a clear path to making the change and verifying it took effect.
Humans learn from outside stimulus. Agents have none of that by default. They will keep doing the wrong thing forever unless we build the infrastructure for humans to watch, judge, and course-correct. The agents that work in production will be the ones designed to be supervised, not the ones designed to be autonomous.
The other pattern I keep seeing is that people build agents as monoliths. One big automation that does everything. The better model is atomic agents — small, single-purpose units with clearly defined inputs and outputs, each independently testable, each swappable for a cheaper model when the task is simple enough. You cannot do that when everything is tangled together. I've argued separately that users, not developers, own iteration — the directory is what makes that handoff possible.
Sources
Related Essays
Users Should Iterate on Agents, Not Developers
Agents are always slightly wrong when first built. The people who know what is wrong are not developers — they are the users who interact with the agent every day.
When Building Is Cheap, Coordination Breaks
Three brand teams in the same Fortune 50 build the same aggregator without talking. AI lowered the cost of building, and the budget process that used to coordinate everyone disappeared with it.
The Coordination Crisis AI Tooling Created
When everyone can build, nobody knows what has already been built. Five teams independently spin up the same Slack bot, and the duplication goes undetected for months.
Key takeaways
- Agents are doing real work in production with no atomic description of what they do, how they work, or whether they are doing it correctly. Most are visible only to the engineer who built them.
- The model to copy is a directory of employees — every agent has a description, a run cadence, recent outputs, inspectable logic, and a clear path to corrections that take effect.
- Humans learn from outside stimulus. Agents do not. They will keep doing the wrong thing forever unless we build the infrastructure for humans to watch, judge, and course-correct.
FAQ
What is the agent directory problem?
Agents are running production work, but unlike employees they have no central record describing what they do, how they decide, or how they perform. Without that record, the organization cannot audit, govern, or improve them.
Why are atomic agents better than monolithic ones?
Atomic agents are small, single-purpose units with clearly defined inputs and outputs. Each is independently testable and swappable for a cheaper model when the task is simple. Monolithic agents tangle everything together and become impossible to inspect or fix.