Corey James- 3weeks at most for a native port, bit-identical in every parity cell
- 24performance promotions in four days, each through an intake review
- 23 of 23close items closed under independent review
Most of the code behind the systems on this site was written by AI coding agents. The interesting part is that the programs finished, passed independent review, and can be trusted: a native port with bit-identical output, an API platform closed under independent review, a fine-tuned model with a frozen evaluation. That happens because I run agent teams the way an architect runs an engineering organisation. I write the rules, I separate the builders from the judges, and I let evidence decide.
The challenge
AI coding agents are fast, tireless and confidently wrong. Left to themselves they over-claim completion, drift from the plan as context fills, and optimise for whatever metric is in front of them, including gaming a benchmark fixture. Human attention is the one resource that does not scale. The question for anyone building serious systems this way is how you know what they built is what you asked for, and correct.
Constraints
Mixed models, by design. The teams use frontier models and open-source models together, because different families have different failure modes and that is useful if you arrange for them to check each other. One human, so review had to be structural. And truth over throughput: a fast program that produces unverifiable results is worth nothing.
My approach
Four rules, applied to every program.
I write the acceptance rules before the work starts: a definition of done, an intake review standard, a certification procedure, and a promotion procedure for performance changes. When a new situation produces a ruling, the ruling is written down and becomes standing doctrine.
Builders and judges are different agents from different model families. The model that wrote a change does not review it. Quality assurance, a gatekeeper and an adversarial reviewer are separate seats.
Evidence decides. A lane's claim that it is done is a claim. The system closes work only on evidence that exists and checks out.
The machinery enforces the rules. My governance system refuses forbidden actions and keeps the authoritative state outside any model's context, so compaction and crashes cannot lose it.
What I built
The programs below all ran this way, and most have their own story.
The ActiveKV native port and performance program: a bit-identical port, then a run of performance promotions, each through an intake review that rejects fixture-tuned gains.
An API integration platform, closed by the system itself on every close item under independent review after I held it open against the agents' own "done".
The capabilities model: data, training, a custom decoder, serving and a frozen evaluation, built under a single work order with an independent quality seat.
Three social-platform integrations built in about two days against a written standard that every tool must satisfy before it is used.
Agent Swarm phases with gatekeepers who never implemented what they judged, and a phase held at conditional pass on two failed gates.
AgentOS itself, the control plane that makes the above routine, built by agents working under it.
Difficult decisions
Voiding passing work. During the port, over a thousand passing trials and over a hundred engineering notes were voided when I found that a native component had been registered after the system had already chosen its code path. Everything downstream was redone with proof of binding order.
The tests were real, and they were testing the wrong thing.
Rejecting a real gain. One performance candidate produced a genuine improvement. Its cutoff happened to match the size of the test fixture. I rejected it and wrote the rule that fixture-derived thresholds never enter the product. That rule runs as an intake screen on every candidate.
Holding work open. When an agent team reported the API platform done, I required the independent close evidence instead. It took longer and it closed clean.
Results
Technologies used
Frontier coding models · Open-source models · git at program scale · Work-order governance · Separation-of-duties review · Written definitions of done · Intake and promotion procedures
What this demonstrates
I lead AI-assisted engineering as an architect and director. I set the architecture, write the rules, review the evidence and own the result. The systems on this site are the proof that this way of working produces systems that hold up under independent review.