Corey James- 5judging seats, each with its own authority and none with the builder's
- 23close items, each needing a current independent pass
- 14 daysa work order held open until every one of them had it
Separate seats for building and judging
AgentOS: agent teams report success because they are built to finish tasks, and saying "done" finishes one. I decided the agent that builds a piece of work never judges it, and that the judging has to finish. AgentOS splits judgment into separate seats with separate authority, and the builders and the adversary run on models from different vendors so their blind spots do not overlap.
When the team proposed a dedicated testing role, I ruled that testing is a developer activity that produces evidence; a separate seat would have let builders stop testing. I hold the one human override on closure, and using it requires a written reason. I wrote the rules these seats follow and made the rulings that shaped them.
I treat AI-produced work the way a quality organisation treats any work: the people who build it do not sign it off, and the sign-off has to end in a verdict.
Checking that converges
To make the checking converge, the first broad quality pass freezes its findings. Later rounds look only at fix claims and changed evidence, and an unrelated new observation becomes separate governed work. A criterion that turns out to be wrong is withdrawn by an explicit decision, never quietly skipped.
All of this lives as data and state transitions in PostgreSQL, and the gatekeeper and quality machinery carries its own contract suites written with fail-before and pass-after controls. Several of these rules came from convergence problems I had to work through one by one.
Closing on delivered evidence
The hardest test was an API integration platform whose agents said it was done. I held the work order open until every close item had a current independent pass. That is the story in Closed by evidence, and the seat design is in Separation of duties for AI work.
What a checking seat receives decides what checking means. I removed checksums from the packets the seats receive, because they had started matching hashes instead of reading what was delivered.
The seat matched the orange hash block and passed; the grey delivery went unread.
Nothing to match, so the delivery itself is read.
Every fix carries a witness proof, and probes clean up after themselves, so checking leaves nothing behind in the system it checks.
Adversarial audits before the seats existed
Inference Fabric: before AgentOS existed, I ran the same practice by hand on AI-built subsystems. I commissioned read-only adversarial audits in parallel lanes, with findings ranked by severity and tied to file and line.
The large orange dot is the finding that overturned an earlier sign-off; the grey one is a report on weak evidence, sent back before anyone acted on it. Dot positions are illustrative.
The audit of the vector shard manager found that a subsystem previously signed off did not match its own documentation, and it traced the systemic cause. I acted on that finding even though it overturned the earlier sign-off. These audits are where the AgentOS seats came from; the seats are AI agents working in my own engineering estate.