Corey James- 244answered, of 512 requests offered by 64 clients to a queue of sixteen
- 268refused with a typed busy reason
- 8 of 8requests in flight answered when the daemon was terminated
The tool-discovery model answers one question, which tool does this request need, for every agent in my estate, and it does so from a GPU it shares with a larger serving model. A router beside a production model tells how it fits in the memory it is allowed. This is how it serves: a daemon whose contract says what happens to every request it is offered, including the ones it refuses.
Two daemons
The serial daemon runs one thread per connection over a Unix socket and holds the engine behind a single mutex. Decode is batch-one there, so the mutex is the queue and nothing more is needed. The fleet daemon is the serving shape: connection threads feed a bounded queue, and one engine thread steps up to the batch limit of rows drawn from every sequence in flight, each with its own position in its own paged cache. A sequence leaves the batch the moment it finishes, so a short answer never waits for a long one.
One request at a time, batch one.
A bounded queue feeds the batch; each sequence keeps its own paged cache and leaves the moment it is done.
The daemon is reachable as a standard tool-protocol server over standard input and over streamable HTTP with TLS, and its wire format is a length-prefixed frame.
The contract
Nine conditions were written first, each with a measurement.
Start-up in a fixed order
A refusal before bind means no client ever saw a half-ready service.
What this demonstrates
Serving a small model with the discipline of a full service: a stated contract, typed refusals instead of silence, a measured load test against the contract, and a start-up order that makes a half-ready daemon impossible to reach.
The contract and the fleet design were mine; AI coding agents, a mix of frontier and open-source models, implemented the daemons and ran the gates under my governance system.