Corey James- 2,048simultaneous clients in one burst, none failed or rejected
- 2,740MiB, the engine's whole footprint, on the card the larger serving model already uses
- 12 of 12bit-identity gates passed across batch compositions
Every tool an agent can reach is described to it on every turn, and the bill and the latency grow with the estate. Meanwhile an agent that cannot see the right tool improvises.
I compiled a tool catalog into a fine-tuned 0.6-billion-parameter model that cannot name a tool outside the catalog, with a decoder whose output bits are identical across batch compositions. Start with A hundred tokens instead of forty-two thousand and Bit-identical across batch compositions.
The catalog's 42 tools are the whole output space; a name outside it is unreachable, in any run.
Every one of each set. Fixed-order reductions, a 32-lane shuffle tree inside a warp and a 256-thread tree above it, regardless of how many rows are in flight.
A wrong real tool name is a recoverable error; a plausible fake one is a silent failure.
I measured it against the strongest retrieval baseline I could build, which tied it, and I published the tie. See The baseline that tied us.
It serves its clients from the memory left beside a larger serving model on one GPU, and it admits work against measured memory, refusing with a number when a request will not fit. See A router beside a production model.