Corey James- 1 scanrebuilds every result after a restart
- 1 job idset once per connection scopes every poll
The input side of my broker is told in Millions of queued items that were not there: two-phase dispatch and in-place tombstones made the count of tokens received equal the count routed, to the token. The output side had the same disease in mirror image. A result could be marked finished, handed to a poller, and lost between the broker's memory and the receiver's hands. So the completion side got the same discipline, and polling got a data structure that scales with what is ready.
A log for results
Results from the GPUs are written to completion segments, append-only files. Recovery after a restart is a scan of those segments, so a result that reached disk is never forgotten by a crash. The pending registry, the hot path that tracks what is in flight, stays in memory as a hash map, and the completion store is a separate pluggable backend, so the two memory profiles never compete.
Two phases on the way out
Delivery has two phases, and nothing leaves the log until the second one has happened.
A result can be re-offered, and it is never silently gone.
Polling that costs what is ready
The original poll checked every pending sequence id for a job, which cost as much as the backlog. The producer now sets a job id once per connection, every completion is appended to its job's ready queue, and polling a job returns up to a requested number of ready results. A multi-job service polls all ready queues at once. The queues are rebuilt from the stored job ids on restart. A work item carries the database primary key it came from, so the completion can be written straight back to its row.
What this demonstrates
Applying one accounting rule to both ends of a pipeline: nothing is counted as done until the physical event that makes it done has happened, and the structures that answer "what is finished" are sized by the answer rather than by the question.
The delivery rule and the job-scoped queues were my design; AI coding agents, a mix of frontier and open-source models, implemented them under my governance system.