Corey James- 4integers describe any GPU consumer to the broker
- 33RPC variants on the vector store's CBOR wire
- 16 MiBframe limit on that wire, with a streaming fetch for larger windows
I choose the wire per link from what the traffic costs and who has to debug it, and I prove cross-language implementations equal with golden vectors.
One audio link, written twice
Helix had to connect Microsoft's real-time media stack, which runs only on Windows, to my voice engine on Linux, without putting call audio in a third party's hands. I decided on a purpose-built binary link over one authenticated full-duplex WebSocket per call, implemented twice, in C# in the .NET media service and in Rust in the voice engine's endpoint.
When the link drops, the bridge reconnects inside the same Teams call. The bridge is told in A self-hosted colleague in a Teams call.
Token-level transport for the broker
Inference Fabric moves billions of tokens and bulk tensors between independently restartable services, and I gave those links binary framing over TCP. On the producer side, the FAFO Core inference client submits work to the broker routed by model id, so a producer can hand off millions of tokenised items and exit. The broker's protocol crate defines the work item, the packed batch and the message catalog.
The broker packs to a contract each GPU consumer states about itself, so different GPU models drain one backlog with no special cases. A GPU worker uses one connection, which replaced a design with two registrations per worker that once sent heartbeats to the wrong session and starved a working GPU.
The broker's protocol carries a version constant it never negotiates. The broker is told in A message broker that understands tokens.
Vectors and hidden states
The Vector Shard Manager speaks length-prefixed CBOR over TCP, with a streaming fetch for moving windows of vectors between nodes. Hidden-state outputs travel in a self-describing binary envelope, which the broker carries opaquely from the GPU runtime to the vector store.
Where I stayed out of binary
The tool-discovery engine in Capabilities model listens on a Unix domain socket, and I chose newline-delimited JSON over a length-prefixed binary frame: the payloads are a few hundred bytes, so framing is not the cost, and an operator can drive the socket by hand and read the answer. Both ends serialise the same Rust types, and every request carries a version the engine reads before anything else, refusing an unknown one with a typed error. The engine reports a fingerprint of the tool catalog its decoder was built from, and a client whose own catalog hashes differently refuses to start and names both. The serving engine is in A router beside a production model.
Binary goes where the traffic is real-time audio, billions of tokens or bulk tensors; where payloads are small, a readable wire wins on diagnosability.