Technology · Protocol

CBOR

Canonical CBOR encodes the vector store's RPC wire and the extraction specifications, so equal specifications are byte-identical.

Inference FabricVector store RPC wireCanonical extraction specs
One length-prefixed frame · request and vectors together
length
prefix
request fieldssmall, structured
binary vector payload
self-describing; no base64 step, no schema compiler
16 MiBframe limit; oversize fails at the boundary
  • 33RPC variants in the vector store's service API
  • 6families they fall into, search among them
  • 3encodings across the fabric's links

One wire for requests and vectors

Inference Fabric's Vector Shard Manager needed a wire that carries small structured requests and large binary vector payloads in the same frame. I chose CBOR, length-prefixed over TCP to mirror the framing of the fabric's broker protocol, through the ciborium Rust library.

One frame per call · request fields and binary payload · illustrative
Payload appends
Generation lifecycle
Search
Residency control
Checkpoints
Window fetch
structured request fields
binary vectors, no base64

Specs that are the same bytes everywhere

Inference Fabric also runs hidden-state extraction, and each run is described by a spec that tells the GPU worker which layer's states to capture and how. Producers attach the spec to every work item and the broker carries it opaquely.

I wanted two equal specs to be the same bytes wherever they were produced, so the spec is encoded as canonical, deterministic CBOR, and the mining tables store spec identifiers derived from that encoding.

Default encoder · same spec, two producers

equal values are not promised equal bytes, so the identifiers can split

Canonical CBOR · same spec, same bytes

one spec identifier in the mining tables, wherever it was produced

I use canonical encoding wherever bytes stand for identity.

The extraction workload is described on the Hidden-state extraction page.

The broker's own transport uses bincode for its fixed Rust-to-Rust message types; that choice and the broker's design are in A message broker that understands tokens. I choose an encoding per link from what crosses it, as described on Binary wire protocol design.

bincodeRust services I fully control
CBORthe vector store
Newline-delimited JSONthe capabilities-model socket
Fixed fields where both ends are mine, a self-describing frame where structure and bulk bytes travel together, readable lines where an operator looks.