Engineering story · Inference Fabric

Asking the engine what it meant

One input's convention was ambiguous in the vendor's documentation, and guessing wrong would silently extract the wrong token; so at startup the runtime runs a tiny batch both ways and refuses to start unless exactly one answer is consistent

C++ runtimeInference FabricStartup self-test
One sequence, one value, two readings
1token between the two readings
Sequence length · or last-token index
  • 3tokens spent at startup to settle the question
  • 2runs of the same micro-batch, one per convention
  • 64output values checksummed per sequence

The mining engine takes an input that tells it which token's hidden state to gather for each sequence. The vendor's documentation leaves its convention open.

Sent the last index, read as a length

The engine steps back one and gathers the second-to-last token.

Sent the length, read as an index

The engine reaches one past the end and gathers the next sequence's first token.

Outlined: the token that was wanted. Orange: the token gathered. Light blue: the next sequence. Neither mistake raises an error, and the numbers look like any other hidden state.

Why guessing was not acceptable

Those hidden states feed a corpus of hundreds of millions of vectors that my research clusters and measures.

A systematic off-by-one would corrupt every vector in the same way, and the corruption would look like a slightly different model rather than a bug.

The silent kind of error, which the detector in A detector for a silent corruption exists to catch after the fact, is the kind worth refusing to risk at all.

The test at startup

So the runtime asks the engine, during warm-up. The output is decoded in software for the checksum, so no assumption about the output format leaks into the test.

The micro-batch
  • Run with the exclusive valueschecksum per sequence
  • Run with the inclusive valueschecksum per sequence
  • Both runs non-zerorequired
  • Which sequence's output changed between the runsnames the convention
Only one convention explains which output moved.
At the startup check (orange line)
Ambiguousfails hard
Runs identicalfails hard
One explanationfrozen
The one function that fills this input for real batches errors while the convention is unknown.

What this demonstrates

Treating a vendor's under-specified interface as a hypothesis to test rather than a fact to assume, and making the runtime refuse to run rather than run wrong.

The decision to detect rather than assume was mine; the detection was implemented by AI coding agents, a mix of frontier and open-source models, under my governance system.