Technology · Discipline

CPU reference implementations and GPU equivalence testing

Every GPU metric has a plain-Rust CPU twin that defines it.

Inference FabricPlain-Rust referenceCUDA kernels
Every kernel group, checked three ways ===refkerneldocplain-Rust CPU reference: the definitionfast-math CUDA kernelper-kernel lineage document Approximate where floating point has to be, exact where the design can promise it.
  • ~9,000lines of plain-Rust reference across thirteen modules
  • 32Rust integration tests, beside fifteen CUDA test files
  • 20older kernels kept with tests as regression baselines

The reference is the definition

In the Inference Fabric's CUDA engine every metric and every search family has a CPU reference in plain Rust. The reference states the formula the obvious way and decides every edge case; the kernel is an implementation that must match it.

I do not trust a GPU kernel to define the number it computes.

A consistency check keeps the kernels, the references and the per-kernel lineage documentation in agreement for every kernel group. A metric that has no kernel yet stays on the CPU with its limit stated. The method is told in Every metric gets a CPU twin.

metric with a kernelCPU=GPUmetric still pendingCPUno kernellimit stated, no hidden cap

Tolerances derived from the arithmetic

The tolerances are part of the method. They come from how single-precision reductions behave on 1,024-dimensional vectors and from the fact that the kernels compile with fast-math; tighter tolerances fail spuriously and looser ones hide bugs.

Where determinism is a design goal, in the search kernels, the comparison is exact: ties break on position, a seeded host-side draw replaces floating-point randomness, and the result is byte-identical run to run.

1e-61e-51e-41e-30search: exact, byte-identicaldot products and cosines: relative 1e-4 with an absolute floor of 1e-5squared distances: absolute 1e-4

Old generations kept as baselines

The Rust tests reach every part of the engine, and the CUDA tests exercise the device library on its own.

metric familiesexact searchtop-k enginelatticefused kernelslease adapterolder kernelsone tick per test

Older kernels are kept with sixteen tests of their own, so a rewrite is measured against the generation it replaces.