Corey James- 256byte patterns, every one, on which the hardware and table eight-bit paths agree
- 2, 4, 8rows per block in the tiled kernels, each bit-identical to the untiled form
- 2.53×denser in memory with four-bit weights, and no faster
The tool-discovery model ships in several precisions, and the smallest is a four-bit floating-point format with a block scale: a nibble per weight holding a two-bit exponent and a one-bit mantissa, and an eight-bit scale per block of weights. The format's documentation describes the arithmetic. It does not pin down, byte by byte, which nibble is which weight and how the block scales are laid out against them, and a kernel written against the wrong layout decodes plausible nonsense.
The scale byte, then one nibble per weight.
Correlate before you write
I had the same weights in a known precision. So before any kernel was written, each candidate interpretation of the layout was decoded on the host and correlated against the known weights.
That gap left no room for doubt, and the layout went into the kernel as a fact with a number attached rather than as a reading of the documentation.
The kernels
The four-bit kernels decode nibbles through lookup tables held in constant memory, one for the two-bit-exponent values and one for the eight-bit block scales, with a matrix-vector form for decode and a row form for prefill. Beside them sit the eight-bit kernels, where a hardware conversion path and a constant-table path exist together and a kernel proves they agree on every byte pattern, so the faster path can be used without a doubt about its correctness. The register-tiled row kernels, including paired projections in one launch, are bit-identical to the untiled form, which is the test that admits them.
Footprint, not speed
On a card shared with a larger serving model, density is the margin that matters, and four-bit weights bought it. They did not make decode faster on this hardware.
The four-bit build is the footprint build, the eight-bit build is the serving build.
What this demonstrates
Treating an under-specified format as a measurement problem, admitting kernels by bit-identity with their simpler forms, and stating what a precision buys and what it does not.
The correlation method and the positioning of each precision were my decisions; AI coding agents, a mix of frontier and open-source models, wrote and tested the kernels under my governance system.