Corey James- 20%off cold 32,000-token prefill, outputs identical
- 48special-value combinations behind the exact-arithmetic rewrite
Patching the vendor's library in the vendor's toolchain
ActiveKV serves a 27-billion-parameter hybrid recurrent-attention open-weight model on one Intel Arc Pro GPU, and on that card prefill and the 4-bit-weight model path run their large matrix multiplications through Intel's oneDNN. The Intel Arc Pro GPU is a new architecture and the open kernel stack had gaps.
I decided to patch oneDNN's own sources inside Intel's toolchain image, so the binary matches what the serving engine loads, and to gate every change on bit-identical output against the unpatched path, measured on the whole workload on a quiet benchmark host. The gate exists because of what sits above oneDNN: ActiveKV's exact mode must match fully resident attention bit for bit.
A GEMM patch that moved a single bit would void every parity result built on it.
One catalogue entry for the Xe2 prefill shape
The patch that mattered most was one line of data. oneDNN picks a GEMM kernel configuration from a catalogue keyed by shape, and for the Xe2 prefill shape it was picking a poor tile. One added catalogue entry, the smallest change that removed the cost, fixed the choice with identical outputs.
Tightening the 4-bit-weight path
Further patches tightened the 4-bit-weight (W4A16) lowering: weights reused once loaded, conversion and scaling done locally, scales broadcast, and the hardware's dot-product-accumulate instructions used on the tail of a computation. A separate patch adds a validation switch that pins the GEMM shapes that otherwise vary from run to run, so shape variance cannot hide a real parity failure. One rewrite produced exact instruction-set arithmetic with no whole-workload speed gain, and I kept it because exactness is a property I want in the product.
The patches travel with ActiveKV as patches to Intel's stack. See Bringing a serving stack up on new silicon and Proving a rewrite is the same system. I can work inside a vendor's math library on hardware the ecosystem has not caught up with, find the one change that matters and keep every output bit where it was. oneDNN is Intel's; the patches, the gates around them and the choice of which changes to keep are mine.