Corey James- 5embedding models from four families in the baseline bake-off
A retrieval baseline strong enough to tie
Capabilities model needed a classical retrieval baseline strong enough to beat the model it was compared with. I ran the baseline's dense embedding models through ONNX Runtime from Rust, using the ort crate with the CUDA execution provider, in the same harness and the same runtime as the model under test, so the comparison differs only in method. The prebuilt runtime package lacked GPU kernels for my card and would have pushed the baseline to the CPU, so I built ONNX Runtime myself.
The embedding model came from a bake-off that replaced an incumbent chosen by habit, and it was fused with BM25 using weights found by sweep. The resulting baseline tied the fine-tuned model at recall 5 and 10. See The baseline that tied us.
One kept. Letters are illustrative.
A precision mode that moved the rankings
The work that mattered was a precision finding. TF32, a reduced-precision mode for matrix multiplication enabled on the GPU path, changed the baseline's rankings relative to the CPU path, which made the baseline's score depend on where it ran.
A baseline's numerical settings are part of the experiment.
Small speech models inside the voice engine
Helix has a conversation engine where latency decides whether talking to it feels natural. I run the small speech models in process in the Rust voice engine through ONNX Runtime and keep the large ones as services. See A self-hosted colleague in a Teams call.
I use ONNX Runtime as a portable inference layer from Rust, for evaluation and for real-time speech, and I measure its numerical behaviour. ONNX Runtime and the models it runs are third-party; the custom build, the harness integration, the precision finding and the voice-engine integration are mine.