Corey James- 38per-cluster metrics, every family with a CPU reference
- 12 of 12bit-identity gates across batch compositions in the decoder
- 256bins in the histogram that yields each Gaussian width
- 3Lloyd iterations, fixed into the silhouette's definition
Per-cluster diagnostics no library offered
Inference Fabric stores clustered embeddings, and a clustered embedding space needs constant quality checks: is a cluster tight, drifting, bimodal, contaminated by outliers, redundant with a neighbour? The GPU libraries I had offered search and whole-dataset clustering, with no per-cluster diagnostics, and none targeted CUDA 13 on Blackwell from Rust. I decided to write the kernels, and I set their numerical rules before any were written.
The engine is called through the raw CUDA driver API and compiled to native Blackwell binaries. It targets Blackwell only and computes in f32 with no tensor-core path. Its per-cluster metrics fall into a few families.
Moments that stay stable in f32
The centroid family gives cohesion and inertia from member-to-centroid distances, spread and skew of those distances, the ratio of centroid strength to the spread of member norms, distance to the global centroid, and the fractions of members beyond two standard deviations or within one.
Welford and Chan merges keep variance and skew stable in f32 across thousands of threads.
Pairwise metrics without a distance matrix
The pairwise family gives coherence from mean pairwise cosine, compactness from mean pairwise distance, and silhouette from a two-means split. The two-means step starts from the first and last member and also yields a clustering tendency from the drop in inertia and a balance between the two halves.
The diversity family counts redundant and distinct pairs against cosine thresholds, measures breadth and coverage, and takes entropy from a 20-bin histogram of pairwise distances. That needs two passes, because the bin edges come from the minimum and maximum the first pass finds. A Gaussian family treats members as potentials of width σ, where σ is a scale times the median pairwise distance taken from a histogram filled during the tile loop; a histogram gives that median without storing the distances, at the cost of one bin width of precision. Neighbourhood metrics compare each cluster with its nearest clusters, including question-to-answer divergence for paired embeddings.
Pairwise kernels are tiled and reduce as they go, so memory is bounded by the tile. Each kernel belongs to a scaling class, and nothing is sampled.
A CPU twin for every family
Thresholds are kernel arguments, and every metric has a lineage file naming its kernel, its destination and its timing. Equivalence tolerances are set from how f32 reductions behave on long vectors.
A bad cluster has to read as bad.
I left diagnostics unclamped, because a metric forced into range hides the failure it exists to catch.
The same bits in any batch
Capabilities model needed a different property from its decoding kernels: the ranking of tools must not depend on which other requests share the batch. I ruled that output bits must be identical across batch compositions and set the structure to get it: one block per output row, fixed reduction trees, no K-split tiling, and row kernels shared between the serial and batched engines. The story is Bit-identical across batch compositions.
partial sums from several blocks, combined in whatever order they finish
one owner, a fixed shuffle tree, the same order every time; throughput recovered elsewhere
I make numerical decisions deliberately: what to reduce in which order, what to approximate and by how much, what to clamp and what never to clamp, and what property the application actually needs.