Engineering story · Inference Fabric

Where a cluster sits

Ten neighbourhood metrics that place each cluster among the clusters around it, computed from precomputed neighbour lists in a handful of GPU launches

CUDA · RustPrecomputed neighbour listsNative Blackwell binaries
One cluster and its neighbourhood · sorted by distance
Precomputed once, no search inside a kernel
20nearest clusters in each sorted row
  • 6values from one sorted row, one thread per cluster
  • 0distance computations inside the cluster-level kernels
  • 24 KBshared memory for a cluster's own and five neighbour centroids
  • 2metrics that need every member

Most cluster statistics look inward: how tight, how spread, how many sub-groups. The questions that decide what to do with a cluster look outward. Is it crowded by neighbours or alone? Do its members belong to it or to the cluster next door? Does it sit on the topic its neighbours share, or off it? Has it drifted from the group it was part of? My engine answers those with a neighbourhood family of metrics that runs from precomputed neighbour lists, so none of the kernels has to search.

The neighbour lists

For every cluster, an approximate-nearest-neighbour index over the centroids returns a sorted row of the twenty nearest clusters and their distances, with the cluster itself at position zero. The member-level kernels take the neighbour centroids themselves and cache them in shared memory.

Cluster-level metrics from one row

The local centroid gap is one over one plus the distance to the nearest other cluster. Overlap is one over one plus the mean distance to the ten nearest. The raw nearest-cluster distance is kept as its own output because the Gaussian family needs it as a barrier height. Neighbourhood density is one over one plus the mean neighbour distance, and cross-cluster similarity is the mean of one over one plus distance across the neighbours.

A second kernel, one block per cluster and dimension-parallel, computes centroid drift as one over one plus the distance between the cluster's own centroid and the mean of its neighbours' centroids. A fused variant runs the row metrics and the drift together in one launch.

One cluster's sorted row
position 0: the cluster itself
position 1: the nearest other cluster, for the local gap and the raw distance kept as a barrier height
ranks 2 to 6: the near mean, isolation's denominator
ranks 7 to 19: the far mean, isolation's numerator
Overlap and density read the ten nearest; cross-cluster similarity reads them all. Drift needs the neighbours' centroids, so it runs in its own dimension-parallel kernel.

Member-level metrics against neighbour centroids

Relevance is the mean over members of the mean over neighbours of one over one plus distance, which says how close the cluster's members are to its neighbourhood as a whole. Boundary clarity is the fraction of members closer to their own centroid than to any neighbour's centroid, which says how many members would be reassigned if the boundary were redrawn. The kernel runs one block per cluster with its own and five neighbouring centroids, and it works in squared distances throughout.

The comparison is the same and the square root is wasted work.

Boundary clarity, drawn
Blue members sit closer to their own centroid; orange ones are closer to the neighbour's and would move if the boundary were redrawn.
Member-level kernel · milliseconds
1,000 members2.2210,00022.48100,000231.82

Topic alignment

Topic alignment asks whether a cluster sits on its neighbourhood's subject. The kernel forms the mean of the neighbouring centroids, takes the cosine between the cluster's own centroid and that mean, takes the mean cosine between each member and that mean, and folds the two into a score in the unit interval. Four variants exist: a base kernel, one that parallelises the neighbour mean across threads, a one-thread-per-cluster batch version for many small clusters, and the streaming version in use, with the cluster offsets in compressed-row form.

Phantom candidates

When a new cluster is proposed, the engine scores it against a set of phantom candidates, hypothetical centroids that a redistribution might create. Phantom proximity is the mean similarity to the k nearest phantoms, with a weighted variant that gives closer candidates more say, and a batched kernel that returns the phantom distance and spread for every cluster at once. Those scores feed the pressure model in Redistribution inside the database.

Question and answer divergence

Clusters of question-and-answer pairs carry one more outward metric: the mean over members of one minus the cosine between the question embedding and the answer embedding, halved. A cluster whose questions and answers point in different directions is one where the pairing has gone wrong. It is a streaming kernel over paired embeddings with a single block reduction and no atomics.

What this demonstrates

Cluster analytics designed around the decisions they inform. Each metric names a question about a cluster's position, the neighbour lists are computed once and shared, and the kernels are the cheapest shape that answers the question at a hundred thousand members.