Corey James- 38raw metrics per cluster, a diagnosis rather than a decision
- 2,000members, above which the spectral metrics are left null
- 1 rowper composite and metric holds every weight
Thirty-eight metrics per cluster is a diagnosis. The decisions, split this cluster, merge those two, leave that one alone, are made inside the database by a pressure model, and the pressure model reads five composite scores rather than thirty-eight raw values. This is how the composites are built and why their weights are data rather than code.
Five composites
Each composite is a weighted average over a named subset of the metrics, and each subset answers one question about the cluster.
Skip-null averaging
A metric can be missing for a cluster: the spectral metrics above 2,000 members, a question-and-answer metric on a cluster with no pairs, a neighbourhood metric on a cluster with no neighbours yet. The averaging skips nulls and renormalises over the weights that are present, and the metric count that went into a composite is stored beside it.
Each gap counts as a failure, so the cluster is marked down for what was never measured.
A composite over the seven present metrics is a composite over those seven, with the count kept beside it.
A composite that treated a missing value as zero would punish the cluster for the gap.
Weights in a table
The weights are authoritative in a PostgreSQL table, one row per composite and metric. The CPU reference reads them, the SQL that computes the composites in the database reads them, and changing a weight is a data change with an audit row rather than a release. The reason is practical: the right balance between structural and semantic signals is something I tune by looking at a corpus, and I wanted to tune it without rebuilding anything.
Operational metrics beside the geometry
The operational metrics are about the cluster's life in the database rather than its geometry: membership stability, processing success rate, data completeness, failed and stale member counts, the ratio of members with no embedding, and hours since the last activity. They sit in the same row as the kernel outputs so a composite can weigh a cluster's shape and its health together.
From score to action
The pressure model in Redistribution inside the database converts the composites and a few raw metrics into split, merge and hold pressures, and the typed Rust functions inside the database act on them. Every step is reproducible from the definition of the metric upward.
What this demonstrates
Closing the loop from measurement to action with each layer kept honest: metrics defined by reference code, composites that skip what is missing instead of guessing, and weights held as audited data.
The composite design and the rule that weights are data were my decisions; the implementation was done by AI coding agents, a mix of frontier and open-source models, under my governance system.