Corey James- 3vector files that activate in one transaction or not at all
- 1active generation per schema and kind, held by a unique index
- 2generations kept so a rollback has somewhere to go
The GPU analytics in Where a cluster sits read three kinds of vector file: the cluster centroids, the members with their per-cluster offsets, and the phantom candidates. The member offsets index into the centroid file, and the phantoms are scored against both. A reader that saw a new centroid file beside an old member file would compute nonsense without an error. So the three files form a coupled set, and the set is the unit of activation.
Each member range sits under its own centroid.
Ranges point at the wrong centroids.
Built apart, activated together
Each kind is built by its own request, and the three requests share a coupled-set identifier. The activation function in the database checks the whole set, and a unique index guards the result.
A partial activation is impossible by construction rather than by convention.
Checked before served
A generation whose files are missing from disk must never be served. Each generation carries a filesystem status, and the check runs before activation.
Named so they cannot be swapped
File names carry the kind and the generation number explicitly, so a centroid file from one generation cannot be mistaken for a member file or for another generation's file even after it has been copied somewhere else. Row order is deterministic, cluster identifier for centroids and cluster then row for members, so two builds of the same data are the same bytes, and every shard carries a signature with a collision-resistant hash over the generation.
What this demonstrates
Treating a set of interdependent artifacts as one transactional unit, with the invariants enforced by the database and the filesystem rather than by the discipline of whoever runs the build.
The coupled-set rule and the single-active invariant were mine; AI coding agents, a mix of frontier and open-source models, implemented them under my governance system.