Research data forensics

Two incomplete witnesses

Research and inventions mentions the curated corpus of golden examples from my first distillation pipeline. When I came back to it, the corpus was split between a PostgreSQL database that had kept changing and a SQLite backup. Neither was complete, and no notes said how its cluster centroids had been computed. I decided to rebuild it from both sources and reproduce its derived data exactly before trusting any of it.

I directed the recovery in phases, each closed by a written report. I treated the database and the backup as two independent witnesses: they were compared entry by entry, every gap in one was explained from the other, and each filled the other's gaps.

PostgreSQL, kept changing

SQLite backup

Rebuilt, one schema

Reproducing the derived data

The centroid formula was reverse-engineered from the stored centroids, so the recovered data is derived the same way the original was.

Stored centroids and recomputed ones

stored

recomputed with the recovered formula

Where it went

BeforeA set of patched tables

AfterOne schemaquestion-and-answer pairs attached to their passages

Checking that the two views agree

The last phase measured alignment between the two representations the corpus carries. The audit also found a small number of phantom clusters with no surviving source text, and I had them set aside, even though that lowered the count.

Two views of each cluster

medoid of a passage cluster

mean embedding of its question-and-answer pairs

Two views of the same data that disagree mislead every model trained on them.