Corey James- 16cells in the frozen workload matrix
- 196 → 0deadlocks in one bulk update after a unique index
- 7Bdense model in FP8, held fixed across every step
A measurement doctrine for ActiveKV
In ActiveKV one metric, inter-token latency, kept declaring success or failure on its own and misled everyone, me included. I wrote a measurement doctrine to fix that, and comparison tables are generated from run data.
The workloads are a frozen matrix. Every performance change after the native port went through an intake screen that rejects behaviour keyed to benchmark fixture sizes, even when the gain is real at the fixture, then an A/B against the last kept change, and a cumulative ledger of engine hashes and runs. When a trace said host-to-device copies serialised with compute, a controlled re-measurement showed the timing events were causing it, and I retracted the conclusion: The observer that changed the answer.
I design the instrument, the baseline and the reading rules as carefully as the system under test, and I retract my own conclusions when the instrument turns out to be the cause.
One layer at a time on the Inference Fabric
In Inference Fabric I kept the model fixed and changed one layer at a time with a dashboard capture at each step. An engine tier sweep found the per-step token budget to be the biggest engine lever. Then I decoupled producers, added a shape-aware broker, made dispatch cache-coherent and put a second GPU on the queue. Three ideas lost on measurement, and I kept them beside the wins. See From 44K to 163K tokens per second.
Measuring writes where they commit
In FAFO Core and Function Forge a bulk update through the shared write path crawled and deadlocked while the queue's counters looked healthy. I measure from the system that commits the work: rows from the database, tokens from the engine. So I read PostgreSQL's own tuple counts, sampled every few seconds with host and Redis load, and compared runs around one schema change. One unique index ended the deadlocks: 196 deadlocks to zero.
Concurrency as a requirement
In the Capabilities model I made concurrency a requirement and drove it with a load ladder of concurrent clients. The full burst was served beside the main model. The sustained target I set in advance for mixed traffic was missed, for a memory reason I explain in A router beside a production model. Every figure I publish carries its hardware, model and window.