Idea

Append-only, with tombstones and lineage

Durable logs are only ever appended, deletion is a mark, and every derived artefact can prove where it came from

Durability principleFour systemsTombstones · generations · hashes
A segment · writes only append
faded rows are tombstoned, never rewrittenthe mark
4 bytesa tombstone at the entry's offset

When the history of what happened can be rewritten, recovery becomes archaeology. I build the durable state in my systems as append-only logs. Completion or deletion is a tombstone written in place, in a fixed order relative to the data it refers to, so that a crash between two writes is repaired by a scan rather than a guess. Every derived artefact, an index, an extraction, a projection, carries a hash or a reference back to its source, so lineage is a property of the data and not a convention.

The shape

Writes append. A segment or a table is never rewritten in place except for the tombstone mark. Derived artefacts carry the identity of their inputs and the parameters of their construction. Recovery is replay: read the log, honour the tombstones, rebuild the rest.

Garbage collection takes whole segments
deleted whole
kept: current entries
kept: a reader holds it
tombstoned
current
A derived artefact
the identity of its inputs and how it was built
its contents
✓build
✓persist
✓load
An artefact that no longer matches its source is refused.

Where it appears

In the Inference Fabric, the broker's segments are append-only on memory-mapped files, recovery reads only tombstones, and a hidden-state extraction leaves as a self-describing payload with a hash under a canonically encoded specification: Spend structure before spending waste and A detector for a silent corruption. The vector store's indexes are bound to their source data and build conditions and refused on mismatch.

In AgentOS, the work graph is written through transitions that each leave a receipt, Markdown views are regenerated projections of it, and a resumed work order is rebuilt deterministically from the stored state.

In FAFOHive, the memory capability's observations are an append-only log of what agents did and decided, and the code graph has generations so an index can be rebuilt without serving a half-built graph.

In ActiveKV, host pages carry generations and leases, so a page cannot recycle its generation while a copy still reads it, and conversion of a page to the compressed tier writes a checksum before the page is published.

What it buys and what it costs

It buys recovery you can trust and history you can reconstruct. It buys cheap deletion, one small write per item instead of a compaction. It buys the ability to check an artefact against its source years later.

It costs space until garbage collection runs, and it costs the discipline of never taking the shortcut of an in-place edit.

I pay both, because the alternative is a second bookkeeping file that can disagree with the data.

The judgment in it

The order is the whole design, and once it is fixed every recovery path becomes a scan.

The order of writes
1 · write output2 · mark entry3 · bump count4 · free input
where a crash can land: a scan of the log finishes the work