The copy of record lives off the accelerator

The expensive resource in every system I build, GPU memory or a model's context window, is also the least durable and the hardest to reason about. So I never let it hold the truth. The copy of record lives somewhere cheap, durable and inspectable: pinned host memory, a PostgreSQL work graph, an append-only log, a parsed code graph. The accelerator holds a bounded working set that is staged in, used, and written back under an explicit budget.

The shape

The authoritative state is written where it can be large and survive restarts. The working set is sized by a stated budget, filled from the copy of record on demand, and released on a fence or a step boundary.

Staging buffers, reused in a ring

The rules

in use for the current step

finished, but a copy or kernel still reads it: not reusable yet

released on a fence or a step boundary, free to fill

Where it appears

In ActiveKV, the inversion is the story of Context beyond VRAM, and the staging lifecycle that makes it safe is Staging without a stall.

In the Inference Fabric the split holds twice, for the broker's backlog and for the vector store, whose GPU-resident indexes are derived serving artefacts that must prove their lineage.

In AgentOS, a job is rendered with exactly the state it needs, and after a crash or a compaction the working picture is rebuilt from the graph: Why long agent runs fail.

In FAFOHive, the memory capability follows the same split.

The whole bar is the copy of record; the filled part is what the accelerator holds

ActiveKV

a per-layer staging set on the GPUpinned host RAM: every layer's attention history

Inference Fabric · broker

bounded batches the GPUs pullmemory-mapped segments on NVMe: billions of tokens

Inference Fabric · vector store

indexes resident on the GPUthe relational database, the only authority

AgentOS

the model's context, rendered at dispatchthe work graph

FAFOHive

a bounded, computed slice in contextthe parsed code graph and the observation store

What it buys and what it costs

It buys capacity beyond the accelerator, durability across restarts, and a place to inspect the truth that no model has edited. It buys swappable accelerators and swappable models, because the working set is derived.

It costs the staging path, which is engineering in its own right, and it costs per-step bandwidth. ActiveKV pays the bus on every token at long contexts, which is why the compressed tier and the exact merge exist.

The judgment in it

The decision is where to draw the budget line and what to do when it is crossed. I draw it explicitly, in whole units an operator chose, and when it is crossed I refuse or shrink with a reason.

An automatic size hides the operator's choice, and a silent degradation hides the failure.