Technology · Discipline

Budget enforcement and cost governance

Spend is a first-class control in my substrate: a fail-closed daily budget on GPU provisioning that agents cannot raise from inside the provisioning path, per-second accounting, and cost attributed to the work order that spent it.

Redis and Lua accountingPostgreSQL write-behindCloud GPU provisioning
Today's spend against today's cap
set by hand
Rented GPU
billing
  • $0the cap on any day with no budget set
  • 1/100 ¢the precision of per-second spend accounting
  • 10 sbetween updates of per-container, per-schema and global spend
  • 30 sbetween health passes that stop paid containers at the cap

I build spend control into the path that spends, and I keep its key in human hands. Rented capacity and metered APIs get hard gates; model usage inside governed work gets attribution.

A daily cap inside the provisioning path

Agents on FAFO Core and Function Forge can stand up GPU capacity, and a rented GPU bills from power-on whether anyone remembers it or not. I decided the spending limit would live inside the provisioning path, and that its only key would be a person at a terminal.

The limit is a daily dollar cap on the operator's calendar. A day runs from local midnight to local midnight in a timezone stored once, so every host computes the same day. Budgets are set per date or per date range, and every change lands in an audit table.

budget
budget
unset: nothing launches
budget
budget
one container across midnight, its cost split between two days
Budgets never carry over, and an ambiguous midnight at a daylight-saving change is refused.

The cap is set with a command-line tool that only a person runs. That tool is the only code that writes budgets. No MCP tool on any gateway changes them, and nothing an agent passes to a provisioning call moves the cap.

The launch check

Every container launch passes the check before a port is leased or a cloud call is made, so a refusal leaves nothing to clean up and setting the budget is the daily switch for all automated provisioning. Work that runs in-process never touches the gate.

spend below the cap
local or LAN, budget set
cap reached
cap is zero
budget unreadable
launched, paid
launched, no spend
denied before any port or cloud call
The mark is the check.

Inference Fabric GPU consumers, Agent Swarm persona containers and Helix render instances all pass this same gate. The persona launcher and the media controller treat any budget error as a refusal and spawn nothing. Admission compares actual spend with the cap, and the termination pass bounds any overshoot.

I kept projections out of the check because an exact rule is easier to trust.

Spend counted per second

Spend is accounted per second, starting the moment the cloud API confirms creation, because that is how the invoice is computed.

spend, per second
Redis loop
write-behind
Lua recompute
health pass
the cloud API confirms creationthe day reaches its cap; the container is stopped
Illustrative timeline. A finalized container cannot be reopened by a stale writer, and lowering the cap never erases spend already counted.

Spend is reported where it is spent. Terminating a node returns its final cost. The budget tool's status view shows the day's cap, spend and remaining budget, paid containers with their hourly rates, time to exhaustion, the next week's budgets and recent changes. Network-volume storage is synced from the provider and held to a monthly soft cap that only warns, because enforcing it would delete pre-warmed models.

A metered API with its own budget

FAFOHive's in-house X module spends a metered data API. It keeps a monthly unit budget and reserves units before each call, reconciling the reservations against the vendor's own usage figures.

The mark warns at 80 percent; the end of the bar refuses at 100.

Every result reports the units, requests, cache hits and rate-limit wait it cost.

Cost as telemetry in governed work

AgentOS attributes model usage to the work that spent it, taken from session transcripts and signed seat events. I ruled that cost inside governed work is telemetry and strategy, and removed a per-dispatch budget gate on quality-assurance work. The attribution design is on the Cost attribution for AI work page, and the provisioning path is on the Federated compute provisioning page.

uncached input
output
cache reads
cache writes
priced
unpriced, kept
Illustrative proportions. Priced against a public model-price catalog with long-context tiers; usage with no price is kept unpriced. Attributed to the work order, role, persona, runtime and model.