Corey James- 15MCP actions on the node-management tool
- 17,303lines of Rust in the provisioning substrate, across 26 files
- 3providers behind one trait: local, LAN and cloud
One call, wherever the GPU is
FAFO Core and Function Forge needed compute from wherever I had it: the workstation, a spare machine on the LAN, or a rented cloud GPU up to datacentre Blackwell. I made location a parameter of one MCP call.
Underneath is one Rust provider trait, implemented for local Docker, LAN Docker daemons reached over mutual TLS, and RunPod for cloud pods. I kept one implementation of ports, health, events and spend, and every family of workloads composes it.
I chose plain Docker with a provider trait and my own CA over Kubernetes.
The estate is a handful of machines, and the scheduler I needed was a budget.
A budget gate before anything exists
The dollar budget sits inside the provisioning call and only a person can set it. I put the budget gate ahead of the port lease, the container and any cloud request, so a refusal leaves nothing behind.
I bill cloud pods from the moment the provider confirms creation, because the provider bills from power-on, and terminating a node returns its final cost. One cloud provider is wired in, and pre-empted spot nodes are cleaned up without automatic replacement.
Images that carry no function
Local and LAN containers never carry a function in their image. I injected the runtime into generic images, which took an image build out of every function change. Cloud images bake in an outbound tunnel and a heartbeat instead.
mounted at launch; a function change rebuilds one binary
baked in; the manifest lists digest, CUDA version, GPU families and environment
The pod dials back to the private network, so reaching it opens no inbound port and no GPU host needs one (zero-inbound exposure). The Blackwell cloud image covers consumer and datacentre Blackwell parts, so one call can ask for datacentre capacity with the matching image.
GPU consumers for the Inference Fabric
A loader mode decides what a node becomes: a one-off job, a long-running inference consumer, a hardened agent container with every Linux capability dropped and privilege escalation off, or a stock infrastructure service such as Redis or a credential broker. Inference Fabric gets its GPU consumers this way.
The node-management tool's MCP (Model Context Protocol) actions cover provision and terminate, status, drain, idle timeouts, LAN host onboarding and teardown, engine sync, and image and engine listings. Provisioning reads the TensorRT-LLM engine's configuration and publishes its token capacity for the batch planners.
Onboarding a bare machine in one call
Onboarding a bare LAN machine is a single call, and the node-management tool provisions member hosts for the agent runtime the same way.
- Probe prerequisites
- Ensure the CA
- Issue certificates
- Copy them to the host
- Docker over TLS
- Handshake
- Inventory GPUs
- Stage the engine
- Commit to the registry
- Start exporters
- Final check
A reconcile loop keeps the cache honest against Docker.
Each turn matches the cache against Docker; a container that misses its lease gives back its ports and scrape targets.
I build the infrastructure layer once and let every other system inherit it. Agent Swarm launches persona containers through the same path in agent mode, and Helix media services start GPU render instances behind the same budget.