Technology · Platform

ComfyUI

A node-based media pipeline runs as a GPU media studio behind my memory gateway, with provenance tracked on every generated asset.

FAFOHive media studioGPU render engine
Agents never touch the engine · the controller decides
Rust controllertyped choices only; no Docker options, hosts, mounts or shell strings
GPU by UUID · admitted
ComfyUI, unmodified, from an approved profile
GPU by UUID · other work
compute budget checked before every start
  • 33media tools any agent can call
  • 1CPU-only controller container per host
  • 9frames in the video workflows that render and deliver end to end

A controller in front of an unmodified engine

FAFOHive hosts a GPU media studio as tools any agent can call, and ComfyUI is its render engine. The risk I saw was agents treating a GPU and a third-party HTTP engine as free and repeatable: a retried render costs GPU time, a crash mid-render leaves an unknown outcome, and an agent-supplied Docker or shell option is an escape hatch.

I decided to leave ComfyUI unmodified and put all of the control in a Rust controller in front of it. ComfyUI's own APIs do the rendering; I built no engine and changed none of its code.

What an agent might try to sendDocker options, hosts, mounts, shell, codec strings
What the controller acceptstyped choices and validated workflow graphs
What ComfyUI receivesits own API, unmodified
Around it the controller keeps node-catalog snapshots flagged when stale, streams job events by long-poll, cancels single jobs and moves finished files by host and path with a hash check.

A render paid for once

A render's identity is written down before ComfyUI sees the request, so a crash never pays for the same render twice.

A render interrupted by a crash · illustrative
identity
persisted first
sent
crash
reconcile
against ComfyUI
registered
immutable
delivered
The caller retries delivery
retry
same file
no second render
On restart the controller reattaches observers to jobs in flight.

Image workflows and video workflows render and deliver end to end through this path.

A person at the licence boundary

Approving a model licence takes a person at a console, and no remote call can do it.

download plan
exact
access
checked
OPapproved
at a local console only
download
resumable, hash-checked
registered
automatically

Inventory reports what is there: a host with zero models reports zero, and an unreachable host is reported as unreachable.

When agents are the callers, I wrap third-party engines in idempotent, crash-safe control and place the human decision at the licence boundary.