Corey JamesCancellation and lock order for the GPU conductor
FAFO Core and Function Forge accepts commands such as pause, scale and drain while jobs run on GPUs, and I knew that combination breeds stuck workers and lock-order deadlocks. I built the conductor on Tokio with two rules.
Each worker handle aborts its task when dropped, so a worker cancelled mid-flight releases its GPU memory without every code path having to clean up. The conductor's locks follow an acquisition order written in the code beside them.
Database notifications as an event stream
Agent Swarm needed agent actions to become searchable observations without a polling loop. I wrote a future that pumps PostgreSQL LISTEN/NOTIFY notifications from a dedicated TLS connection and feeds them to the promotion worker.
Every service crate runs a multi-threaded Tokio runtime. The agentic loop applies a timeout and retry policy per tool step, the message router has a backpressure drainer, and the registry's heartbeat and reaper loops fall back from push to polling when queues saturate. See Tools are the platform.
Voice and Teams on Tokio and Axum
Helix builds its voice engine and control center on Tokio and Axum. The Teams endpoint is an Axum WebSocket server that admits one session per call and feeds the same audio interfaces the local microphone uses. Cancellation is structured so a failure in one part of a conversation cannot strand the rest. See A self-hosted colleague in a Teams call.
One binary, several surfaces
FAFOHive is one Tokio binary that serves several surfaces at once over one database and one cache. AgentOS runs its member runtime, the service between public tools and agent containers, as a Unix-socket server that fans requests out with join sets.
Where I left Tokio out
Inference Fabric is where I left Tokio out. The broker's dispatch path is a single-owner actor on blocking threads with lock-free channels, because task scheduling overhead there is paid on every item. Tokio serves the broker's network edges. Tokio is the open-source runtime; the concurrency design on top is mine.
I use an async runtime as an architectural tool: I design cancellation, backpressure and lock order up front, and I know where to leave it out.