Engineering story · FAFOHive

Two thousand concurrent workers

Load-testing the memory and code-intelligence service with a fleet of AI agents on one workstation, with zero failures

Rust over PostgreSQLOne 24-core workstation256 GB DDR5
Two thousand agents · one workstation · one test window
30 swarm-up
60 s measuredevery request counted
30 scool-down
each tile a slice of the fleet calling search, trace, list and write at once
1,479peak requests a second, observation search
  • 2,000concurrent AI workers on the retrieval tools at once
  • 0failures across the run
  • 1,479requests a second held in keyword mode, from a floor of 1,271

Every agent in my estate asks the memory service before it acts: what did we decide, what calls this symbol, how does this path work. If that service stalls under a fleet, every agent stalls with it. So I load-tested it the way the fleet uses it: two thousand concurrent AI workers hammering the retrieval tools at once, on a single workstation. Nothing failed.

The fixture

One workstation: a 24-core Intel Core Ultra with 256 GB of DDR5 and a consumer Blackwell GPU that was not used to accelerate retrieval. The service is Rust over PostgreSQL.

Concurrent workers per tool
Observation search, tracing, index listing, observation writes, code search2,000 Reference search1,500 Path explainer100
The path explainer calls a model and is the most expensive tool by design.

The numbers

Peak requests per second · concurrent workers · p50 and p99
Observation search2,000 · 1.34 s · 1.90 s1,479 Dependency trace2,000 · 1.65 s · 2.37 s1,191 Code search2,000 · 19.5 s · 40.4 s98 Path explanation100 · 8.79 s11
Hybrid search held a sustained rate at 1,200 concurrent workers. Observation search and code search ran in keyword mode.

Semantic search ran at about 1.2 seconds median across a working codebase.

The slow row is honest: code search over a large index at two thousand concurrent callers is bound by the database, and the numbers say so.

Why it held

Every endpoint is bounded. The reranker saturates rather than errors under load. Observation writes go through a dedicated connection pool so that capturing what an agent did never competes with answering what it asked. The caching layer answers repeat questions in under 100 milliseconds, and in a fleet most questions repeat. The code graph itself is precomputed, so a dependency trace is a bounded traversal rather than a model call.

What this demonstrates

Capacity engineering for the workload I actually run: a fleet of agents, not a handful of users. The service was designed with backpressure in every path and then tested against the fleet it serves, with the weak row published next to the strong ones.