Technology · Model

Reranker model

A cross-encoder reranker sits behind my hybrid retrieval, on a seam that tells load from breakage.

Cross-encoder rerankerLocal HTTP serviceFAFOHive search
Lexical and dense candidates, rescored as pairs

The reranker is the published model, used without training or tuning; the engineering is in where it sits and how it fails.

FAFOHive serves search to many agents at once, and the most expensive step in a search is the cross-encoder that reorders the candidates. the reranker model is that cross-encoder: it rescores the merged results of lexical and dense retrieval for code, reference and observation search. It runs as a local HTTP service. The call stays in process, with no queue hop between search and rerank, so its latency budget is predictable. The retrieval design is on Hybrid retrieval.

lexical and dense retrieval, then the response
merge
rerank
each request
automaticonoff
Illustrative timing. Each search tool sets its own default.

Busy or broken

The memory service calls the reranker through a client that I had wrapped in a bounded-endpoint primitive built for swarm-scale concurrency. The primitive sorts every failure into one of two classes, with separate counters.

Busy
the mark is the shed threshold
results
load counter
Broken
integration
results
alarm counter
Busy: the queue past its shed threshold, a permit wait out of budget, or transient errors after full-jitter retries; results come back with fusion ranking and a flag. Broken: authentication, configuration, a refused connection, TLS or a missing route; results come back with a loud warning.

The capacity design is on the Capacity engineering, backpressure and load testing page.

When the reranker saturates, search quality degrades and search availability does not.