Technology · Model

Multilingual embedding model

the multilingual embedding model is the embedding model behind my code and reference search, served from a local text-embedding server.

Dense embedding modelText Embeddings Inference on a GPUFAFOHive and FAFO Core
One query · one dense vector
every search starts here
Embedding server · out of process
1,024dimensions in every embedding
  • 19,204documents in the code index for my monorepo
  • 294,362chunks in that index, embedded with the multilingual embedding model
  • 294,033of those chunks also covered by the code graph

I treat an embedding model as a service with a contract and a failure mode, and I engineered how the model is served, cached and degraded around that. I did not train or fine-tune the multilingual embedding model.

Serving it out of process

FAFOHive searches code, reference documents and stored observations for every agent seat in my estate, and every one of those searches starts with a dense embedding from the multilingual embedding model. The first versions tied GPU work to the life of the memory service, so I moved the model out of process, which keeps the service's lifecycle independent of the model's.

First
memory service
Second
memory service
GPU subprocess
Kept
memory service
embedding server, GPU
The first ran through PyO3; the kept server is a Text Embeddings Inference container called over HTTP through a provider abstraction that also takes CPU models and hosted APIs.

The service's own API did not change in the move.

The embedding dimension is a contract every consumer relies on, which let the serving backend change underneath without touching stored vectors or callers.

A cache before the GPU

Every search meets a cache before it meets the GPU, and embedding calls are gated only on cache misses, so cached queries never wait behind GPU work.

cache hits: answered with no GPU wait
misses embedded on the GPU
misses while saturated: lexical results
Illustrative proportions. Saturation never becomes an error for the agent; the design is on Capacity engineering, backpressure and load testing.

Batch embedding for Core and the lattice

FAFO Core and Function Forge carries an embedding function that wraps the same model for batch jobs. The first-generation distillation pipeline used the multilingual embedding model to embed its knowledge lattice, and later stages clustered and curated those vectors.

Embedded
Clustered
Curated
Distributed embedding workers wrote the lattice's vectors. In Core the same model runs as a batch function with a schema backlog, priority jobs and worker spawning. Illustrative cells.