Technology · Model

Speech synthesis model

The neural speech model that gives Helix its voice in Teams calls, a local room and produced narration.

Helix voice engineFastAPI streaming serverDedicated GPU host
Two speech backends · one interface
In processAn open voice modelONNX Runtime
Chosen, sealedthe speech synthesis modela GPU service behind a fixed seam
A reply, streamed clause by clause
speaking starts before the whole reply is rendered one voice registry per persona
  • 776 msto first audio in the first Teams call, synthesis sharing a GPU host
  • 2speech backends behind one interface, settled by a bake-off
  • ~78%of the slow turn spent in synthesis while it shared the host

A voice server behind a fixed seam

Helix speaks in Teams calls and in a local room, and I wanted the voice to run on my own GPUs with no hosted speech service in the loop. Behind one interface the voice engine can use an 82-million-parameter open voice model in process through ONNX Runtime, and the speech synthesis model as a separate GPU service. I ran a bake-off between them, the speech synthesis model won, and I sealed it so later work could not drift the voice.

the speech synthesis model is a third-party open model used as published. I wrapped it in a FastAPI streaming server in a GPU container, and the engine treats it as a remote audio component on the network. Because the seam is fixed, the server can change host or model without any change to the engine.

Clause text sent to the speech serverAudio streamed back to the listener
the reply beginsfirst audiothe reply ends
Conditioning is cached per voice, so each clause pays only for its own synthesis. Timing is illustrative.

Reading the shape of a latency curve

Helix had to answer fast enough to feel like a colleague. In local testing before streaming, first audio took the same time whether a clause was short or long. A flat line across lengths is a fixed server overhead, not synthesis, so the server was reworked to stream instead of being given a faster model.

Time to first audio, before streaming
Short clause~1.6 sLong clause~1.6 s

Moving synthesis off the shared host

In the first Teams call, speech synthesis shared one GPU host with everything else, and it dominated the slow turn. The speech services went to a dedicated NVIDIA Blackwell GPU host. The call is told in A self-hosted colleague in a Teams call.

One slow turn in the first Teams call
Speech synthesis
Everything else in the turn

When synthesis dominated a turn on a shared host, I fixed placement and kept the model.

Narration in the same voices

Helix's media tools let agents produce voice-over in the approved voices. One voice registry maps each persona to its voice for spoken conversation and for produced narration, so both sound like the same persona. A narration renders through the same endpoint without disturbing the voice service.