Engineering story · Helix

Three hundred milliseconds of hello

The lines an assistant says most, the greeting, the thinking fillers, the short acknowledgements, are pre-rendered per voice at session start and play in under a millisecond, through the same cancellation path as synthesised speech

Rust voice enginePer-voice audio cacheStreaming synthesiser
The lines said most · rendered once per voice at session start
hello
one moment
let me check
filler
acknowledgement
joining
First frame of audio
synthesised from the cache
0.06 msfirst frame of a cached line
  • ≈8 sadded to session boot to fill the cache
  • 5.3 sto condition a voice the first time, nothing after
  • 8voices held in the bounded conditioning cache

The streaming synthesiser in Building the streaming my speech model did not ship produces its first frame of audio about 300 milliseconds after it is given a sentence, and that figure is flat: eleven characters or two hundred and thirty, the first frame costs the same. For a long answer it is a fine price. For "Hello", "One moment" and "Let me check", which a voice assistant says many times an hour, it is a tax on the words that most need to be instant.

Sentence length · characters
"Hello"-sized11A long answer230
First frame of audio
"Hello"-sized≈300 msA long answer≈300 ms

The opener cache

At the start of a session the engine renders each persona's greeting, its thinking fillers and its short acknowledgements through the streaming synthesiser and keeps the audio in memory, keyed by voice and exact text. The first line of a session is synthesised on the spot, so the session never waits silent for the cache to fill.

A guard test makes sure the boot order cannot leave the cache half-filled.

Rules cached audio obeys, like synthesised audio
  • Turn idssame
  • Flush when a person interruptssame
  • Ring to the speakersame
  • Empty cachepass-through
One ring to the speaker a cached greeting's frames, queued where synthesised frames go

Cached audio has to obey every rule synthesised audio obeys.

The other caches

The synthesiser conditions each voice from a reference recording, and that conditioning is cached too, so a room with many personas pays the cost once per voice and an evicted voice pays it again.

Voice conditioning · boundeda ninth voice evicts one; an evicted voice pays conditioning again
Clauses ahead
Presence lines are cached per session; thinking fillers rotate and can be tuned without a restart.

What this demonstrates

I know which words carry the latency. The cache is small and the mechanism is ordinary; the judgment was in measuring that the first frame is flat and noticing which lines are repeated.

The cache design and the rule that it shares the synthesis path were mine; AI coding agents, a mix of frontier and open-source models, implemented it under my governance system.