Corey James- 8synthesisers in the bake-off
- 12tokens before the first chunk is released
- 56voices registered, each persona with its own
The voice model was chosen in a bake-off on the criteria that matter in a conversation: time to first audio, real-time factor, memory, licence, voice cloning and streaming. The winner carried a permissive licence.
One law came out of the bake-off and stands over every voice on the site: never clone a real human voice without documented consent.
The winner had one problem. Its released package produced audio in a batch: about 1.6 seconds to the first sound regardless of how short the sentence was. That is a fixed tax on every reply, and the package exposed no way to stream.
Porting the loop
The model generates speech tokens with a transformer and turns them into audio with a diffusion decoder and a vocoder. The released code ran the token loop to completion and decoded once. The streaming server ports the token loop so tokens are consumed as they are produced. The decoder's encoder is bidirectional, so each chunk is decoded with the full accumulated context rather than in isolation. The vocoder is kept continuous across chunks with a left warm-up overlap and a right margin held back until the next chunk arrives.
Pinning the noise
A diffusion decoder starts from random noise, and re-rendering a region with fresh noise produces audio that is different enough to click at the join. The server pins the noise: the same noise is reused for the re-rendered frames, so the overlap renders identically and the joins are seamless.
A small patch over the sampler is the difference between streaming that works and streaming that crackles.
the re-rendered overlap comes out different: a click at the join
the same noise for the re-rendered frames: the overlap matches and the join is silent
The rest of the path
First audio dropped to between 150 and 350 milliseconds warm, measured at the client as a first frame of 306 to 318 milliseconds. The client reads chunked audio per clause over a socket with the delay algorithm turned off, after a reader rewrite removed a 1.5-second regression, and a permit semaphore lets two clauses be synthesised ahead of playback. The server cancels cooperatively when a person interrupts, with checkpoints inside each request and a lock that cannot be won after cancellation, so an interrupted sentence stops in the synthesiser and not only at the speaker. The server preloads its compiled kernels for the Blackwell architecture.
Speech synthesis then moved to a dedicated GPU host, after a measurement showed that sharing a card with an embedding service was the cause of slow Teams turns, in One percent network, seventy-eight percent GPU.
What this demonstrates
I can take an open-weight model past what its authors released, by reading the architecture rather than the API: the token loop, the decoder's context and the sampler's noise each had to be understood before streaming could be made both fast and clean.
The choice of model, the consent law and the decision to build streaming were mine; AI coding agents, a mix of frontier and open-source models, implemented the server under my governance system.