Engineering story · Helix

One percent network, seventy-eight percent GPU

A slow Teams turn was blamed on the cloud hop; measurement showed the network was about one percent of it and a speech GPU shared with an embedding service was most of the rest, so the compute moved and the cloud stayed

Teams media bridge on AzurePer-stage turn timingDedicated speech GPU
The Teams voice path · suspect and cause
Teams call
Azure bridge: the suspect
Voice engine at home
Speech GPU: the cause
6 send of a question to first sound, on the slow turn
  • 65 msround trip between the cloud bridge and the home lab
  • 1.317 send of speech to first reply audio on a responsive Teams turn
  • 100%speech GPU utilisation at peak in the slow window, never below 95
  • 5,859frames validated in a closed-loop call, zero rejects

The Teams voice path crosses a cloud: audio leaves Microsoft's media platform, reaches a Windows bridge on an Azure virtual machine, and crosses an authenticated WebSocket to the voice engine on my own hardware, then back. When a Teams turn took about six seconds from the end of a question to the first sound of the answer, the cloud hop was the obvious suspect.

The obvious suspect was wrong.

Measuring the transport

The transport was measured on its own first.

Milliseconds
WebSocket setup, once per session165–218Session open104–115Typed event and ack76Audio frame and ack64–67One-way event38Opener's first sentence273

The bridge's own numbers were sound. A thirty-minute soak carried 91,643 receive buffers at 50.001 per second with zero stalls, one send stall and no reconnects. A forced reconnect mid-call resumed the same session after one 250-millisecond retry with both call legs alive, and the 36 audio callbacks that landed during the break were dropped rather than replayed, so the agent never answered a question from before the gap.

None of that explains six seconds.

Where the six seconds went

The slow turn was decomposed with the same per-stage discipline as the local chain in From two seconds to under one. Recognition took 253 milliseconds. The synthesiser took 4.847 seconds from receiving its text to its first frame of audio, and inside that were two serialised waits of 12.4 and 4.1 seconds across clauses.

The slow Teams turn · by cause
the network, about 1 percent
the contended speech GPU, about 78 percent
everything else in the turn

What moved

Speech synthesis moved to a GPU host of its own, and the cloud bridge stayed where it was.

Before · one shared GPU
Speech and embeddings
After · a dedicated host
Speech only
Cloud bridge
unchanged
Orange cells are the embedding service's share.

That decision would have gone the other way without the measurement: a faster network path would have bought one percent, and an engineer who trusted the obvious story would have spent a week on it.

What this demonstrates

I resist the obvious attribution: the cloud hop was measured, found cheap and left alone, and the real cost was fixed by moving one service.

The decision to measure before moving anything was mine; AI coding agents, a mix of frontier and open-source models, ran the benchmark and the move under my governance system.