Technology · Discipline

Conversational turn-taking and barge-in

I built a voice engine in which several AI voices share one conversation without talking over each other, and a person speaking stops the agent.

Helix voice engineLocal voice roomMicrosoft Teams calls
A person speaks · every agent yields
Facilitator
Specialist, about to speak
A person
The moment they start the floor is theirs
1 voiceat a time; people may always cut in
  • 0stale audio after an interruption in the local voice room

A person speaking stops every agent

In Helix, a voice colleague that keeps talking when someone starts to speak is unusable, so I made interruption the first requirement of Helix's voice engine. When a person starts speaking while an agent talks, the agent stops. If several agents are speaking or about to speak, all of them stop, and nothing from before the interruption plays afterwards.

I judge an interruption by what plays afterwards.

The conversation history keeps what the room heard, so a reply cut off halfway is stored cut off and later reasoning builds on the words people actually heard.

Written by the model Stored in the history

Addressing a persona by name

In the local room, Helix acts as facilitator alongside specialist personas, each with its own voice and prompt over shared model back ends, which makes a new specialist a configuration change.

"Analyst, pull last week's numbers."
ANanswers
"We could ask the analyst later."
ANstays quiet
The same habit people use with each other.

Work handed off aloud, results at the pauses

The facilitator can bring a specialist in and hand it work aloud, and the specialist starts on it while the conversation continues.

The roomSpecialist results
results arrive at the markseach speaks in the next pause, in order
Timing is illustrative.

The room and the call share one engine

On Microsoft Teams, a participant can talk over Helix in a call and it stops. A desk microphone and a Teams call run on the same conversation engine, so behaviour proven in the room carries into calls. A self-hosted colleague in a Teams call covers the Teams side.

MICa desk microphone in the room
CALLa participant in a Teams call
one conversation engine