Helix

Organisations adopting AI end up with a drawer of chatbots and dashboards. Helix is one place instead. On an open-source team chat server it is a multiplayer AI workspace: several humans and several model seats in one room, each seat a frontier model working under its own authenticated session with its own identity, answering only when addressed, keeping a conversation that survives restarts. In council mode the seats deliberate, challenge one another and ratify a plan with the human as chair. The same brain answers by voice, in a Microsoft Teams call, and in a meeting as a governed digital employee with its own Microsoft 365 identity. Durable work leaves the room as governed work orders in AgentOS and comes back as results, documents and evidence.

The idea

A conversation is where decisions are made; a work order is where work is done; a brain is what carries context between the two. Helix keeps those three distinct. The chat shell is unmodified and holds no Helix state. Helix-owned state, conversations, leases, documents, grounding, lives in its own PostgreSQL schema. Model seats are vendors' authenticated command-line sessions, hosted durably so a conversation resumes across restarts and account switches. Durable work is routed to AgentOS and the memory service; the conversation itself stays in Helix. Helix consumes AgentOS and never modifies it.

Multiplayer AI on Mattermost

Each model seat is a participant with its own chat identity, connected over the server's real-time API. A seat responds only when addressed, in the right thread, with a bounded context: recent channel and thread history, attached files, and a grounding packet for the scope the conversation sits in. Turns are serialised per conversation, so two seats cannot talk over each other and there are no unbounded agent-to-agent loops by construction. Replies are attributed to the seat that made them, with progress and cancellation visible to the room. A thin plugin renders the system, project, work-order and channel hierarchy in the sidebar, so the room's structure mirrors the work graph.

Agents deliver real documents into the conversation instead of walls of text: Markdown documents created in-conversation are published as native attachments, with immutable versions, optimistic concurrency on revision, idempotent operations and an explicit pending-or-published state, so publication happens exactly once.

The council

Council mode makes several models deliberate toward an agreed plan with the human in control and no runaway loop. The operator picks two or more seats, each with its own reasoning effort and an optional persona, and a facilitator negotiates an explicit objective with the operator before anything starts. Then every seat runs at once in its own session, all reading the same room. A seat can post an interim finding or a challenge without ending its turn, any seat may object to any other, and any seat may propose a candidate outcome with a revision number. The council closes only when every seat states an exact agreement on the same revision. No chair rules, Helix cannot declare consensus or override an objection, and a budget pauses the council rather than forcing a shallow answer. I built the council four times before it behaved like a group of colleagues: a chaired floor, then a coordinator that fanned work out, then a single floor with peer challenge, each rejected after real use. The history is in Three councils I threw away, and councils have since produced real planning and costing packages.

I rejected three of my own council designs in real use, because agreement without deliberation is worthless.

Grounding and shared context

Context is engineered as an explicit, bounded, scope-correct service, never prompt stuffing. Every conversation sits at a canonical scope, system, project, work order, participant or private conversation, and a seat joining late or switching rooms receives a fixed-budget grounding packet for that scope, reconstructed for the current moment. Durable decisions are kept as typed entries at the right scope with idempotency. Shared authority is separated from each vendor's native session continuity: the authoritative thread survives any one provider's session, and a model carries its own continuity beside it. The design is one of my filed inventions, Family Three.

From the room to the work

Durable work goes to AgentOS. A conversation can open a governed work order, and the results, documents and evidence come back into the room. From Teams, work can be delegated to coding agents with tenant-correct ownership, exactly-once claims, host enrolment and on-demand cloud execution, observed and cancellable from the chat. Model acquisition is operator-gated: an agent can plan a download, but a human approves it interactively, noting the repository, revision, account and licence terms, and the registration lands in a single transaction with a checked download. There is no remote path that can approve a model.

One brain across chat, voice and Teams

Talking to Helix and typing to Helix reach the same durable brain and history. A control-only bridge keeps local voice as the authority for audio and the floor, turns finished utterances into durable Control Center turns, returns a short spoken projection for speech, and publishes the full written answer in the attached chat location. Voice can navigate locations, address seats, operate a council and delegate work. A lightweight reasoning model keeps speech fast while the heavy brain does the durable work, and identity is kept separate at every step: the operator, the acting employee, the executing seat and the voice session are distinct.

Natural voice

The voice engine is a Rust conversational runtime of about 51,000 lines with 750 tests, and most of its work is in what a listener never notices. It runs full duplex: it listens while it speaks, a person can talk over it and it stops at once with nothing stale played afterwards, and its own echo and the room's noise cannot stop it, because what counts as an interruption was measured rather than guessed. It starts preparing an answer while a person pauses and drops the preparation if they go on. A short opener arrives within a few hundred milliseconds while the detailed answer is still being produced, and the lines it says most are pre-rendered per voice and play in under a millisecond. Speech recognition runs on a GPU in tens of milliseconds, with the confidence gates that keep it from inventing words; synthesis streams from an open-weight voice model that came without streaming, with the joins made seamless; and a deterministic layer turns written answers into speakable ones while the screen keeps the written form. On a desk microphone an optional voiceprint, enrolled without keeping a recording, lets the engine listen only to its operator. Several AI voices share a room under a floor that only a person can take.

The Teams media bridge, a .NET service on Microsoft's real-time media SDK joined to the Linux voice engine by a purpose-built binary audio link, passed a thirty-minute soak and a forced mid-call reconnect: A self-hosted colleague in a Teams call. The meeting action surface lets any seat act in a meeting as one specific digital employee, with tokens minted per employee on the server: Solving the digital-employee identity and A meeting attendee that will not fake it.

Also in Helix

Media renders run on a budget-admitted local GPU studio with reconcile-never-resubmit semantics, so a render is never produced twice. Generation fencing and conversation-scoped leases make a pending intent reconcile rather than re-run after a crash. Prompt delivery to a seat is proven by an observed transcript event, never assumed. A cross-host file plane carries checked files and typed transforms between hosts.

The judgment in it

I kept the chat shell unmodified and all Helix state outside it, so the shell can be replaced and the state cannot be corrupted by it.

I kept the brain on hardware I own, which forced a cross-cloud, cross-OS bridge for Teams instead of a hosted speech service.

I gated model acquisition on a human and made the gate structural: no remote procedure can approve a download.

What it demonstrates

Multi-model orchestration with durable state, context engineering as a service, Microsoft identity and real-time media, speech AI, and a wrapper over a governance platform that turns talk into governed work.

Stories

Several minds in one room · Three councils I threw away · The same brain, typed or spoken · From a conversation to a work order · A self-hosted colleague in a Teams call · Solving the digital-employee identity · A meeting attendee that will not fake it · Interruptions that only people can make · Listen only to me · From two seconds to under one · Building the streaming my speech model did not ship · Three hundred milliseconds of hello · Saying numbers out loud · Fast transcripts that were wrong · One percent network, seventy-eight percent GPU · A tool catalogue bigger than the model · A proof that proved nothing

Technologies

Rust with Tokio and Axum · PostgreSQL · Mattermost plugin and real-time APIs · Frontier model command-line sessions · C# and .NET 8 · Microsoft Graph Communications real-time media SDK · Microsoft Graph and Entra Agent ID · Azure · Custom binary wire protocols · Real-time audio framing · the speech recognition model speech recognition · Neural speech synthesis