Corey James- 13meeting actions, each with an exact schema
- 1action that refuses by design: raising its own hand
- 55 of 55unit tests passing before a real meeting had proved the tools
I wanted the digital employee to be a useful participant in a Teams meeting: follow the chat, put slides on the stage and drive them, notice who has a hand up and call on them, share a terminal or a browser so people can watch it work. It does all of that. It does not raise its own hand, because Microsoft's platform offers no supported way for it to, and I built that as an honest refusal instead of reporting a hand raised from local state.
The challenge
Meeting participation is many small capabilities, each with its own platform seam, and some of those seams do not exist. Native PowerPoint Live exposes no control or event hooks. The hand-raise state is visible to a bot and not writable. The agent's pixels originate in a Linux container while publication to Teams goes through a Windows media bot. And AI-assisted engineering teams are very good at producing plausible claims that something works, which is the wrong property when tests in the real tenant are scarce.
Constraints
No supported path means no feature. Video had to cross two operating systems and a vendor platform with a single typed frame contract. And every test against the real tenant had to count.
My approach
A strict evidence vocabulary: implemented is not wired at runtime is not proven in a real meeting. Each action moved through those states with its own evidence, and nothing was called done until a real meeting had proved it, however many unit tests passed. Where the platform had no supported path, the action failed closed with a named reason.
What I built
Meeting actions with exact per-action schemas, served from a Rust MCP server bound to the employee identity.
A room-state projector that gives the agent a current picture of who is present and who has a hand up, and a chat adapter bound to the employee so its messages are its own.
A Teams meeting-stage app plus an authenticated relay, so the agent drives a deck on the stage itself. This replaced the PowerPoint Live route, which had no seam to drive.
A hand coordinator that returns a typed "hand mutation unavailable", because the platform does not support it.
A headless desktop as a video source: a virtual display, window manager, browser and terminal, captured by FFmpeg and framed as H.264 to the media bridge, published into the meeting as screen share. Terminal content is typed at a human-readable speed so watchers can follow it.
Acceptance scripts that clean up after themselves even on failure.
Difficult decisions
Honest refusals over local state. The agent could have tracked a "raised" flag locally and reported it. Nobody in the meeting would have seen a hand.
Pivoting presentation. PowerPoint Live was the obvious target and had no control surface. Moving to an installed meeting-stage app cost a rewrite and gave the agent a deck it can drive.
Not done at 55 of 55. The second-generation tools passed every unit test and every stage test. I did not call them done until each one had worked in a real meeting.
Tests that pass are not a meeting that worked.
Results
Technologies used
Rust MCP server · Microsoft Graph · TeamsJS meeting-stage app · Node.js relay · Xvfb, Openbox and Chromium · FFmpeg H.264 framing · C# screen-share publication · Shell acceptance harnesses
What this demonstrates
I build systems that report their limits as accurately as their capabilities, and I can carry video end to end across Linux, Windows and a vendor meeting platform with one typed contract.
I set the evidence vocabulary and the fail-closed rule and made the presentation pivot; AI coding agents, a mix of frontier and open-source models, implemented the actions and harnesses under my direction. The meeting platform is Microsoft's.