Corey James- 88remote tools offered to the voice brain
- 300provider operations kept out of voice by configuration
The voice engine reaches the world through tools: native ones for assigning work, saving and searching memory, and setting the voiceprint policy, and remote ones over the tool protocol, including the meeting-action server that lets it join a Teams meeting, raise a hand or put a deck on screen. One day the assistant began speaking its short opener and then falling silent where the full answer should have been.
The arithmetic
The tool list alone was larger than the model's entire 32,768-token window, so every full request was refused by the serving layer, while the tool-free opener sailed through and made the failure look like a stall.
The detour
The first fix was a two-stage design: ask the model to pick which tools it needed, then make the real request with only those.
It worked and it was unusable.
The design was rolled back to the previous backend the same day.
The fix
Tools are scoped to the surface, because a spoken assistant does not need every provider operation in its working memory to answer a question in a meeting.
the general tool estate
The engine also repairs tool arguments the model gets slightly wrong, renaming known-wrong keys and logging the rename, because a small model will write a variable name where a value belongs. Identity for delegated work comes from the credential the engine holds, never from a spoken tenant or id, and receipts are idempotent so a spoken retry after a lost reply does not run the work twice.
The permanent answer to the same problem, for every agent rather than one voice, is the tool-discovery model in A hundred tokens instead of forty-two thousand, which compiles the catalogue into a small model so no agent carries the schemas at all.
What this demonstrates
I diagnosed the failure from the one lane that worked, rejected a plausible fix on its measured cost, and drew the boundary where it belongs: each surface gets the tools its job needs.
The rollback and the scoping decision were mine; AI coding agents, a mix of frontier and open-source models, implemented and measured both under my governance system.