Technology · Discipline

Multi-vendor LLM integration

Three frontier model families and local open-weight models run side by side in my programs, through subscription sessions, API seats and my own GPUs, with a terms-of-service firewall deciding which traffic may go where.

Auth and access plane and FAFOHiveOAuth client profilesOne provider interface
Many assistants, one way in
each difference held in a client profileplus an edge proxy
One authorization flow into one private tool estate, the same for every vendor
  • 30 daysa temporary client stays pending, because some assistants never register again
  • 5language-model providers behind one interface
  • 3frontier model families whose tools and apps come in through one flow

One way in for every assistant

Auth and access plane had to let several vendors' assistants into one private tool estate: the command-line tools and desktop apps of three frontier model families, an agentic IDE and Microsoft Copilot Studio. Each behaves differently within what the standards allow.

I decided to keep one authorization flow and move every vendor difference into a client profile held in configuration.

Frontier command-line tools
Frontier desktop apps
Agentic IDE
Copilot Studio
No remote MCP
client profile
edge proxy
adapter
one authorization flow
The differences absorbed: discovery read in a different order, a client ID cached for good, a public client with PKCE and no secret, renamed parameters and uncredentialed probes.

I set the pending-client lifetime from how real assistants behave, after watching clients cache an ID and never return. See The hardening fix that locked everyone out.

One provider interface for memory work

FAFOHive needs a language model for background work: turning session activity into observations, writing summaries and narrating proven code paths. I did not want that work tied to one vendor, so I wrote one provider interface, and the memory pipeline stays vendor-neutral behind it.

One long input, split to each provider's limit · illustrative
Frontier family one, command line
tools offFrontier family two, REST
A compatible message API
OpenRouter, open models
Any OpenAI-compatible local server
replaces all

Every response carries token counts, so I can weigh what memory work costs against the context it saves. In the code explainer, any provider may narrate only edges the code graph has proven; see A code explainer that is not allowed to invent.

Retries by kind of error

One retry policy wraps every provider.

Transport error
Rate limit
Server error
Authentication
Bad request
Parse error
An attempt; gaps grow by exponential backoff
Returned at once
A bad credential surfaces at once and burns no retries.

The models and assistants are the vendors' products, used through their public interfaces; the profiles, adapters, retry and splitting logic and the trust model around them are mine, and the system stays free of their quirks.