Technology · Discipline

Provider-terms compliance in model routing

A terms-of-service firewall in the agent runtime decides which traffic may reach which provider, so a subscription session is used as its vendor permits.

Agent Swarm model routerLocal GPUs and hosted providersOne endpoint view, no second registry
Request · automated back-end work · persona
load on the endpointdimmed: filtered out for this request

How an automatic router breaks terms by accident

In Agent Swarm a router picks a model endpoint for every agent request, from my own GPUs or from hosted providers, and some providers' terms forbid automated, unattended back-end use of their models. In an automatic router that rule is easy to break by accident.

One persona's model list, top to bottom · a background job arrives
preferred · busy
fallback · busy
fallback · an endpoint registered for another purpose
fallback
Illustrative. Without a terms rule in the router, the job takes the first free endpoint, here one whose provider forbids unattended back-end use.

I made provider terms a routing constraint in code, so compliance does not depend on whoever configures the next endpoint. The rule separates foreground use for a person from automated back-end use, because the terms draw that line.

How the router narrows

The router works over an endpoint pool that is a view joining the GPU inference nodes with the registered provider bridges, so each endpoint exists in one place and there is no second registry to fall out of sync. A request carries its role, foreground work for a person or automated back-end work, and the persona making it.

One back-end request for a persona
endpoint pool · GPU nodes and provider bridges
the persona's preferred, then fallback classes
endpoints this persona may use
the terms rule for this role
least loaded
Endpoint counts are illustrative. When no dot survives, the router returns an explicit "no eligible endpoint" result instead of a quiet guess.

Checked twice

The terms rule runs twice: at registration and on every routing decision.

At registration
Local GPU node as back-endregistered
Bridge, provider allows automated useregistered
Bridge, provider forbids automated use, as back-endrefused
the terms rule
On every routing decision
Foreground request for a personmay reach the bridge
Back-end job via an edited fallback listrefused again
the terms rule

The second check costs one comparison and prevents a violation nobody would see in an unattended system, so I kept it. The rule is small; the judgment is in where it sits.

I put compliance inside the mechanism that could violate it, and I check twice where a mistake would be invisible.

The bridges that make hosted providers routable present them in the same OpenAI-compatible shape as local models, rate-limit in Redis, and register and deregister themselves. The tool side of the same runtime is told in Tools are the platform.