Runs infrastructure · AI models and performance

Model Serving Engineer

Builds a version of the model sized for each GPU card you own, measures how fast each card runs it, sets each card's limits for the scheduler, and copies the right build to every machine.

Asked byThe Model Launch Manager
The Engine Builder
Serves
  • The Model Launch Manager
  • The Engine Builder

What it does

Builds a version of the model sized for each GPU card you own, measures how fast each card runs it, sets each card's limits for the scheduler, and copies the right build to every machine. You get the most out of mixed hardware without hand-tuning each box.

For an ML platform engineer.

Roles it serves

The Model Launch ManagerFor a platform lead shipping a new model version.The Engine BuilderFor an engineering lead shipping an AI engine to more than one kind of graphics processor.

Works alongside

The specialists it shares the most crews with.

AI Answer Accuracy CheckerAudits and guardsAI Engine Build and Benchmark AgentRuns infrastructureAI Model LibrarianKeeps knowledgeAI Service Load TesterAudits and guardsAI Speed InvestigatorRuns infrastructureAnswer Consistency SetterRuns infrastructure

Routes it travels

A new model becomes a story
  1. 1The Model Shrinker
  2. 2The Model Launch Manager
  3. 3The Spend Controller
  4. 4The Newsroom