The Watch Office

For an operations lead who wants one screen and alerts that mean something.

One signal · 10 specialists · one result

Does the work1
Speaks to people3
Supervises other agents1
Runs infrastructure4
Audits and guards1

How the work moves

Four steps, each done by one specialist

Starts with

Numbers the services already report

First

1Builds dashboards from the numbers the services reportOperations Dashboard Builder 2Tests every alert rule and its route to a phoneAlert Tuner

Then

3Delivers each alert as a short text to the on-call phoneOn-call Pager 4Answers plain questions about what runs where and today's costFleet Help Desk

Ends with

One screen of dashboards and alerts that reach a phone

the signalthe resultEach step is done by one specialist on the crew

The story

The agent builds the dashboards from the numbers the services already report. It tests every alert rule and its route to a phone from start to finish. Anyone on the team can ask in plain words what is running on which machine, what today has cost or which machine is free.

The crew

The specialists behind this role, grouped by the kind of work they do.

Does the work

Operations Dashboard BuilderThis agent builds the dashboards for jobs, queues, mining and system resources from the numbers the services already report, and checks that they load.

Runs infrastructure

Alert TunerThis agent keeps alert rules and text-message delivery working by testing each rule and its route from start to finish.Also serves The Night Shift
GPU Capacity Inventory ClerkThis agent takes stock of memory, model weights and idle capacity on each machine and rolls it up into one picture for planning.Also serves The Engine Builder
Memory Service OperatorLooks after the service day to day.Also serves The Search Keeper
Night-Shift Fleet WatchWatches machines, GPUs and services overnight, texts your phone only for what matters, runs the standard health checks before waking you, and has a summary ready in the morning.Also serves The Night Shift

Speaks to people

Fleet Help DeskAnswers plain questions such as "what is running on which GPU", "what has today cost" or "which machine is free" by picking the right fleet tools for the question.Also serves The Studio Engineer
On-call PagerIt turns each alert into a short text message on the on-call phone.Also serves The Night Shift
Live Operations AnnouncerIt posts each event to the team in plain words as it happens.Also serves The Circuit Breaker

Supervises other agents

Agent Fleet Health MonitorIt removes silent agents from the roster, updates the list of available machines, and reports fleet status.Also serves The Control Room

Audits and guards

Index Stall WatchdogResets the stuck job so scheduled indexing carries on, and records the reset.Also serves The Search Keeper

Where this crew also works

Each bar counts the specialists this role shares with another.