Innovations

Most of what I build starts as a failure I could name precisely, and the fix usually turns into a mechanism I reuse somewhere else. This page lists those mechanisms system by system, each in a few sentences, with a link to the longer account where one exists.

AgentOS

Drift as working-memory overflow: Long agent runs failed the same way every time, because the lead agent had to hold the rulebook, the plan and every teammate's state in its head. I treated that as a memory problem and moved all of it into a durable control plane, so each agent receives a small, freshly generated job with a contract attached and carries nothing between jobs. Models become swappable and a crash loses nothing that matters. See Why long agent runs fail.
One work graph, documents as projections: Every work order, task, piece of evidence, finding and cost line lives in a PostgreSQL graph of typed objects, edges and transitions. The Markdown an agent reads is generated from that graph, and edits to it are discarded, so an agent cannot change the truth by editing a file. See AgentOS.
The platform writes every packet: A coordinating agent never re-types a task, a report or a finding into another agent's prompt. AgentOS renders each packet from the graph and delivers it, and the lead issues one-line decisions, so it has no way to alter anyone's work in transit. That also keeps the lead's context small enough to last.
Envelopes: Work is grouped into envelopes, ratified boundaries inside which a team proceeds without asking anyone. Changing an envelope is its own typed request with a blast-radius class, so autonomy has a measurable edge and a change to that edge is visible and governed.
Continuations with receipts: Each step of a workflow is a typed continuation that the platform executes and receipts. Agents do not pick their next move from a menu. Resume and recovery replay the graph, so the system knows what happened without asking a model to remember. See State-machine design for multi-agent workflows.
A compiler for workflow law: A change to the workflow is a declarative specification that a compiler turns into one complete bundle: schemas, the database migration, the runtime site, dictionary rows, a probe and a manifest. A separate checker re-derives the bundle from scratch before it is admitted, and handwritten workflow law is refused. The database, the runtime and the tests cannot drift apart. See Schema and migration discipline.
Rules enforced through the agent's own hooks: One compiled Rust policy engine plugs into more than forty hook points of the coding agent's runtime: spec locks, path walls, mandatory gates, automatic continuation, and cost and knowledge capture. The agent cannot talk past a rule, because the rule lives outside its prompt. See Policy as code.
A hook may never lie: My rule is that anything a hook tells an agent to do must be a real operation that this actor can perform in the current state. I made it a type: a refusal carries the state, one sanctioned next action and who acts next, and a command can only be built from a verb resolved against a dictionary generated from the shipped binaries at every deploy. An agent is never told to run a command that does not exist. See Rust.
Completion an agent cannot fake: Each dispatched job carries a one-time token, and a job counts as finished only when a typed response carrying the matching token comes back. Text in the output that merely says it is done has no effect, and a mismatch becomes a typed failure that the platform recovers from on its own terms. Self-reported completion stops being a risk. See Family One.
Evidence first, then dispatch: The graph already knows which evidence each task requires, so AgentOS refuses to launch a quality check until every required item exists and is bound to the task, and names what is missing and who owes it. Checks are also anchored to the work graph's state at that moment, so an unrelated byte change in a file cannot invalidate a finished check. Known-missing inputs never burn a run.
Prose for behaviour, files for work: No work item, scope list or piece of evidence travels in prompt prose. A packet carries behavioural instructions plus pointers to files, because a model reads a file more reliably than it absorbs a pasted object. I adopted this after finding a declared size limit that nothing enforced and a multi-megabyte packet that went straight through it.
Follow-up checks bound to a frozen set of findings: The first quality pass is broad. Every later pass may judge only the fixes claimed against a frozen set of findings, and the budget for those passes is itself an object in the database. AI quality loops converge, because no later round can open a fresh concern. See Separation of duties for AI work.
Separation of duties across vendors: Builders, quality checkers, a gatekeeper that accepts only delivered facts, an adversarial check run on a different vendor's model, and a final closer are separate roles with separate authority. Classic assurance practice applies to AI-produced work, and a builder and its checker never share one model's blind spots. See Separation of duties for AI work.
Close as one transaction: A work order closes only when an independent closer's evidence covers every close item, checked inside a single database transaction. A human override exists, is explicit and is logged. See Closed by evidence.
Three-party change control: When a team discovers work mid-delivery that fits the agreed scope, an architect agent, an operator-proxy agent and an adversarial check approve it and it lands on its own. Anything that changes scope stops and pages me with a prepared decision packet. History is preserved; an amendment adds to the plan and never rewrites it. See AgentOS work tools.
Refusal-first recovery: Stand-down and crash recovery form a state machine. A new resume generation supersedes the old one under an idempotency key, the new team is grounded again from the database and the code, and two teams can never act on the same work order. A crash at three in the morning costs a restart, never the work.
Leases granted by the kernel: Work-order execution leases come from a socket-activated bridge that reads the kernel's peer credentials and grants a lease only to the dedicated service account. Authority comes from the operating system, so there is no token an agent could copy. See Linux security primitives.
Close-readiness by graph reachability: A pure walker builds a typed graph of control, impact and lineage edges and decides whether a work order can close by reachability. It also finds dead ends in the workflow before an agent hits them, and it found gaps in the platform itself before the compiler existed. See State-machine design for multi-agent workflows.
Test isolation at build time: After a test quietly wrote synthetic rows into a live database, I added a test that scans the crate's own source and fails the build if a database connection is created outside an approved entry point. A rule people forget became a build that refuses. See Static analysis as tests.
A helpdesk for the platform itself: An agent team that hits a defect in the platform reports it over an authenticated channel to a supervised support session, and keeps working on its own task. Doing the work and repairing the machine stay separate, so a team never patches governance to get unstuck.
Every token has an owner: Usage is parsed from each coding tool's own transcripts into signed, replay-safe events and attributed to the work order, role, persona and model, including the planning that happened before the work order existed, without any transcript content crossing the tool boundary. Gaps show as red, amber or green. Mixed-vendor teams become costable per piece of work. See What the work cost.
Human-equivalent hours that resist inflation: I measure the human-equivalent engineering effort of accepted, evidenced AI work with a two-layer formula computed in the database, where accepted work dominates and token volume is capped. My first version drew almost all of its result from token volume, so I discarded it and calibrated the second against work orders anchored by a human. See Cost attribution for AI work.
Authoring with an architect, a proxy and one approval: Intent becomes a schema-validated work order with scope, acceptance criteria, envelopes, team shape and close bars. An architect agent decomposes it, an operator-proxy agent challenges it, and two developers independently re-derive the facts it depends on, so disagreements surface before anyone builds. I approve it once. See AgentOS work tools.
Agent mailboxes: Every team lead and every seat has a mailbox. Seats can message only their own team, the sender's identity is derived by the server from the authenticated caller, and the final result of any long-running tool call arrives in the mailbox after the call itself has returned. Messaging is part of governance, so every message is attributable and nothing leaks across teams.
Events that drive agent work: An external system registers as an event source against a work order. A signed webhook becomes a governed event, lands in the owning team lead's mailbox, and the lead turns it into a finite operational task with its own evidence and close. Agents act on what happens in the world, with the same accountability as planned work. See Webhook ingestion and verification.
Standing work orders: A work order can have an open-ended envelope: a standing lead admits finite operational tasks as they arrive and brings specialists online only when a task needs them. Continuous operations reuse the governance built for projects, with no second engine.
A durable return path for webhooks: Provider events are checked at an edge listener against the exact bytes received, stored, batched and delivered across hosts into the containers of the work order that owns them. A batch whose delivery is uncertain is held and retried under the same delivery identity, never dropped and never double-applied, and receiving an event causes no side effect at the provider.
Teams as configuration: Each seat in a team is a declaration in the work order: vendor, model, persona, tools and login. The runtime launches every seat in its own container with an identity the server derives, gives each seat an isolated, fingerprinted copy of an operator-held login that never enters an image or a log, and lets a human watch any seat live or take control of it under an exclusive lease. A lead can start specialists only when needed. See AI subscriptions as governed workers.
A supervisor for terminal tools: Vendor coding tools run as interactive sessions behind a supervisor with a delivery adapter per method: literal paste, the vendor's app-server turns, a session inbox socket, or the agent client protocol. A prompt counts as delivered only when the tool's transcript shows it arrived, a message whose acceptance is unknown is never blindly resent, and the model a seat asked for and the model the vendor reports are stored as separate facts. Adding a vendor is a data entry and one adapter, with acceptance proofs. See tmux.
A fence around the most privileged seat: A fail-closed proxy is the team lead's only way out of its container. It allows exact vendor hostnames, refuses IP addresses and any DNS answer that points into private or metadata space, and requires the encrypted connection to name the same host that was approved; rollout probes prove every bypass fails. See Network egress confinement.
One command to deploy two hosts: A dependency-free Rust deploy tool swaps binaries atomically, pushes them to a second host, regenerates the command dictionary, and refuses to finish if a checksum differs anywhere across the two machines. See Cargo.
Done means coverage: Before porting a large open-source agent gateway to Rust, I pinned a census of the original source as the denominator, and judged the port against it. An agent's claim of completeness never decides. See Governed AI-assisted software engineering.
Refactors gated on symbol parity: Two monolith files of about forty thousand lines each were split into twenty-one modules by parallel agent lanes, accepted only when no public symbol had disappeared and an independent witness confirmed it. See Large-scale refactoring with parity proof.

Agent Swarm

One pipeline for every tool call: Validation, authorisation, rate limits, quotas, retries, sanitisation, audit and observation run in one fixed pipeline, and a new tool implements only its own step. Tool count becomes a data problem and every action leaves a trail. See Tools are the platform.
Agents as tool providers: Tools are declarations of a few kinds: composed, shell template, HTTP template or peer service. A language model can write them and they are still validated and attack-tested, and a peer service turns any agent into a tool another agent can call. See Declarative tool and agent definition.
Catalogs with a drift detector: Agent catalogs carry content hashes and version pins and reload without a restart. A detector on each host compares its catalog with the canonical one and flags any difference, and golden tests run through the production pipeline. See Catalog versioning and drift detection.
Personas, profiles and tool classes kept apart: A persona says how an agent behaves, a profile says which model and provider run it, and tool classes say what it may touch. Tool classes are resolved once and frozen into the launch snapshot, operator controls can never be delegated, and a lint stops prompts from smuggling tool rules into the wrong layer. See Agent Swarm.
A terms-of-service firewall: Model choice is routable infrastructure, and each provider's terms are code: traffic that a provider's terms forbid cannot be routed to that provider. See Provider-terms compliance in model routing.
Signed messages with bounded delegation: Every agent-to-agent message is a signed envelope, call chains are depth-checked, and a live registry with heartbeats and a reaper tells agents who is present and what each can do. Confused-deputy tricks and runaway delegation between language-model agents have nowhere to start. See Multi-agent system architecture.
Signatures that survive a float: Signatures on agent messages broke when a floating-point value re-serialised with different digits on the other side. I fixed it with one canonical form for hashing signed content, so the same message always produces the same bytes on every host and in every language.
Agents that never hold the key: An agent that needs a privileged action asks a gatekeeper agent that holds the credential and offers one narrow service, such as opening a merge request. The worker never sees the secret. See A coding agent that never holds the keys.
Launches that are safe to retry: Agent launches are idempotent and tied to governance authority. A retried launch never creates a second agent and never silently changes the first one's configuration, and a failed launch rolls back in stages. See Docker.
A credential broker the kernel authenticates: A broker pinned to a service user checks callers by their kernel credentials and hands out secrets by reference. Secret bytes never appear in a prompt, an environment variable or a log, and a written permissions matrix fixes who may hold what. See Credential brokering and secrets management.
Three planes across hosts: Control, messaging and durability traffic run on separate planes with written rules about what may cross between them, so agents on different machines keep talking when the coordinator is down. See Separate planes by role.
Durability that never becomes delivery: Agent events are journalled hot, written to PostgreSQL by a separate writer and promoted into searchable memory, without agents ever waiting on that path. Every tool call and lifecycle event becomes an attributed observation automatically, with secrets removed before the event leaves the process. See Distributed systems reliability.
Carve gates: Code is carved into zones, and product crates may not import platform code; the vector daemon may not link CUDA. Continuous integration enforces it, so reusable parts stay packageable on their own, and agent teams work under a written rule about which parts of the platform they may change. See Static analysis as tests.
Sandboxed agent containers: Every agent container runs with CPU, memory, process and runtime limits, scratch-only writable space, path checks, shell allowlists and an egress allowlist, combined with authority checks at launch. See Agent tool execution safety and sandboxing.

Helix

A media wire into Teams: A Windows media service and my Linux voice runtime exchange framed audio over a wire I designed, paced to what Teams expects. A network drop reconnects inside the same call without anyone redialling, and the C# and Rust sides share golden test vectors so they cannot disagree about a frame. See A self-hosted colleague in a Teams call.
Several AI voices, one floor: Several AI voices share one conversation, hand off to each other aloud and raise a hand to speak, and every one of them stops the moment a human talks. The agent can begin preparing an answer before you have clearly finished and still never speaks over you, and only the operator's voice may interrupt, so a television or a colleague in the room cannot. See Voice agents in meetings.
Silence beats a guessed transcript: The voice agent treats speech-model confabulation as a defect. A doubtful transcript is dropped, there is no silent fallback to a weaker model, and a misconfigured production setting fails loudly, so the agent never acts on words nobody said. See Real-time audio pipeline engineering.
A token per employee, per action: My meeting tool server separates who may connect from which digital employee acts. Every call resolves the employee and obtains a token scoped to that employee for that action, tool definitions come from a catalog file with no action lists for the model to choose from, and the platform's undocumented calling limits were settled by probing them directly. See Solving the digital-employee identity.
Presenting on stage, sharing a screen: With no bot interface for live presentation control, the agent presents slides through a supported in-meeting app bound to its own identity and refuses any command it cannot confirm on stage. It can also share a headless browser or terminal from a Linux container into the meeting. See A meeting attendee that will not fake it.
Council mode: A chair negotiates an objective with me, seats from different model vendors take ordered turns, challenge each other and revise candidates, and the outcome captures consensus, reservations and dissent. Councils survive restarts. I discarded my first version because it was parallel fan-out with no real deliberation in it. See Mattermost.
Many model bots in one thread: Several fixed-model bots share a thread with directed routing, so agent-to-agent loops cannot start, and each bot's provider session survives an account switch. Documents written in the conversation move through explicit pending and published states and are published exactly once. See AI teammates in Teams and chat.
Grounding by scope: Context is served as bounded packets per system, project, work order and participant, so a model sees what its scope needs and nothing else. See Agent365.
Fencing on four axes: The collaboration service's durable state carries generation fences on four independent axes and an exclusive lease per conversation. After a crash or a restart, a stale worker that wakes up cannot write anything, because its generation no longer matches.
Work from Teams without inbound shells: A task submitted in Teams goes to a coding-agent worker on an enrolled machine or a cloud job. Claims are exactly-once even when two dozen identical submissions arrive together, hosts only dial out, and every execution gets its own scoped credentials by reference.
One brain, two surfaces: A light voice model answers quickly while the durable brain does the work, and voice and text reach the same project context. Operator, employee, executing bot and voice session each keep their own identity, so a spoken request is attributed as precisely as a typed one.
A media studio agents cannot misuse: Any agent can render, transform and narrate media on shared GPUs within a budget. A render whose outcome is unknown is reconciled and never blindly resubmitted, media transforms are typed so no caller supplies codec strings, transfers between hosts are checked, and only a human can approve a model licence. See ComfyUI.

FAFOHive

Memory that pays for itself: Persistent memory for coding agents captures every tool call, extracts what matters, compresses long transcripts and re-injects context later. It tracks whether the memory saves more tokens than it costs, so memory earns its place in the context. See Memory and code intelligence for agents.
Search that degrades gracefully: Semantic code search routes by query mode over partitioned indexes, and under overload it keeps answering with fusion-only results. The embedding model moved out of the search process into its own service without breaking a single caller. See Two thousand concurrent workers.
Compute first, narrate second: Call graphs come from the code graph; the language model only narrates edges it was handed, and the query order and filters enforce it. An explainer that is given the evidence has nothing to invent. See A code explainer that is not allowed to invent.
Decisions kept apart from telemetry: Durable decisions, discoveries and fixes are stored separately from raw activity, built asynchronously and tagged by cost class. A fleet of agents can recall what was already settled and move on. See Memory and code search.
A governed provider gateway with a crash-safe outbox: payments, email/CRM and social-publishing tools, rebuilt from each provider's API, sit behind my gateway. Every call is attributed to a work order through an outbox whose replay never calls the provider again, and risky operations are checked before they mutate anything. See Agents with real tools and no keys.
Tools shaped like jobs: My social tools expose campaign-level jobs built around the decision an agent is making, with costs and guards. I generated eighty-three endpoint tools from an API description and withdrew them in favour of fewer, designed tools. See MCP tool design.
Read it back or say you are unsure: Every social write is confirmed by reading it back or is marked uncertain, and an uncertain action is never resent, because a duplicate post is worse than a missing one. One process per provider owns all writes, with a shared rate limiter, budget reservations and a durable claim queue in SQLite, and no message broker. See Idempotency, exactly-once effects and crash recovery.
The browser you already trust: For a platform with no usable API, a browser extension runs same-origin requests inside the session the operator already has, through a native messaging host and a private socket. Credentials never leave the browser and the server sees only results. Each operation's route, official API, data API or browser, is an explicit choice, so a refused call never turns into a silent retry somewhere else. See Chrome extensions (Manifest V3).
A signed event plane with a response clock: Platform events are signed over the exact body with a replay window, deduplicated, delivered to agents, marked answered when an agent replies, and swept for missed response targets. Events may be delivered twice; actions never happen twice. See Webhook ingestion and verification.
Keep every byte you paid for: Every API response is captured append-only. A cached metric carries the moment it was captured, and a failed capture is reported. Metered data is never fetched twice and never misrepresented. See X API.
A tool explorer that distrusts tool descriptions: A read-only explorer over more than six hundred tools across three servers carries AI-written guides that are hashed against their source, sandboxed and checked against exact catalog IDs. Tool descriptions are treated as untrusted input. See Prompt-injection-aware tooling.
Identity bound on the server for website and inbox events: A business website, an inbound email route and provider inboxes are each bound on the server to the work order that owns them. An agent never passes its own identity as a parameter, so a confused deputy has nothing to forge. See Business systems as safe agent tools.
Child identities that die with their parent: Sub-agent tokens are minted in one atomic step with a parent-chain check, so a child identity is bounded by its parent and stops working when the parent does.

FAFO Core and Function Forge

One function, every execution shape: A single function trait runs as a job, a service or a stream, batched or unbatched, in-process or in a container on any host. Lifecycle, batching and location are independent dimensions, which replaced a combinatorial pile of runners. See FAFO Core and Function Forge.
Claims that never release: Workers claim rows with time-bounded leases and never write a release; expiry frees the work. At multi-million-row scale this avoids the dead-tuple churn that claim-and-release update pairs cause in PostgreSQL. See Lease-based work claiming.
One ready signal per slice: The conductor owns the set of outstanding slices, each slice raises exactly one ready signal, and stale signals are ignored. Polling touches only what is ready. See Distributed job orchestration.
Registration at link time: A function registers itself when the binary links, so a function is either compiled fully wired as a tool or the build fails. There is no runtime registry to fall out of step. See Link-time plugin registration.
A sandbox for AI-written functions: Function Forge generates a fully wired tool into the running estate, and the AI-written business logic receives only a bounded batch context with no raw connections and no direct database writes. Generated code stays inside the governed write path by construction. See Tools that build tools, under governance.
Reversible estate edits: Generating a function edits several services deterministically and reversibly, and the single human approval sits at the privileged restart that makes it live. See Reversible infrastructure mutation.
A write path sized by the database: Writes go through a dual-lane queue to a batch writer whose batch size follows PostgreSQL's bind-parameter limit, with no schema lookups on the hot path. Statistics run on a side-effect-free plane, and each queue's storage chooses its failure mode, so the write-behind queue never evicts; it fails loudly. See 196 deadlocks to zero.
A runtime schema resolver: Functions write to any PostgreSQL table through runtime schema resolution, type-directed binding and injection-safe identifiers, so a function written once runs against any schema. See Schema-agnostic data access.
Live control of running jobs: Agents steer running jobs over a dedicated control lane with nineteen actions such as pause, scale and drain, and a job promotes atomically to a service. GPU resources are released when a worker drops. See FAFODev.
A spend ceiling in the critical path: Provisioning cloud or LAN GPUs checks a fail-closed daily budget resolved in the operator's own timezone, with per-second spend tracking. Agents can add capacity; only a human can raise the ceiling. See Budgets and refusals.
Location as configuration: Runtime injection and loader modes let one function binary run on any host that can run its image, and the same substrate provisions inference consumers, agents and infrastructure. See Federated compute provisioning.
An API field is real once a call returns it: API descriptions are normalised and enriched by live probes, and the integration profile admits only the fields a successful live call returned, before any tool is built on them. See Evidence-gated API integration.
Writes against APIs that change under you: A mutation against a third-party API that may change or disappear is reconciled against what the API shows afterwards, with bounded replay, so every governed write ends confirmed or explicitly uncertain. See Family Five.
Any API by configuration: A declarative connectivity profile describes an API, and one generic MCP server exposes it as governed tools with namespace isolation, exclusive claims and token-bucket pacing. Integrating a new system is a profile and a validation run. See Business systems as safe agent tools.
Routing generated from YAML: The gateway's tool routing, handlers and modules are generated from per-function YAML at build time, so a new function becomes a tool with no hand-written dispatch. See YAML.

Inference Fabric

A broker that understands tokens: The broker reshapes its queues continuously by token geometry, packing items into shape-pure lanes that cut padding to fractions of a percent, and orders delivery so shared prefixes stay hot in each GPU's cache. A hard guard refuses mixed-shape batches. See A message broker that understands tokens.
Tombstones on an append-only log: Acknowledging an item is a four-byte write in place inside a memory-mapped, append-only log segment. Each message gets its own acknowledgement at log-structured cost, and durability needs no distributed transactions. See Spend structure before spending waste.
Lossless reconciliation after a crash: After a crash, the broker reconciles in-flight work against what each GPU consumer reports it actually holds, so work is neither lost nor run twice. See Millions of queued items that were not there.
A runtime that pulls: GPU consumers pull shape-pure batches and keep the device saturated, with no request server in the path, and every hop restarts on its own. See From 44K to 163K tokens per second.
No allocations after warm-up: The rule that nothing allocates GPU memory after warm-up, and that the compute stream never synchronises on the hot path, is enforced in code at the boundary between Rust and C++, and a trace confirms it. See C++.
A detector for silent corruption: The hidden-state extraction backend captures a model's internal states in one pass. Before trusting an engine, it measures what the engine's token-index tensor actually means and refuses to run if the answer is ambiguous, and each extraction specification is canonically encoded and hashed so every stored vector traces to the run that made it. See A detector for a silent corruption.
A small service that closes the loop: The completion service turns inference results back into rows in the database. Each function owns its completion logic through static dispatch with no RPC or plugin machinery, which makes a large pipeline submit-and-walk-away. See Inference Fabric.
Vector storage with lineage: Vectors never pass through SQL. Shards are append-only, lineage-checked and materialised from the database that owns the source data. See Append-only with lineage.
A lease between storage and the GPU: Storage hands GPU executors leases over shard memory through a dedicated interface crate whose constants are mirrored at compile time. An executor cannot outlive or corrupt the memory it reads, and a residency manager decides which shards live on the GPU within an explicit budget. See GPU vector search and corpus analytics.
Indexes you can rebuild identically: Inverted-file and lattice indexes build from a single seed with stable tie-breaks and carry a content hash, so an index can be rebuilt later and shown to be identical. See Approximate nearest-neighbour search.
Search and health in one pass: One GPU pass over shard-resident vectors returns nearest neighbours and thirty-eight cluster-health metrics, and its search results are byte-identical to running search alone. Only results are written back. See Thirty-eight metrics in a dozen passes.
Cluster-health kernels: Hand-written CUDA kernels compute per-cluster quality metrics in batches, each with documented numerics, memory budget and occupancy, race-freedom tests and a CPU reference it must match. See GPU numerical kernel design.
First-party search kernels: Exact top-k, inverted-file and lattice search run on my own CUDA kernels, including a fused lattice-expansion kernel that writes coalesced results on a single stream. See GPU vector search and corpus analytics.
CUDA from Rust on a new GPU: The analytics engine drives the CUDA driver API from Rust and compiles native binaries for the newest architecture before mainstream bindings supported it. See CUDA driver API.
A spare PC becomes capacity in one call: Onboarding a LAN machine, syncing the engine and starting a consumer that attaches itself are one agent call each. The provisioner reads the model artifact it deploys and sizes the consumer from it, SSH is used only for bootstrap, and steady-state traffic runs over mutual TLS. See Federated compute provisioning.

ActiveKV

Host memory holds the history: Attention history lives in host RAM and the GPU holds a bounded working set staged per step, so context length follows host memory. One 32 GB Intel Arc Pro GPU serves 128,000-token histories for a 27-billion-parameter model this way. See Context beyond VRAM.
Staging without a stall: Host-to-device staging uses fence-protected leases and generation counters, and a buffer whose fence outcome is unknown is quarantined and never reused. Asynchronous copies stay correct under preemption without stalling the step. See Staging without a stall.
An exact merge across tiers: Attention computed over different tiers of history is merged with log-sum-exp arithmetic, so the result is identical to fully resident attention, and bit-identical output is the gate for the exact tier. See Half the bytes, exactly merged.
An INT8 history tier: Older history can be held in a compressed INT8 format with its own SYCL kernels and a stated quality contract, and a hybrid mode mixes precision tiers. See Half the bytes, exactly merged.
Judge a representation by the bytes it removes: On a transfer-bound path, a slower kernel wins when it removes enough bytes. My gate compares total time at the target size and never kernel speed alone. See Judge a representation by the bytes it stops moving.
A page pool with refusals: GPU memory is a fungible page pool with per-context lifecycle, admission and checkpoint and restore. A request that cannot be served safely is refused, and no limit may be tuned to the size of a benchmark fixture. See Budgets and refusals.
Parity that proves a rewrite: A rewrite counts only against references run in the same mode, comparisons that feed the exact objects the served path uses, and proof that native code is bound on the real startup path. See Proving a rewrite is the same system.
A native engine with the smallest Python boundary: The runtime moved from Python into a C++ engine inside a third-party serving engine, with results, exception types, order and telemetry unchanged, and every remaining line of Python justified. See Rewriting an engine extension without changing a token.
Instrumentation that knows it is watching: Profiling runs at selectable capture levels. When timing events turned out to serialise the copies they were measuring, I withdrew the conclusion and intercepted GPU driver calls directly with a preloaded shim and a Rust log parser. See The observer that changed the answer.
A day in the life of a token: Per-step traces follow each token through scheduling, staging, attention and completion, so every millisecond of serving latency is attributed to the stage that spent it. Performance work starts from those traces, which name the stage worth changing before anyone touches a kernel.
Accepted improvements become core code: A performance candidate passes an intake check, a side-by-side run on a quiet host and a check on every serving route. An accepted change has no off switch, and a gain tuned to the test fixture is rejected. See Directing AI coding agents at program scale.
A drop-in binary with receipts: The extension is delivered as native binaries plus a patch applied to the user's own vLLM, with receipts that tie source to binary to the served process. See Bigger models and longer context on fixed GPU memory.
Kernels for new silicon: I brought serving up on Intel's Xe2 architecture by patching SYCL attention and recurrent-layer kernels while keeping outputs identical. See Bringing a serving stack up on new silicon.

Capabilities model and distillation

A tool catalog inside a small model: A 0.6-billion-parameter model trained on the tool registry holds the catalog in its weights and returns pointers to the right tools. When tools change, data, fine-tune, quantisation and evaluation rerun in minutes with byte-identical results, and a failing check blocks the swap. See Capabilities model.
About a hundred tokens of tool context: One discovery tool replaces the whole catalog in an agent's context, and the size of its description is a budget that tests enforce. See A hundred tokens instead of forty-two thousand.
No invented tools: Decoding is constrained by a token trie built from the registry, so the model can emit only tools that exist. See Grammar-constrained decoding.
A constraint cheaper than none: My CUDA decoder puts the trie inside the sampler and computes only the legal rows of the output head, so constrained decoding costs less per token than unconstrained selection over the full vocabulary. See Constraint made cheaper than no constraint.
Bit-identical in any batch: A request's output does not depend on what it is batched with, alone or among 512, with an equality gate on every kernel optimisation. See Bit-identical across batch compositions.
A router beside a production model: A fleet daemon serves 2,048 simultaneous tool queries with no cross-request contamination on the same GPU as a 30-billion-parameter production model. See A router beside a production model.
Training data from a panel of model families: Training data is written from the registry by models from several families, with style-matched hard negatives, and leakage is defined across the training and evaluation pair. See Synthetic training data generation.
An instrument before the model: A frozen evaluation pair, a tuned classical retrieval baseline and a synthetic distractor catalog came before the model, and that instrument caught a headline error before publication. See The baseline that tied us.
An undocumented format as a probed contract: A 4-bit weight format with incomplete documentation was characterised by probes and pinned as a contract before any kernel relied on it. See CUDA C++.
Redistribution inside the database: Typed Rust functions maintain a vector knowledge lattice, moving units between regions, admitting candidates through a gatekeeper and building embeddings, all queue-driven and controllable through tool calls. See Redistribution inside the database.
Translation proposes, mining decides, a gatekeeper judges: In my distillation research a translator maps between a model's activation space and a curated knowledge lattice. Its proposals count only after mined activations agree and a gatekeeper accepts them, which keeps a learned mapping from grading its own work.
Activation capture you can replay: Probe inventories are frozen and every run is identified by the hash of its content, and recovery journals restore buffered work after a failure, so a capture can be repeated exactly. See Large-scale GPU data pipelines.
Tens of millions of generated probes: A decoupled producer streamed a multi-million-item probe field through the fabric with provenance on every item. It is the workload that shaped most of the platform. See The workload that shaped the platform.
A 27B model on one 32 GB card: I quantised a 27-billion-parameter model to 4-bit weights for one Intel Arc Pro GPU and attributed every throughput gain to representation or to freed admission capacity, with quality reported as a trade-off. See A 27B model on one 32 GB Intel card.

Auth and access plane

The consent screen is the control: Locking client registration broke legitimate assistants that cache their client IDs, so I placed the control at consent, the step every grant must pass, and temporary clients are promoted by the operator. See The hardening fix that locked everyone out.
One identity layer for four assistants: One authorization server handles the different discovery, consent and client-caching behaviour of four vendors' assistants without a server per vendor. Every bearer token is introspected on every request and a request fails when introspection cannot answer. See Auth and access plane.
Audit you can trust, off the hot path: A security library keeps a hash-chained audit log, privacy-preserving device fingerprints and anomaly monitoring, and stays out of the request path so it never slows a grant. See Application security hardening.
Private tools for a hosted agent platform: An edge worker checks tenant-pinned tokens with WebCrypto, rotates keys through two slots without downtime, and absorbs connector behaviour I found only in edge logs. See Microsoft Copilot Studio.
No open ports: Every public endpoint is reached through outbound tunnels and a hardened reverse proxy, one per purpose, so a compromise stays small. See Zero-inbound network exposure.

Cross-cutting

Disconnected producers and consumers: Producers submit and exit, consumers pull when ready, and completion is its own step, so every hop restarts independently. See Disconnected producers and consumers.
Authority outside the model: Rules, state and identity live in code and databases the model cannot edit. See Authority outside the model.
The token as the unit: Batching, budgets, cost and memory are reasoned in tokens. See The token as the unit.
Measure the instrument: Profilers, baselines and benchmarks get checked before their numbers do, and I withdraw a conclusion when the instrument changed the answer. See Measure the instrument.
Uncertain is an outcome: Wherever a side effect crosses a boundary, in vendor tool sessions, social writes, renders and event delivery, an outcome that cannot be confirmed is stored as uncertain and reconciled, and never blindly retried. See Reconcile, never resubmit.
Identity comes from the caller: Across tools, event bindings and digital-employee actions, the acting identity is derived on the server from authentication and is never a parameter. See Identity from the caller.
A human at the irreversible step: Budgets, privileged restarts, model licences, scope changes and work-order approval each carry one human gate at the point that cannot be undone, and everything else runs on its own. See A human at the privileged step.
Audits that can overturn a pass: I commission independent adversarial audits of my own systems and act on them, including one that overturned an earlier acceptance of an AI-built subsystem whose behaviour did not match its documentation. The finding changed how acceptance works across the platform.