Corey James- 50golden fixtures for 40 declarative tools
- 1,200lines in the Rust test runner
- 3platform-semantics defects the Facebook proof found, each now a test
The same standard governs the social integrations: every write read back, every create with a delete, every inbound item an event. I set acceptance rules that turn "it works" into counted cases and side-channel measurements, and I make the build enforce them.
Fixtures that fail the build
Agent Swarm adds tools as YAML declarations, and I knew a tool nobody tests is the hole an agent eventually finds. So a tool without fixtures fails the build, and attack fixtures are required even for one-line tools, because exceptions to a safety rule are where the holes appear.
The test runner is a Rust crate that loads the catalog with the same loader the runtime uses, registers the built-in and declarative tools, and sends every fixture through the real invocation harness. Beside it sits a tracer-bullet regression: synthetic in-process agents and a minimal persona that cross the registry, signed messaging and observation capture with no model loaded. The design is in Tools are the platform.
Qualifying the X module by parameter class
FAFOHive lets agents act on real social platforms through in-house modules, where a mistake is a public post. My rule was that no tool reaches an agent until each of its parameter classes is backed by a real-API or fixture test. Counting by parameter class states exactly how much of a tool surface is covered, and no more: qualification covers a finite set of classes.
The X module was qualified twice, once on fixtures and once against the metered data API, with boundary tests at the platform's own limits.
at 10,000post at
25,000
Facebook and Reddit, managed through the tools alone
For Facebook I set a harder rule: any post must be fully manageable through the agent tools alone. The proof harness called the tools through the same path the gateway uses and blocked and counted every HTTP request made outside a tool body, which turned "the tools can do it" into a measurement. It created every object kind on an unpublished test page with generated media, checked each one, deleted every created object through the tools, and confirmed they were gone. For Reddit, a short proof ran through the governed gateway with stable idempotency keys.
1 create · 2 read back exactly · 3 edit, read back · 4 delete, confirmed gone
Every proof leaves the platform as it found it, on test pages and test content. Writes were switched on only inside the proof process; by default they stay off.
A write whose outcome is uncertain is reconciled and cleaned up, and never resent, because on a social platform a duplicate does more harm than a gap.