Corey JamesA reference library beside the code
The memory capability in FAFOHive serves agents that port patterns from vendor SDKs and call vendor APIs.
An agent that guesses an API produces code that compiles and fails.
I decided agents should cite the vendor's own source and documentation, so I built a reference library next to the code index: vendor SDK repositories, API documentation and crawled developer sites, each in its own named index so a vendor sample never surfaces as the team's code, and all searched under one contract with the same tools as the working code.
Other systems lean on these indexes. Microsoft's Teams SDK, Graph API reference and Copilot Studio samples back the work in Helix, and the vLLM source backs ActiveKV. The same evidence-first idea governs the code tools in A code explainer that is not allowed to invent.
A crawler that keeps only the documentation
The first ingestion path is a documentation crawler I wrote in Rust. It walks a site breadth-first, depth-first or best-first by link score, honours robots.txt and rate limits by default, stays inside the documentation that matters and converts pages to Markdown. Pages are cleaned before indexing, because menus and footers repeated on every page distort retrieval. Pages that render only in a browser go through optional headless rendering, and re-crawls are incremental.
Documents turned into citable structure
The second turns documents into structure. PDF, Markdown and HTML split into hierarchical sections with breadcrumbs and, for PDFs, page ranges. Each chunk carries a prefix naming where it sits, and a language model adds section summaries before embedding.
PDF text extraction uses a vendored open-source parser; the crawler, the sectioning and the enrichment are mine, and the sites and SDKs belong to their owners.