Why We Built a Context Hub MCP Server for Coding Agents

TL;DR: We index 16,284 chunks from 12 repositories into a local ChromaDB + SQLite Full-Text Search (FTS5) store and expose them through seven Model Context Protocol (MCP) tools. Coding agents that used to grep through .venv source for every Pipecat question now get filtered, ranked results in a single call. The retrieval failures that we observed were piped back into the development process to improve the Pipecat context hub.

The Problem With Feeding Docs to Agents

In January 2026, I started building kai-pipecat, a voice AI application that handles long disparate conversations. The bot stores conversation history locally in a SQLite database and then performs complex search while maintaining conversations. This means it has to context engineer on the fly and makes use of several low-level Pipecat features.

Claude Code is doing all the coding. It was also spending an absurd amount of time grepping and globbing through .venv/lib/python3.12/site-packages/pipecat/ to make sense of parallel pipelines, frame types, and async API calls.

I tried the obvious fix first: a Claude Code skill that embedded the full llms-full.txt docs dump (~800KB of markdown) that it could pull from. Two problems surfaced immediately. Skills are static text: no filtering, no ranking, no awareness of what is relevant to the current question. Pipecat releases a new version each week, sometimes with architecture changes, which means the skills need to be updated frequently.

We needed structured retrieval with filters, not a text dump. That pointed me toward MCP.

From Raw Files to Chunks

The Context Hub transforms three kinds of raw content into indexed chunks. Each pipeline exists because agents search for these content types differently and each one fixed a specific failure mode we observed.

Documentation comes from docs.pipecat.ai/llms-full.txt. The crawler splits pages on markdown headings (h1–h6), skipping headings inside fenced code blocks, then applies a 512-token window with 50-token overlap. Splits follow paragraph boundaries first, falling back to sentences. We chose heading-aligned splits over fixed token windows because they preserve semantic boundaries — the trade-off is size variance (8 to 8,361 characters per chunk), but heading-aligned chunks produce better search results than cuts mid-paragraph. Before chunking, a Mintlify tag cleaner converts <Note>, <ParamField>, and <Card> tags into standard markdown. This produces 3,722 chunks with a median of 361 characters.

Example code spans 12 repositories. The GitHub ingester discovers example directories through two layout patterns: examples/foundational/NN-name/ subdirectories for the main repo, root-level scanning for community repos (pipecat-examples). It then chunks at 256-token boundaries aligned to function and class definitions (def , class , async def ). The reason we index community repos at all: official docs cover the API surface but not how people actually use it. For example, pipecat-cloud-daily-sip-pstn is the only indexed source showing SIP telephony integration. Each chunk passes through TaxonomyBuilder, which infers capability tags, execution mode, and key files from directory names, READMEs, and Python imports. This structured metadata powers filtered queries like search_examples(query="Deepgram", execution_mode="cloud"). This produces 6,160 chunks with a median of 1,002 characters.

Abstract Syntax Tree (AST) source is the layer that reduced .venv grepping the most. Python’s ast module extracts four chunk types from every .py file in the framework’s src/ tree: module overviews (530 chunks listing classes, functions, and imports), class overviews (1,258 chunks with base classes, constructor signatures, and method indices), method chunks (4,270 with full source bodies), and standalone functions (344). Only methods with 3+ lines get indexed. Each chunk carries rich metadata — module_path, class_name, method_signature, base_classes (stored as JSON to avoid corruption from generics like Base[Foo, Bar]), and is_dataclass flags. This metadata powers a symbol lookup filter cascade: try exact class_name, then method_name, then semantic fallback. Without AST indexing, get_code_snippet(symbol="MLXModel") searched example code instead of framework source, returning irrelevant results. This produces 6,402 chunks with a median of 597 characters.

How the Index Is Organised

Every chunk becomes a ChunkedRecord and carries a chunk_id, content, content_type (doc, code, source, or readme), source_url, repo, path, commit_sha, and a metadata dict whose schema varies by content type. Chunk IDs are deterministic SHA256 hashes.

An EmbeddingIndexWriter computes 384-dimensional embeddings via all-MiniLM-L6-v2 (runs local) before upserting into two parallel backends. We chose this model over larger alternatives like bge-large because the full 16K-record index fits in memory on a laptop.

ChromaDB stores vectors with flattened metadata, batched in groups of 5,000. Search uses cosine similarity with pushdown filters on exact-match fields and 3x over-fetching when post-filters are active.

SQLite FTS5 stores full content with Porter stemming and unicode61 tokenisation, auto-synced via triggers. BM25 (Best Matching 25) keyword search catches the exact matches that embeddings miss. At query time, HybridRetriever runs both backends in parallel, merges via Reciprocal Rank Fusion (normalised to 0–1), and applies symbol boosts and staleness penalties.

A separate index_metadata table stores per-repo commit SHAs and a docs content hash, powering incremental refresh, i.e., unchanged sources get skipped entirely, dropping refresh time from ~90s to ~23s.

Evolving with Data from Real Agent Sessions

We analysed three coding sessions (18MB of JSONL (JSON Lines) logs) where the agent built features on kai-pipecat with the Context Hub active. The pattern was consistent: roughly 100 MCP tool calls and 80 .venv source reads per session. MCP handled discovery and orientation; direct source reads handled implementation details. These transcripts were fed to the context hub, which then improved the search and the embeddings.

Before I begin with failures, a clear win is deprecation detection. The team does a great job of placing these in the changelogs, docs, and the code itself. In our example: the agent called search_api(query="InputParams") and discovered that DailyTransport’s constructor signature had changed — a parameter was deprecated in favour of a new configuration object. Without indexed source metadata, the agent would have used the old parameter and we’d have found the issue later in testing.

For example, a search for GoogleLLMService in the context-hub got zero results, and the agent immediately fell back to grepping .venv. The class existed but our AST extractor had split the function incorrectly, causing the parse to fail and return no results.

Another pattern: get_code_snippet(symbol="DailyTransport.configure") returned a truncated method. configure() is 180 lines; our default max_lines was 50. We raised it to 100 after analysing the distribution — 97% of 4,270 methods fit under 100 lines (P90=56, P95=77). The median method is 21 lines, but the methods agents actually ask about sit in the 76–100 range. Optimising for the median punished the methods that matter.

Another interesting finding was a workflow pattern. Agents consistently use MCP in two phases. Phase one is orientation: "what exists, where is it, what is the API surface?" This is where search_docs, search_api, and search_examples earn their keep. Phase two is implementation: "show me the exact source, including private helpers." This is where agents switch to .venv reads when our chunks do not include call graphs.

So one thing that we strive to do with the context hub is reducing the search space. An agent that has access to 450 framework files and 12 community repos cannot efficiently grep its way to the correct answer. The goal is for the agent to start with 5 ranked results from search_api and get implementation pointers before diving into source files.

Visualising 16,284 Chunks

Claude built an interactive explorer, dashboard/public/latent-space.html, that shows the latent space of the context hub, with each chunk represented as a point in a three-dimensional space. For the core Pipecat functionality, it shows that the doc chunks overlap with the implementation chunks, suggesting that the API surface is well-documented. The example code chunks are well-separated from both, suggesting that they are distinct from the API surface.

What I Would Do Differently

Cross-reference metadata, from the start. The biggest reason agents fall back to .venv reads is tracing call chains: "method A calls method B which yields frame C." Our chunks are isolated. Adding an imports field and a proper call graph would cut the .venv reads substantially.

I also need a better feedback loop. When search_api returns unhelpful results, we only know by manually reading session logs or when someone reports a poor result. An MCP tool accepting "this was not useful" signals could drive re-indexing priorities. The gap between retrieval-returned and retrieval-useful is my main focus for the next round of improvements.

Updated (2026-03): In v0.0.8 we shipped tracing call chains.

What changed. The AST extractor now walks each method’s executable body and extracts two new metadata fields: yields (frame class names from yield FrameType(...) expressions) and calls (method names from self.method(), ClassName.method(), and super().method() patterns). These are stored as structured lists on every method and function chunk, and surfaced as filter parameters on search_api and as fields on the ApiHit output.

For example, search_api(query="TTS audio", yields="TTSAudioRawFrame") returns only TTS service implementations that actually yield that frame type — Kokoro, ElevenLabs, Rime, Speechmatics, and others. Previously, an agent would have had to open each service file and read the source to find which ones produce audio frames. Similarly, search_api(query="frame processing", calls="push_frame") finds every method that calls push_frame, which is the core pattern for forwarding frames through a Pipecat pipeline.

Scope boundaries matter. The extraction only walks executable function bodies. Decorators, parameter defaults, and return annotations are excluded. Nested functions, lambdas, and nested classes create scope boundaries that the walker will not cross, so yield AudioFrame() inside a closure is not attributed to the enclosing method. Comprehension calls are intentionally included since they are part of the method’s runtime logic. yield from is excluded because the generator name is not a frame type.

The index also grew. Pipecat-internal imports (including relative imports like from .utils import X) are now propagated to class and method chunks, so agents can answer "what does this method depend on?" without a second lookup to the module overview.