MCP = Mega Context Problem - Matt Carey

Matt Carey, an engineer working on MCP and agents at Cloudflare, delivers a talk challenging the conventional approach to exposing large APIs via MCP servers and proposing a fundamentally different model centered on progressive tool discovery and programmatic code execution.

Carey opens by framing the core problem: Cloudflare's REST OpenAPI spec is 2.3 million tokens. Naively converting all API endpoints into MCP tool definitions produces approximately 1.1 million tokens of tool descriptions — far exceeding any context window a client agent can handle. The standard industry response was to split large APIs into multiple product-scoped MCP servers (Cloudflare ended up with 16), but this approach yields incomplete coverage, forces users to manually select the right server, and still misses many endpoints (e.g., a product suite with 30 endpoints might only expose 6 via its MCP server). Carey argues this approach got "all wrong" and introduces the concept of progressive tool discovery as the correct framing.

Three solutions are explored: First, CLI-based discovery, where agents invoke a CLI tool (like Cloudflare's Wrangler), call --help flags recursively, and introspect available commands dynamically. This is self-documenting by design and works well with tools like Claude Code, but requires shell access — a meaningful constraint. Second, tool search, where clients (agents) issue a keyword search against a tool catalog and load only k relevant tools (e.g., k=8) into context at inference time. Claude Code implements this pattern. The downside is that multiple tools remain in context even when only one is used, resulting in ~2,100 tokens with only ~500 actively needed. Third — and Carey's preferred solution — is "code mode": instead of loading tool definitions, the agent writes TypeScript code against a typed SDK generated from the OpenAPI spec. Types are a far more token-efficient representation of API inputs and outputs, allowing coverage of thousands of endpoints in roughly a thousand tokens.

For code mode execution, Carey addresses the critical security concern of running LLM-generated untrusted code. Cloudflare solves this with Cloudflare Workers (WorkerD), which runs code in isolated V8 isolates. The sandbox is programmable: network access can be toggled per execution, secrets are not exposed (process.env is empty), and node compatibility can be disabled entirely. This is contrasted with other emerging primitives for safely running untrusted code: Deno with `deno run` sandboxing, and Pydantic's Monty library for sandboxed Python execution. Carey frames this as a broader infrastructure shift — analogous to 1950s punch-card computing — where AI agents as users will increasingly write and execute code against APIs, rather than calling discrete tools.

Carey then outlines the expected evolution of the MCP client landscape. He expects programmatic tool calling to become standard in clients, with agents generating and persisting mini-scripts for recurring tasks (e.g., web scraping cron jobs that self-repair when they break). He also predicts the MCP TypeScript SDK will become so lightweight it will be natively embedded in every major full-stack framework (e.g., Next.js), turning MCP exposure into a single boolean flag on API routes — "MCP equals true" — making every existing API instantly accessible to agents without rebuilding infrastructure.

The talk concludes with a live demo where an MCP client embedded in the presentation slides successfully authenticates and accesses the entire Cloudflare API with read-only permissions, covering all ~2,600 endpoints. Carey urges API providers — especially accessibility data providers — to adopt code mode, referencing a Cloudflare blog post titled "How we gave agents an entire API in a thousand tokens."