Bending a Public MCP Server Without Breaking It — Nimrod Hauser, Baz

April 8, 202640:501,361 views

Summary

Nimrod Hauser, Founding Engineer at Baz, delivers a practical, code-driven talk on adapting public MCP servers for production agentic workloads. Using Baz's spec reviewer product as the concrete example — an agent that compares Jira/Linear tickets and Figma designs against live implementations by browsing the actual system with Playwright — Hauser walks through five progressive improvements to how third-party MCP tools are consumed, showing a failing V0 baseline evolve into a production-ready V4 implementation.

The talk opens with a live demo of the out-of-the-box Playwright MCP server, which ships 21 tools with shallow, one-line descriptions (e.g., 'Press a key on the keyboard', 'Close the page'). The baseline agent immediately fails: it hallucinates a non-existent route (buzz.co/spec-reviewer), returns a false 'failed' verdict, and cannot even save a screenshot correctly. Hauser argues this is expected — public MCP servers must be generic and cannot know your architecture.

His five-part framework addresses this systematically. First, tool curation: filter out tools irrelevant to your use case. For the spec reviewer, drag, resize, and run-code-in-browser tools are excluded, reducing the tool set from 21 to 16 and shrinking context window load. Second, tool wrapping: replace shallow descriptions with use-case-specific guidance. Hauser's wrappers instruct the agent to always call the accessibility snapshot tool (which returns a text representation of all buttons and menu items) before clicking or hovering, significantly improving navigation reliability. Third, deterministic guardrails: intercept tool invocations before execution to enforce constraints. The example guards the screenshot tool against rogue file paths — if the agent attempts to save an image outside the designated screenshots directory, the call is blocked and the agent receives a structured error message explaining where it must save files, causing it to self-correct on the next attempt rather than crashing the workflow. Fourth, tool composition: synthesize new domain-specific tools from existing primitives. Hauser creates a dedicated 'evidence screenshot' tool whose description ties directly to the flow's final step, prompting the agent to name files with the relevant ticket number and behave differently than it would with a general screenshot. Fifth, treating tools as deterministic functions: pull sensitive, always-required, or structurally complex steps entirely out of the agentic loop. Login to Baz — which involves injecting JWT tokens into browser local storage — is executed deterministically before handing control to the agent, eliminating a common failure vector and keeping authentication secrets out of the agent's context.

With all five improvements applied, the V4 agent successfully navigates to the agents tab, identifies the configuration drawer, captures an evidence screenshot with a ticket-referenced filename, and returns a correct 'passed' verdict. Hauser notes pixel-perfect validation is handled in the production product through additional tooling beyond this toy example. He frames the overall approach as context engineering plus deterministic guardrails — not a fixed recipe, but a repeatable debugging framework: profile tool behavior, identify friction points, and iteratively mold third-party tools to your architecture rather than adapting your architecture to theirs.

Key Takeaways

•Public MCP servers have generic, shallow tool descriptions by design — always wrap them with use-case-specific guidance rather than using them out of the box.
•Curate your tool list aggressively: removing irrelevant tools reduces context window load and lowers the chance of the agent picking the wrong tool.
•Intercept tool calls before execution to enforce deterministic guardrails; return structured error messages (not exceptions) so the agent can self-correct rather than crashing.
•Use the Playwright accessibility snapshot tool before click/hover calls — it returns a text map of all interactive elements and significantly improves navigation reliability.
•Compose new domain-specific tools from existing MCP primitives to give the agent different behavioral cues in different flow contexts (e.g., a dedicated 'evidence screenshot' tool vs a general screenshot tool).
•Pull always-required, structurally complex, or security-sensitive steps (like authentication) entirely out of the agentic loop and execute them deterministically.
•For multi-tenant architectures, enforce folder/schema boundaries with deterministic path validation — do not rely on prompt instructions alone, as agents can and do ignore them.
•Agent improvement is iterative: profile V0 failures, identify whether the issue is tool selection, description quality, or missing guardrails, then address each gap in order.
•A failing V0 baseline is a feature, not a bug — it gives you a concrete target to optimize against and demonstrates the value of each subsequent improvement.
•There is no single optimal strategy; some improvements shrink context (curation), others expand it (verbose descriptions) — the goal is the best fit for your specific use case.

Transcript

Hi everyone, and welcome to our talk today about bending a public MCP server without breaking it. But today we may have just broken it because our MCP server seems to have caught on fire. We're glad you're here. We need all the help we can get. Let's go through our talk and see how we can improve whatever is going on right here. I'm Hauser. I work at Buzz. We've been building AI powered code reviewers for the past few years now, as well as a bunch of other features, anything that can can make th...