Combine Skills and MCP to Close the Context Gap — Pedro Rodrigues, Supabase

May 15, 202618:2710,210 views

Summary

Pedro Rodrigues, AI tooling engineer at Supabase, presents findings from building and evaluating Supabase's official agent skill, arguing that MCP servers and skills are complementary rather than competing technologies. The talk reframes the debate: MCP provides tools and environment access, while skills provide the guidance and domain knowledge agents need to use those tools correctly in production contexts.

Rodrigues opens with a concrete security failure that motivated Supabase's work: when given only an MCP server and asked to create a SQL view over a table with row-level security (RLS) enabled, Claude Sonnet 4.6 silently bypassed the RLS by omitting the required `security_invoker = true` flag. The agent was not incapable — it simply lacked knowledge of this Supabase-specific security constraint. The same agent equipped with a skill containing that security rule implemented the view correctly. This asymmetry, between general reasoning capability and environment-specific constraint knowledge, is the central problem the Supabase skill is designed to close.

The team ran structured evaluations using Braintrust, testing six real Supabase project scenarios across four agents from two vendors: Claude Code with Opus 4.6 and Sonnet 4.6, and OpenAI Codex with GPT-5.4 and GPT-5.4 mini. Each model was evaluated in three conditions — baseline (no MCP, no skill), MCP only, and MCP plus skill — scored on a graded test-completeness metric. The results were unanimous across all models: MCP plus skill outperformed both the baseline and MCP-only conditions. The performance gain was consistent regardless of model vendor or tier.

From the process of writing the skill, Rodrigues distills three principles. First, do not duplicate documentation — skills should point agents to the authoritative, up-to-date source of truth rather than maintaining a parallel copy. Supabase is experimenting with exposing their documentation over SSH so agents can navigate it as a file system using familiar Linux-based tools, enabling potential augmentation with semantic search over the doc corpus. Second, if something can be skipped, it will be skipped — agents are reluctant to make expensive tool calls or load reference files, and will rarely load more than one reference file per task. Security-critical constraints that agents must not miss should live directly in skill.md, not in bundled reference files. Supabase learned this the hard way by initially placing their RLS security checklist in a reference file, observing that agents routinely skipped it, and moving it directly into the skill body. Third, be opinionated — skill authors know their product's optimal workflows better than any general-purpose agent. Supabase encodes a specific schema management workflow in their skill: perform DDL changes freely on development or staging, run Supabase's schema advisor to catch security and performance issues, fix those issues, and only then generate the migration file. This prevents the agent from creating a migration file on every schema change.

On distribution, Rodrigues acknowledges a significant unsolved problem: there is no standard registry or package manager for skills. The current approach at Supabase and elsewhere is to package skills inside repositories as `.claude` or `.cursor` plugin directories. Vercel's skills package offers one distribution mechanism, but it is model-specific, and the ecosystem has not converged on a general solution.

The talk closes with a practical summary for skill authors: start minimal, iterate, point to a single source of truth, and be willing to be opinionated about workflows.

Key Takeaways

•MCP alone is insufficient for production reliability — agents need skill-based guidance to avoid environment-specific pitfalls like the Supabase RLS security_invoker bypass.
•Combining MCP with a skill outperformed MCP-only across all tested models (Claude Opus 4.6, Sonnet 4.6, GPT-5.4, GPT-5.4 mini) in Braintrust evaluations measuring test completeness.
•Security-critical constraints must go directly in skill.md — agents reliably skip reference files and will rarely load more than one even when explicitly bundled.
•Do not duplicate documentation in skills; instead, point agents persistently to the canonical, living documentation source and instruct them to fetch it rather than relying on training data.
•Be opinionated in skills: encode the optimal workflow for your product explicitly (e.g., Supabase's DDL → advisor → migration sequence) to prevent agents from inventing suboptimal patterns.
•Agents default to training data over tool calls because fetching information is expensive — skill authors must assume agents will take the lazy path and design accordingly.
•Starting a skill minimal and iterating is universally recommended; avoid trying to achieve comprehensive coverage from the outset.
•Exposing documentation over SSH so agents can navigate it as a file system is a promising pattern for keeping agent knowledge current without duplicating content.
•The skill distribution problem is unsolved — current best practice is packaging skills inside repositories as .claude or .cursor plugin directories and using tools like Vercel's skills package for sharing.
•Evals (document-level tests run on CI via tools like Braintrust) are now a practical way to test and iterate on agent skill quality across models.

Transcript

All right, I get I have the green light so we can get it started. Hello everyone. You might have noticed that the the titles changed a bit from the one that we have in on the schedule. That's because when I submitted the talk, there's the MCP versus skill debate was still going on was a topic. I think now we settled on they're both different. They all have their own roles and now the I think the debate is more on the MCP versus CLI. So I thought it would be more useful to come and explain how we...