Beyond Code Coverage: Functionality Testing with Playwright MCP — Marlene Mhangami, Microsoft

Marlene Mhangami, Senior Developer Advocate at Microsoft and GitHub, delivers a talk at AI Engineer Europe 2026 arguing that AI-generated tests create a false sense of security — and that flipping the standard AI coding workflow to adopt test-driven development (TDD) with Playwright is the cure.

Mhangami opens with data from GitHub's Octoverse 2025 report: over one billion commits were pushed to GitHub in 2025, the platform's most active year ever. By 2026, the growth is accelerating further — their COO Kyle Daigle tweeted that GitHub is seeing roughly 275 million commits per week, extrapolating to approximately 14 billion commits by year-end, a 14x increase year-over-year. A growing share of those commits are co-authored by AI agents, traceable via co-sign patterns from tools like Claude and Copilot.

This explosion in AI-generated code raises a productivity question. Mhangami references a Stanford University study of 120,000 developers, surfaced in an AI Engineer talk from the prior year, which found that clean codebases amplify AI productivity gains, while unchecked AI amplifies entropy. One case study from the research showed a team whose PR output increased under AI usage, but whose code quality simultaneously fell — requiring so much rework that net productivity improvement was only around 1%. The key implication: AI value is gated on codebase cleanliness, requiring good test coverage, type coverage, documentation, and modularity.

The talk then addresses a specific failure mode of AI-assisted development: self-affirming tests. When an LLM writes tests after writing implementation code, it tends to confirm what the code does rather than what the user should experience. Test suites go green while real behavior remains unverified.

Mhangami advocates for behavior-driven TDD using the red-green-refactor cycle, specifically citing Simon Willison's recent blog post on this approach. The discipline: write a failing Playwright test the moment a feature request arrives, then generate code to make it pass (without worrying about quality in the green phase), then refactor. She also references Ian Cooper's talk 'TDD Where It All Went Wrong' for deeper context on why unit-test-focused TDD often fails by over-indexing on implementation details rather than observable behavior.

Playwright, Microsoft's open-source end-to-end testing framework, is positioned as the ideal behavioral test layer. It supports Python, TypeScript, and C#, automates browser interactions, and can run in headed or headless mode. There are three integration paths for AI agents: the Playwright MCP server, the Playwright CLI tool, and Playwright Agents. The Playwright Agents approach installs three agent markdown files — a planner, a generator, and a healer — that respectively plan test execution, generate tests, and self-heal failing tests.

The live demo walks through GitHub Copilot CLI connected to the Playwright MCP server, building a search and filter feature for a fictional toy store app called Tailspin Toys. Mhangami uses a Microsoft skill called Work IQ to pull the feature request email from Outlook directly into the terminal, then prompts Copilot to implement the features using red-green TDD: write failing Playwright tests first, then generate passing code. The browser is visible as Playwright types search terms ('Furby'), clicks category filters, and validates price range results — all autonomously.

Best practices offered: attach Playwright screenshots to every PR; prefer headless mode in CI; always commit before letting the agent modify code to preserve history; and scope one Playwright test per feature to keep failures isolated. For state-heavy apps, Playwright Agents (with its specialized agent.md instructions) handles complexity better than raw MCP server usage. Playwright also supports cross-device testing — the same tests can validate both desktop and mobile viewports.