Building a Chess Coach — Anant Dole and Asbjorn Steinskog, Take Take Take

Anant Dole and Asbjørn Steinskog from Play Magnus (Magnus Carlsen's chess app, also known as Take Take Take) present their AI chess coaching system and the autonomous feedback loop they built around it at AI Engineer Europe 2026. The talk covers architecture decisions, latency trade-offs, and a live demo of Claude Code operating as an autonomous debugging agent.

The core architectural insight is a strict separation of concerns: Stockfish, the leading classical chess engine, handles all position evaluation and best-move calculation. A suite of tactical and positional detectors extracts human-readable concepts from those evaluations — identifying forks, pins, skewers, doubled pawns, structural weaknesses, threats, and defensive resources. A novel engine called Maya, developed by the University of Toronto, adds a complementary signal: trained on millions of human games keyed by Elo rating, it outputs a probability distribution over all legal moves for a given rating level (e.g., 1500), allowing the system to quantify how hard a move is for humans to find. All of this structured context is then passed to an LLM, whose sole role is translation into natural English. The speakers emphasize that the LLM is deliberately kept as a translator, not a reasoner — when given free rein, LLMs rapidly hallucinate in chess positions because they cannot perform the sequential calculation that chess requires. The demo shows generated commentary explaining why a move is brilliant: it mentions the tactical theme, the specific threat created, and the defensive response the opponent missed.

For latency, the team targets sub-3-second end-to-end response because users want near-instant game review. Gemini 2.0 Flash is their production model: time-to-first-token is approximately one second, and average end-to-end latency is around three seconds. Reasoning models (Claude with extended thinking) scored about 59% on their internal eval suite but with unpredictably long latencies — unsuitable for the instantaneous review UX. GPT-4o mini offered lower latency but also lower accuracy. A planned "chat with your coach" mode will accommodate longer wait times and is where reasoning models may become viable.

Evals consist of 16 scenario benchmarks extracted from real games, covering tactical patterns (knight forks, pins), blunder detection, and hallucination suppression. LLM-as-judge is used for automated scoring. Models are tested via OpenRouter, which lets the team quickly swap in new model versions (Gemini, Claude, GPT-5) as they are released. Gemini Flash currently reaches approximately 75% pass rate on these scenarios; no model has cleared all 16.

The feedback loop closes via Claude Code Channels, described as a research-preview MCP feature that allows external events to be injected into a running Claude Code session. When a user flags bad commentary in the app, the event is posted to a Slack channel and simultaneously injected into the live Claude Code session. Claude Code invokes a custom "commentary triage" skill that outlines a structured investigation process: it runs internal scripts to re-generate commentary, can modify prompts or detectors, creates new detectors if needed, verifies its own output, and posts clarifying questions back to Slack. During the live demo, Anant received a Slack message from Claude Code asking what specifically felt wrong about the flagged commentary, then resolved the investigation from his phone and could have merged the resulting PR directly from mobile. Key learnings the team shares: separate data pipeline from language generation; close the feedback loop with autonomous agents; build a detailed structured context extraction layer iteratively (start large, prune over time); use automated evals anchored to domain experts; and partner with subject-matter experts (both speakers are strong chess players) for ground-truth quality judgments.