Feedback Loops are All You Need — Mehedi Hassan, Granola

Mehedi Hassan, a product engineer at Granola, delivers a candid account of what shipping AI features into production actually looks like — beyond playground demos. Granola is a meeting notes application that records real-time transcription by accessing system audio and microphone input, then generates structured notes at the end of meetings. The talk centers on a chat feature Granola built that allows users to ask questions across meetings, and the hard lessons learned in making it work reliably at scale.

Hassan opens by demonstrating Granola's core product: it transcribes live meetings and produces notes that blend the AI-generated summary with the user's own handwritten annotations, prioritizing user-written content in the final output. He then describes the seductive simplicity of 'one-shotting' an AI feature — web search as a single tool call, one generic prompt for all users — and why that illusion collapses in production.

On web search, Hassan explains that adding a web search tool looks like one line of code, but reality introduces three compounding problems. First, complex queries cause token usage and context length to balloon, with individual chats costing around 10p each — unworkable at millions-of-users scale. Second, LLM providers control which web search backend is used, and Granola experienced a silent overnight model update that degraded web search quality with no warning and no recourse beyond switching providers. Third, web search as a category has billion-dollar companies dedicated to it, which signals its complexity far exceeds what a simple tool call abstracts.

On prompt personalization, Hassan argues that a single prompt genuinely cannot serve Granola's diverse user base. A salesperson wants deal-focused summaries; an engineer wants blockers and Linear ticket suggestions; an HR manager wants something else entirely. LLMs are, in his words, 'stubborn,' and making them behave differently for different roles requires more than prompt tweaking.

Granola's response was to build the machinery for iteration rather than trying to prompt their way out of the problem. They built custom internal tracing tooling — described as a one-shot build enabled by LLMs themselves — that captures the full agent loop: individual tool calls, reasoning traces, search trails, and per-request cost, all visualized in a UI designed for the entire company, not just engineers with CloudWatch access. The Granola founder uses this tool to trace agent runs front-to-back to diagnose failures. The stack is intentionally simple: events saved to a database, a wrapper around the AI SDK, and a purpose-built frontend. Hassan notes that OpenTelemetry or third-party SaaS observability tools are valid alternatives, but the internal tool gave them the exact data shape and UI their team needed.

The second major infrastructure investment was refactoring Granola's Electron desktop app so its frontend could run as a web app. By abstracting Electron's IPC system APIs to fall back to web standards, and moving React routing, sessions, and query layers to web standards, they made the renderer Electron-agnostic. This unlocked CI preview links: every pull request now gets a deployed preview URL. More significantly, it enabled Cursor to automatically test PR changes, navigate the app, take screenshots, and upload them directly into pull request threads — dramatically accelerating feature verification and variant testing that was previously blocked by the one-instance constraint of desktop apps.

Hassan closes by framing the talk's thesis: the answer is not to one-shot better. It is to build the feedback loop — custom tracing for observability and a web-deployable frontend for rapid iteration — so that developing an AI feature feels like a tight tennis rally with the LLM rather than deploying into a black box and hoping for the best. The goal is shipping AI features with conviction that they will actually connect with users.