Full Workshop: Build Your Own Deep Research Agents - Louis-François Bouchard, Paul Iusztin, Samridhi
Summary
This hands-on workshop, led by Louis-François Bouchard (CTO, Towards AI), Samriddhi Vaid (ML engineer and technical writer), and Paul Iusztin (LLM Engineers Handbook author), teaches participants how to build a production-grade MCP-powered deep research agent connected to a constrained technical writing workflow. The system was developed out of genuine internal need at Towards AI, where producing high-quality technical AI content requires senior engineers, editors, and iterative review — making it expensive and time-consuming.
Bouchard opens with a foundational framework for AI engineering design decisions, framing every architecture choice around an 'autonomy slider' from simple prompting to multi-agent systems. He emphasizes that most client requests for multi-agent systems are actually better served by single-agent or simple workflow designs, illustrated by a real CRM client case where a multi-agent plan was simplified to one agent with specialized tools. He introduces the concept of context rot — the performance degradation that occurs well before the 1-million-token limit (practically around 200,000 tokens) due to the 'lost in the middle' problem in long-context training.
The deep research agent is built with FastMCP and exposes three tools to Claude Code acting as the agent harness and MCP client: (1) a deep_research tool that calls the Gemini API with grounding to return cited answers and structured source URLs; (2) an analyze_youtube_video tool that passes YouTube URLs as file URIs to Gemini's multimodal API, which watches the video in real time rather than pulling a transcript — taking 2–3 minutes per video; and (3) a compile_research tool that reads from a shared .memory folder and writes a final research.md artifact. All intermediate outputs are persisted to disk in the .memory directory for traceability and debugging. The agent architecture separates the MCP server (capability layer) from the LLM (reasoning layer), so the agent dynamically decides which tools to call and iteratively fills research gaps by issuing follow-up queries. Model selection settled on Gemini Pro/Flash over Anthropic Claude and OpenAI models primarily because of native YouTube URL support, good quality-cost tradeoff, and a free tier for students.
The writing workflow is deliberately non-agentic — it uses a deterministic pipeline because writing quality benefits from constraints, not autonomy. The workflow loads three context inputs: (1) a user-authored guideline.md (topic, angle, key points, narrative flow, character count override), (2) the research.md artifact, and (3) static writing profiles for structure, terminology (a curated banned-words list covering 'delve,' 'vibrant,' 'leverage,' and similar AI-slop terms), and character voice. Few-shot examples consist of three high-quality representative LinkedIn posts, chosen to balance coverage with context window efficiency. The team advises starting with more examples and trimming downward until quality degrades.
The evaluator-optimizer loop runs the writer and reviewer in separate context windows to eliminate self-bias, iterating three to four times. The reviewer outputs structured Pydantic objects with profile, location, and comment fields pointing to specific violations rather than freeform critique. Priority ordering (guideline > research > profiles) resolves conflicts when reviews clash on the same paragraph. Posts are versioned (v0 through v4) so users can pick the preferred draft.
Observability is handled by Opik, which captures full traces per workflow run including LLM calls, tool calls, token usage, latency, and cost per model. For evaluation, the team built a labeled dataset of 20 real LinkedIn posts with reverse-engineered guidelines and research inputs, binary pass/fail labels, and two-to-three sentence critiques. An LLM judge — implemented as a binary classifier with few-shot examples drawn from a train split — is calibrated against dev and test splits using F1 score, tracking relative agreement between splits to detect overfitting rather than relying on absolute scores. The live demo showed perfect F1 on both dev and test splits (5 samples each), with the team acknowledging that expanding to 20–30 samples is recommended for production use.
Key Takeaways
- •Default to the simplest architecture: if a deterministic workflow covers the use case, avoid agents — multi-agent complexity rarely pays off for sequential, predictable tasks.
- •Context rot begins around 200,000 tokens due to lost-in-the-middle training limitations; manage context budgets actively by trimming, summarizing, or delegating to sub-agents.
- •Separate exploratory research (needs agentic flexibility) from content generation (needs deterministic constraints) into two independent systems communicating through shared files.
- •Pass YouTube URLs as file URIs to Gemini's multimodal API to get real-time video analysis without needing a separate transcription pipeline.
- •Use Gemini grounding API instead of Perplexity for web searches — it returns structured sources with URLs, title, and snippets that can be scraped for deeper content.
- •Run the LLM reviewer in a completely separate context window from the writer to eliminate self-bias in the evaluator-optimizer loop.
- •Output structured Pydantic review objects with explicit field descriptions as a prompt engineering technique; it outperforms freeform critique in directing the editor.
- •Build an eval dataset from real system outputs — never ask an LLM to generate synthetic labels or outputs you intend to measure; always generate inputs synthetically, outputs from your real system.
- •Calibrate an LLM judge by tracking F1 score alignment between dev and test splits, not absolute score values — large dev/test divergence signals overfitting.
- •Agent skills (Claude Code .md skill files) offer progressive disclosure — only the front matter is loaded until the skill is invoked, keeping context lean and workflows shareable.
Transcript
Hello everyone. Not too loud. I hope Yeah, I hope it's fine. Perfect. Uh so, yeah, we'll start to kick off the workshop and we'll introduce ourselves shortly. But first, we just wanted to present the slide because that's basically LinkedIn this year. It's the type of content that when you ask ChatGPT, that's basically what you get. A very generic response and there there are a few things wrong with this actual response or that we don't really like seeing. The obvious ones are the the slop words,...
Topics
Related Sessions
