10 meta-themes that emerged independently across multiple conference tracks.
The conference showed a decisive pivot from prompt engineering to context engineering — strategically curating what information enters an agent's context window, when, and in what form. Practitioners now treat context as the primary lever for reliability, not model capability or prompt wording. This shift touches RAG, tool design, skills architecture, memory, and harness construction.
Implications: Practitioners should invest in context architecture before model selection. The key primitives are: progressive skill disclosure (load context on demand, not upfront), external memory stores so agents retrieve rather than carry history, tool descriptions as routing rules not documentation, and per-session context budgets. Evaluate context quality independently from model quality — they fail for different reasons. Organizations building agent platforms should treat context packaging, versioning, and registry as first-class engineering concerns.
•Agentic Search for Context Engineering (Monigatti, Elastic) explicitly framed 'context engineering is approximately 80% agentic search' and demonstrated progressive skill disclosure as the fix for tool-parameter failures.
•Agentic Engineering (O'Leary, Kilo Code) warned that context window quality degrades noticeably above 50% utilization and identified context poisoning from stale or mixed tasks as a primary failure mode, prescribing the research-plan-implement loop.
•How We Solved Context Management in Agents (DeLucia, Arize) documented a production vicious loop — spans growing until context limits were hit — and resolved it with head-plus-tail truncation plus a retrievable memory store.
•Context Is the New Code (Debois, Tessl) introduced a full Context Development Lifecycle (CDL) with linting, eval, packaging, registry, and observability stages, arguing context is now the primary creative input.
•Mergeable by Default (Werry, Unblocked) benchmarked a 6x speed and 52% token reduction by adding a context engine, avoiding the 'satisfaction of search' failure mode in naive RAG.
•Mario Zechner (Building Pi) argued long context windows are merely a hack that most teams will discover is insufficient this year, suggesting the industry may be over-indexing on context as the solution.
•Some sessions (Bounded Autonomy, Oliver) argued that the real answer is minimizing context rather than engineering it — using bounded autonomy and curated documentation over broad retrieval.
Across dozens of sessions, practitioners converged on a shared insight: the reliability of an AI agent is primarily determined by its harness — the scaffolding of guardrails, verification steps, retry logic, and context management that surrounds the model — not by the model itself. The harness abstracts model weakness, enforces constraints, and catches failures before they propagate.
Implications: Teams should invest in harness components before adding model capability: max-iteration caps, deterministic verify steps, cryptographic proof of task completion (not model assertion), external state stores, and reviewer agents triggered on CI push. The harness should be model-agnostic — constraints implemented as tests and lints produce acceptable output regardless of which model runs. Every synchronous human intervention point is a harness failure to fix. As models improve, prune harness components that address solved limitations.
•Harnesses in AI (Kumar, IBM) demonstrated that GPT-3.5 Turbo (a 2023 model) with a well-engineered harness completed a Hacker News upvote task reliably by adding max-iteration guardrails, context compression, a deterministic verify step, and a programmatic login handler — without modifying the prompt.
•Anthropic Workshop (Prabaker & Wilson) introduced a GAN-inspired generator-evaluator architecture where the evaluator uses Playwright MCP to open live pages and score output, catching bugs the generator would never self-identify, achieving fully functional apps from vague one-line prompts.
•Harness Engineering (Lopopolo, OpenAI) defined the discipline as 'surfacing the right instructions to the model at the right time via failing tests, lint errors with actionable remediation steps, and reviewer agents triggered on every push,' and explicitly said 'every time a human must type continue to an agent is a harness failure.'
The Model Context Protocol crossed a critical inflection point at this conference — moving from protocol novelty to enterprise deployment reality, with the accompanying discovery of production complexity: context window explosion from large tool sets, authentication gaps, security vulnerabilities, observability voids, and the need for gateway infrastructure. Multiple sessions independently diagnosed the same set of problems and proposed complementary solutions.
Implications: Organizations adopting MCP at scale need gateway infrastructure before individual server proliferation. The gateway provides the root of trust for auth, enables scoped access control, centralizes observability, and decouples agent harnesses from data layers. For large API surfaces, code mode (LLM writes typed code against the API) is dramatically more token-efficient than one-tool-per-endpoint. Tool descriptions are routing rules — invest in tool calibration evals. Progressive discovery (load tools on demand via search) is the scaling pattern for tool-rich environments.
•MCP = Mega Context Problem (Carey, Cloudflare) quantified the core problem: 2,600 API endpoints naively converted to MCP tools produces 1.2 million tokens — completely impractical. Proposed solution: two tool calls accepting code strings (search + execute) reducing context to ~1,000 tokens.
•Gateways are All You Need (Sampath, Anthropic) identified the 'three-headed hydra' blocking enterprise MCP deployment: observability (no visibility into tool usage), access control (no scoped permissions), and security (no root of trust), and proposed an MCP Gateway as the mandatory infrastructure layer.
•One Login to Rule Them All (Galow, WorkOS) addressed the OAuth consent proliferation problem with Cross-App Access (XAA), demonstrating silent SSO delegation to MCP servers using short-lived tokens that expire on IDP session revocation — critical for enterprise security.
Evaluation of LLM systems evolved from academic benchmark comparisons and offline golden datasets into a continuous, production-embedded engineering discipline. Sessions across the conference diagnosed eval failure modes, proposed new architectures (adversarial evals, self-curating test suites, production trace-based evals), and demonstrated quantified eval-driven improvement loops in live systems.
Implications: Production eval pipelines should be built as continuous services running against live traces, not offline batch jobs against golden datasets. LLM-as-judge must be calibrated against human annotations using precision/recall measurement, not deployed off-the-shelf. Eval harnesses should be accessible to domain experts (clinicians, lawyers) who can annotate trace quality in natural language. Error budgets (multiple eval passes, percentage-based gates) are more appropriate than binary pass/fail for non-deterministic systems. Self-diagnosing agents (a 'report' tool that fires before final answer) provide cheap, high-signal production monitoring.
•Agent Optimization with Pydantic AI (Colvin) demonstrated GEPA lifting accuracy from 87% (manual prompt) to 92% (expert-crafted prompt) to 96.7% (genetic algorithm) on a 65-case golden dataset, while explicitly warning about overfitting narrow test distributions and model-specific prompt lock-in.
•Agentic Evaluations at Scale (Kang & Aaron, Google DeepMind/Kaggle) diagnosed three systemic failures: scattered/stale benchmarks, opaque/non-reproducible evaluations, and dangerously narrow benchmark coverage from 30,000 researchers for 30 million technical practitioners.
•Malleable Evals (Koc, OpenClaw) argued for 'eval calcification' as the core problem — static benchmarks are incompatible with adaptive agentic systems — and proposed self-curating test suites that mine agent traces and intent-based outcome definitions.
Multiple case studies from production engineering organizations documented not incremental productivity gains but structural transformation: PR throughput doubling, Scrum ceremonies eliminated, CI/CD pipelines melting under volume, and engineering roles shifting from code-writer to context-director and factory-operator. The conference treated organizational change as a technical problem.
Implications: Engineering organizations should treat agentic adoption as a systems engineering problem, not a tooling adoption problem. The bottleneck shifts from implementation capacity to alignment capacity, review capacity, and context management capacity. CI/CD infrastructure must be redesigned for agent-scale commit volume before deployment. Engineer profiles matter: curious self-directed engineers succeed; prescriptive engineers who rely on detailed specifications struggle. Progressive rollout (starting with engineers who have deepest system knowledge, in non-critical systems) outperforms company-wide hackathon-style launches.
•Agents Don't Do Standups (Spitz, PFF) reported a 25x increase in deployment frequency, 10x output by code complexity weighting, and CSAT rising from 7.5 to 8.6 — with standups, sprint planning, sprint refinement, and retrospectives all eliminated, replaced by a QA agent and ticket auto-updates from PR state.
•How Building with AI Can Double the Throughput (Scanlan, Intercom) documented doubling PR throughput in under a year, 17.6% of PRs now auto-approved with SOC 2/ISO 27001/HIPAA compliance, and CI infrastructure melting under the surge — treated as a positive signal.
•CI/CD Is Dead (Santos & Faulkner, Namespace) argued that traditional CI/CD is architecturally incompatible with agent-scale development: runner saturation, cache thrash, and merge queue serialization make time-to-commit the new bottleneck, requiring stateful execution environments and intent-plus-outcome review.
Security threats against AI agents were shown to be more sophisticated, more numerous, and more production-relevant than the industry's current defensive posture assumes. The conference documented six distinct attack vector classes, demonstrated novel multi-agent attack patterns, and argued that sandboxing is necessary but insufficient — comprehensive security requires zero-trust design at every layer.
Implications: Production agent security requires layered defense: encoder-model classifiers (ModernBERT fine-tuned on InjectGuard) for prompt injection at 35-40ms latency, explicit separation of instruction space from data space and forbidden-action space, OS-level deterministic execution boundaries (Fence-style) for probabilistic classifier failures, capability-based sandboxes with zero-capability defaults, spend-scoped credentials for any economic actions, and security logging of agent reasoning (not just actions). The MCP tool-description asymmetry is a structural protocol issue requiring human review of full tool descriptions before deployment.
•$1 AI Guardrails (Carpentero) catalogued six production attack vectors: direct prompt injection, indirect context injection (via web pages, emails), GCG gibberish token attacks that transfer to closed models, RAG poisoning (5 chunks in 8M-document corpus force target answers), MCP tool-description asymmetry exploits (hidden instructions in tool descriptions), and agentic escalation (coding agents installing malicious NPM packages from GitHub issues, affecting 4,000-5,000 developers in February 2026).
•Building safe Payment Infrastructure (Kaliski, Stripe) addressed the economic attack surface: four categories of agent transaction risk (buying from the wrong place, buying the wrong thing, spending the wrong amount, using the wrong credential), proposing shared payment tokens scoped to specific sellers with spend caps as the structural fix.
Building Agent Interfaces (Hablich, Google Chrome DevTools) introduced a three-tier browser agent security model and explicitly refused to persist allow choices for the autoconnect feature, citing Simon Willison's 'lethal trifecta': personal data access + untrusted content + communication ability.
As agents are deployed on tasks lasting hours and days rather than seconds, the absence of robust memory architectures, persistent state management, and context compaction strategies has emerged as a first-class production problem. Sessions across the conference independently arrived at similar solutions — file-based state, retrievable memory stores, knowledge graphs — while acknowledging significant open problems remain.
Implications: Production long-running agents require explicit memory architecture: external state stores (JSON files on disk rather than markdown, which agents overwrite), tiered memory with promotion mechanisms, timestamped breadcrumb JSONs for future sessions, and retrievable memory stores with agent-invocable retrieval tools. Knowledge graphs provide the best structure for reasoning traces and decision provenance in regulated domains. Context compaction strategies should be version-specific — as models improve their context self-awareness, compaction components should be pruned. Demand-driven context construction (agent failure as knowledge discovery) is more reliable than top-down knowledge base curation.
•Agents Need More Than a Chat (Lauritzen, Legora) named 'context rot' as the fundamental failure mode of long-running legal agents: a 30-minute run produces a contract, a correction triggers context compaction, and the revised output silently changes far more than requested.
•Anthropic Workshop (Prabaker & Wilson) traced the evolution from the Ralph Loop (plan.md + stop hooks) through Claude Code Agent SDK to Opus 4.6 with server-side compaction and 1-million-token context GA — showing harness components being pruned as model memory improves.
•I Gave an AI Agent the Keys to My Life (Sienkiewicz) built a production personal agent with soul.md, critical-rules.md, and a dreaming mechanism for memory promotion, running overnight cron maintenance including vault indexing and QMD refresh — showing practical tiered memory management.
The conference documented a qualitative shift in on-device AI capability: models with billions of parameters running at usable speeds on consumer phones, laptops, and IoT hardware, with native function calling, structured output, and multimodal capabilities. The edge-vs-cloud decision is no longer latency vs. capability; it is now a nuanced engineering choice with well-defined tradeoffs.
Implications: On-device AI is no longer a research project — it is a production deployment option for applications requiring low latency, privacy, offline operation, or cost reduction. The engineering decision framework: use system-level GenAI (Gemini Nano via AI Core, Gemma via LiteRT) for zero-bundle-overhead apps with broad capabilities; use app-level fine-tuned models (Function Gemma, LFM2.5) for high-accuracy task-specific agents on constrained hardware. Fine-tuning on synthetic data generated by frontier models is the practical path to 90%+ accuracy at sub-billion-parameter scale. NPU acceleration (Qualcomm, MediaTek) should be profiled before claiming edge latency numbers.
•Accelerating AI on Edge (Parikh, Google DeepMind) reported LiteRT running up to 35x faster than llama.cpp on mobile, achieving ~56 tokens/second on iOS and 3-10x performance improvement from NPU acceleration, with Gemma 4 E2B/E4B supporting native function calling and chain-of-thought on-device.
•From 46% to 90%: Fine-Tuning Tiny LLMs (Brick, Google) showed Function Gemma (270M parameters) achieving 46% to 90%+ accuracy on app intents after fine-tuning with synthetic data, at 2,000 tokens/second prefill on a Pixel 7.
•Running LLMs on your iPhone (Grondin, Locally AI) demonstrated Gemma 4 at 40 tokens/second on iPhone via MLX, with 4-bit quantization maintaining output quality and the MLX Community publishing quantized versions within 30 minutes of new model releases.
A counterintuitive but strongly argued thread ran through the conference: classical software engineering fundamentals — Domain-Driven Design, test-driven development, deep modules, architecture decision records — have become more important, not less, in the age of AI code generation. Poor software architecture blocks teams from capturing AI productivity gains, while good architecture compounds them.
Implications: Teams should audit their codebase for agent legibility before expanding agent scope: unique function names (no ambiguous naming), consistent module boundaries, deep modules with simple interfaces, and a ubiquitous language documented in markdown and fed to agents at planning time. ADRs should be machine-enforceable via CI rather than advisory documents. TDD is not optional for AI-assisted development — without tests, agents outrun their headlights. The 'gray box' pattern (engineer designs interface, agent implements internals, tests run against boundary) is the optimal division of labor.
•Software Fundamentals Matter More Than Ever (Pocock) argued that 'bad code is the most expensive it has ever been' because a hard-to-change codebase prevents capturing AI productivity gains; introduced six concrete failure modes of AI coding and their classical software engineering solutions (DDD ubiquitous language, TDD, deep modules, shared design concept).
•Can LLMs Generate Enterprise Quality Code? (Sarkar, Sonar) empirically demonstrated that across 53 models on 4,444 Java assignments, all models with 80%+ functional correctness still have meaningful security, maintainability, and complexity problems — and that newer models generate progressively more verbose, harder-to-review code.
•BDD, ADR, PRD, WTF (Cichra) showed that machine-enforceable specifications (ADRs with git hook rejection, BDD with Cucumber, module import linting) survive context compaction where prompt-based rules do not, enabling 20-50 context compacts per session without degradation.
A consistent thread across talks from Vercel, Cloudflare, Anthropic, GitHub, and practitioners argued that the primary consumer of APIs, tools, and software interfaces is shifting from human users to AI agents — requiring a fundamental rethinking of interface design, documentation structure, and distribution. Chat UIs are transitional; the permanent form is agent-native interfaces.
Implications: API and platform teams should audit developer experience for agent consumers: LLMs.txt and llms-full.txt for documentation discoverability, CLI interfaces for API access (agents excel at shell tools), MCP servers with well-calibrated tool descriptions and progressive discovery, schema-validated APIs with actionable error messages that agents can self-correct on, and platform metrics that include agent vs. human traffic separation. For consumer-facing applications, MCP apps (sandboxed iframe UIs returned from tool calls) are the emerging standard for agent-readable interactive surfaces. Design documentation assuming LLM consumers, not just human readers.
•Agents for Everything Else (swyx) cited Malte Ubl's keynote data: 60% of Vercel page views come from AI agents rather than humans, calling MCPs, CLIs, and APIs more important than dashboards, and framing Agent Experience (AX) as the primary design target.
•Code Mode (Pai, Cloudflare) argued APIs should be designed for code-generating agents, not human callers: TypeScript SDK types are a far more token-efficient representation than tool definitions, and 'LLMs dream in types and syntax errors' — agent-oriented documentation (markdown docs, actionable error messages, discoverability via search) is the new DX requirement.
•Building Agent Interfaces (Hablich, Google) introduced 'tokens per successful outcome' as the canonical AX metric and explicitly reframed UX design as evolving into AX design — agents are a distinct user class with non-functional requirements around efficiency, discoverability, security, and stability.
•BDD, ADR, PRD, WTF (Cichra, Safe Intelligence) showed that encoding architectural decisions as machine-enforceable git hooks and CI pipelines — not prompts — produces context-compact-resilient systems: 20-50 context compacts per session caused no problems because the rules live in CI, not in context.
•Building pi in a World of Slop (Zechberg) demonstrated that Terminal Bench's minimal harness (two tools: keystrokes + tmux output) consistently outperforms larger, richer harnesses, and argued agents compound errors with zero natural bottlenecks without structural harness controls.
•Some practitioners (Zakariasson, Cursor; Koc, OpenClaw) operate extremely successfully with minimal harness — 70-80 active Git worktrees, direct conversational interaction — suggesting expert operators can substitute cognitive harness for structural harness at the cost of scalability.
•The rapid improvement in model self-awareness (Sonnet 4.5 context self-awareness, Opus 4.6 eliminating context anxiety) suggests some harness components are temporary compensations for model limitations that are being trained away.
•Scaling GitHub for your Agents (Morrow, GitHub) reported serving 7-8 million MCP tool calls per week and achieving 49% context reduction by redesigning tool sets, with OAuth 2.1 + PKCE deployed in response to widespread long-lived over-privileged PAT token misuse.
•Bending a Public MCP Server Without Breaking It (Hauser, Baz) demonstrated a systematic five-part framework for adapting third-party MCP servers: tool curation, tool wrapping, deterministic guardrails, tool composition, and treating tools as deterministic functions.
•The Arize AI eval (Voss) found MCP used 12 tool calls vs 5 per task and incurred 6x higher cost vs CLI/skills, concluding 'the debate is a false dichotomy' — CLI wins for tools with strong training-data coverage. This suggests MCP's current implementation costs may exceed its value for many use cases.
•DeepMind's KP Sawhney expressed skepticism about MCP as a long-term standard, preferring skills plus CLI interactions for many workflows.
•Judge the Judge (Mabrouk, Agenta AI) demonstrated LLM-as-judge calibration with human annotations lifting accuracy from 61% to 74% on airline policy compliance, using GEPA optimization and Pareto frontier candidate selection.
•How I deleted 95% of my agent skills (Nisi, WorkOS) showed that 95% of generated skills degraded performance — discovered only through eval measurement — and replacing them with 553 lines of handwritten 'gotcha' notes improved pass rates and cut eval time from 68 to 6 minutes.
•The high compute cost of rigorous evals (Kaggle's 400,000 poker hands for statistical significance, Colvin's $20,000/night hedge fund evals) creates a practical floor that most teams cannot reach, suggesting the field may be more eval-aspirational than eval-operational.
•Lawrence Jones (Incident.io) noted that production traces are better eval ground truth than hand-authored golden sets, which implies many existing eval investments are systematically miscalibrated.
•Dark Factory (Koc, OpenClaw) described a single-night refactor producing 2,700 commits touching ~1 million lines of code and modifying 82% of core codebase, running 60-70 concurrent agents between two engineers — framing this as the Industrial Revolution parallel: the bottleneck has shifted from code production to taste and judgment.
•Collaborative AI Engineering (Appleton, GitHub) inverted the productivity framing: implementation collapse means the real bottleneck is now team alignment on what to build, and the missing infrastructure is multiplayer agent environments (ACE) that preserve alignment checkpoints.
•The METR study cited by Gergely Orosz (Pragmatic Engineer) found engineers felt 20% more productive but demonstrated 20% less productivity — suggesting the subjective and objective productivity narratives diverge significantly.
•Maggie Appleton (GitHub) argued that multiplying individual output while ignoring team alignment produces duplicate work, merge conflicts, and surprise features nobody agreed to build — suggesting organizational restructuring must precede tool adoption.
•Architecturally Ready, Behaviorally Unvalidated (Ghita, Accesa) documented production failures invisible to infrastructure monitoring: an interview agent hallucinating a non-existent action returned HTTP 200 while a candidate was trapped in a conversation loop for 40 minutes, visible only through human feedback.
•AIE Europe Keynotes (Steinberger, OpenClaw) reported 1,142 security advisories in five months — 16.6/day, twice curl's lifetime total — and described the 'lethal trifecta' of agentic risk.
•Diego Carpentero's ModernBERT classifier achieves ~85% accuracy at 35ms — sufficient for many applications — but a 15% false negative rate on sophisticated attacks in production remains a meaningful risk surface.
•The diversity of attack vectors (prompt injection, RAG poisoning, MCP exploits, agentic escalation) suggests no single defense is sufficient, yet most organizations are only beginning to deploy first-layer defenses.
•Context Graphs for Explainable AI (Kollegger & Zaim, Neo4j) proposed three-tier agent memory: short-term (conversation state), long-term (organizations, entities), and reasoning traces (policies, prior decisions) — with every decision persisted as a queryable graph node for future agents to inherit as precedent.
•How we solved Context Management in Agents (DeLucia, Arize) documented the evolution from naive truncation (breaks multi-turn reasoning) through LLM summarization (unreliable) to head-plus-tail truncation plus retrievable memory store as the current production-stable solution.
•DeLucia acknowledged true long-term memory is currently absent in their production system — the memory store is session-scoped. Multiple speakers noted cache invalidation hygiene for long-lived memory stores remains an open problem.
•The demand-driven context methodology (Navakoti, IKEA) found that average per-domain context fits within ~96K tokens, suggesting full-context retrieval often outperforms RAG for typical domain scopes — which may reduce the apparent need for sophisticated memory architectures in practice.
•Everything I Learned Training Frontier Small Models (Labonne, Liquid AI) showed LFM2.5 1.2B outperforming GPT-4o-mini against optimal opponents after RL training, and argued small models excel at agentic reasoning with tool access compensating for limited knowledge capacity.
•Gemma 4 Deep Dive (Hardin, Google DeepMind) introduced Per-Layer Embeddings enabling a 5.1B representational-parameter model to operate at 2.3B effective parameters on-device by paging embeddings from flash memory, and the E2B/E4B supporting native audio on mobile.
•The device reach tradeoff remains sharp: ML Kit GenAI APIs require flagship-class hardware from the last 1-2 years, vs. classic ML Kit running across 1 billion+ devices. Many production use cases require supporting older hardware where on-device inference remains impractical.
•Fine-tuning for task-specific accuracy (46% to 90% on Function Gemma) requires significant engineering investment per use case, which may not be economical for applications with broad, general-purpose requirements.
•Harness Engineering (Lopopolo, OpenAI) implemented a 750-package monorepo with strict dependency edges, 350-line file limit enforced by test, one programming language, one ORM, one CI script pattern — and explicitly framed this as making 'model-produced tokens more predictable' rather than standard software engineering.
•Collaborative AI Engineering (Appleton, GitHub) cited a Stanford study of 120,000 developers showing clean codebases amplify AI productivity gains while unchecked AI amplifies entropy; one team's PR output increased under AI but code quality fell, requiring so much rework that net productivity improvement was only ~1%.
•Ramp's Inspect agent now authors 30% of merged PRs, and Linear expects auto-resolved bugs to approach 100% — suggesting that at sufficient agent quality, code quality maintenance may become an agent responsibility rather than a human prerequisite.
•Vincent Koc's dark factory model (2,700 commits overnight, 82% of codebase modified) succeeded despite bypassing conventional software quality gates, suggesting the relationship between code quality and AI productivity may be less linear than the fundamentals-advocates argue.
•Beyond Components: Designing Generative UI for MCP Apps (Casas, Postman) presented a three-tier generative UI spectrum (static components, declarative, fully generative) and argued MCP apps — with sandboxed iframe execution and bidirectional message passing — are the correct distribution layer for agent-rendered UI.
•MCP UI (Yosef & Salomon, MCP Apps) demonstrated adoption across VS Code, Cursor, ChatGPT, Claude, GitHub Copilot, and Postman, with Shopify sending MCPUI chunks for all storefronts and Hugging Face Spaces as MCPUI widgets — positioning 2026 as the year MCP Apps becomes the global standard for UI inside chat applications.
•The jury is still out on whether generative and agent-native UI genuinely replaces human-facing interfaces or merely complements them. Most sessions showed agent-native patterns alongside, not instead of, conventional UIs. The 60% Vercel bot traffic is crawlers/agents, not users operating via agents on their behalf.
•RL Nabors (WebMCP) acknowledged the 'starfish' chat UI anti-pattern is a transitional phase, but the replacement isn't clear — WebMCP is still under active standards body discussion and MCP Resources are poorly implemented across clients.