Agentic Search for Context Engineering — Leonie Monigatti, Elastic

Leonie Monigatti (Elastic) delivers a hands-on workshop arguing that context engineering is approximately 80% agentic search — the mechanism by which agents decide what information to pull from files, databases, memory, and the web before generating a response. The talk traces the evolution from fixed RAG pipelines (where a user query was used verbatim as a vector search query against a single database) to agentic RAG (where an LLM autonomously decides whether and when to call retrieval tools), and then to full context engineering across multiple source types: local files, databases, the web, and long-term memory.

Monigatti introduces the concept of a "shell tool" (called bash tool in Anthropic Claude Code, exec tool in OpenAI, shell tool in LangChain) as a maximally versatile retrieval interface: agents can run ls and grep for file system navigation, curl for web/API calls, custom CLIs for database access, or write full scripts on the fly. She warns that it must be used in sandbox environments due to absence of built-in safeguards.

The workshop demonstrates three progressively complex retrieval patterns using conference session data stored in Elasticsearch, with LangChain as the agent framework. The first demo uses a simple semantic search tool backed by jina-embeddings-v5, limited to returning top-3 results using similarity search. This works for broad topic queries but fails on acronym lookups like "GDPA" — returning semantically adjacent but irrelevant results (Gemma models, harness engineering), illustrating keyword-search failure modes of pure vector retrieval.

The second demo replaces the semantic search tool with an ESQL (Elasticsearch Query Language) execution tool, upgrading the model from GPT-4.1-nano to GPT-4.1-mini to handle the more complex parameter generation task. The agent correctly attempts a wildcard query but uses SQL-style `%GDPA%` syntax instead of ESQL's asterisk wildcard `*GDPA*`, returning zero results. This illustrates how parameter complexity is itself a failure mode. The solution is to introduce an agent skill — a markdown file whose metadata (name, description) is injected into the system prompt via progressive disclosure, with full syntax documentation (including wildcard rules, double-quote string literals, and basic ESQL structure) loaded into the context window only when the skill is needed. After integrating the skill and reinforcing in both the tool description and system prompt to "always load the ESQL skill before calling the query tool," the agent generates correct ESQL on the first attempt, finding the GDPA session. The same agent then handles aggregation queries (counting sessions on April 8th via ESQL COUNT with filters), avoiding the known LLM counting failure mode by offloading arithmetic to the database.

The third demo switches to local file system retrieval using the shell tool, with session data organized as per-session flat files under typed subdirectories (keynotes/, workshops/, etc.). For the regulatory constraints query, the agent chains grep calls across multiple synonyms — "regulatory," "compliance," "GDPR," "governance," "sovereignty" — achieving semantic-search-like recall through synonym enumeration. Monigatti notes this is inefficient for broad semantic queries (e.g., "movies with animal superheroes" cannot be reliably covered by enumerated synonyms). She then demonstrates Jina Grap, a CLI-based semantic grep tool, which on the first shell invocation correctly finds the regulatory constraints session without synonym enumeration.

Practical recommendations close the talk: build a balanced tool stack with a "low floor" (specialized, simple-parameter tools that work reliably out of the box — e.g., lookup-by-ID, semantic search) and a "high ceiling" (general-purpose tools like query execution or shell tool for unexpected or complex queries). If agent query behavior is unknown, start with general-purpose tools, log behavior, and identify patterns where the agent takes 4-5 tool calls per question — a signal the tool is too hard to use and a specialized replacement is warranted. A referenced Vercel blog post ("Is Bash All You Need?") found that a hybrid agent combining a shell tool and a database query tool achieved higher accuracy than either alone, due to cross-validation behavior.