Architecturally Ready, Behaviorally Unvalidated — Ioana Ghita, AI Architect, Accesa

Ioana Ghita, AI Architect at Accesa, argues in this fireside-chat-style session that the critical gap in agentic AI systems is not architectural or infrastructural readiness, but behavioral validation — the failure to verify how a system behaves when real-world inputs deviate from modeled test scenarios. The core thesis: agentic systems are predictably failing in production, not because they are poorly built, but because they have never been tested against the true complexity of what they will encounter.

Ghita distinguishes between 'testing' (running scenarios you designed) and 'behavioral validation' (exploring what you did not think of). She describes a 'gray zone' — the space between product definition, development, and testing — where cross-functional collaboration must happen but rarely does, because this type of work is not recognized as a standard engineering, product, or QA task.

Several concrete production failures are presented. A communication agent that handled hundreds of test conversations successfully began producing invalid JSON in production because real users sent long, multilingual, emoji-laden messages. The model was not broken — the inputs simply combined edge cases that had only been evaluated in isolation. A document processing agent collapsed when non-technical users interrupted workflows or interacted inconsistently — scenarios never modeled during MVP design.

A particularly striking example involves a manufacturing anomaly detection agent that correctly identified a large spike, correctly labeled it, but triggered a decision involving hundreds of millions of euros that was strategically wrong — not due to a bug or hallucination, but because no decision fallback existed for that valid-but-unmodeled semantic state. Ghita calls this an 'unmodeled semantic and decision state' — a class of failure invisible to technical monitoring.

Another case: an interview agent hallucinated a non-existent action in response to an unexpected candidate answer. The orchestrator caught the exception, executed a fallback, and returned HTTP 200. Infrastructure dashboards showed zero errors and normal latency. Meanwhile, the candidate was trapped in a conversational loop for over 40 minutes. The failure only surfaced through human feedback — illustrating that infrastructure-level observability and business-level correctness are fundamentally different.

Ghita introduces several engineering principles drawn from production experience. First, design for failure: in agentic systems, failures cascade — each wrong decision builds on and amplifies the previous one. Second, explicitly define and separate the instruction space, data space, and forbidden-action space for every agent; blurring these boundaries creates prompt injection vulnerabilities from external inputs like emails, documents, or web pages. Third, stress-test continuously — every added agent exponentially increases failure combinations, so agents should only be added when a simpler workflow will not suffice.

On observability, Ghita argues that logging agent actions is insufficient; systems must log agent reasoning — what the agent observed, what plan it formed, and why it chose a specific action — at decision checkpoints with an end-to-end trace ID to reconstruct decision chains after the fact.

For state management, she warns against relying solely on prompt-resident context, coining the phrase 'amnesiac with permissions' for an LLM-based agent that lacks an external state store: it will lose or distort intermediate decisions over time while retaining the ability to act.

Behavioral evaluations (evals) are presented as the mechanism that closes the loop: each identified edge case or production failure is turned into a concrete eval run continuously across every model update and prompt modification. Ghita connects this to emerging agent certification frameworks requiring risk quantification, predictability, and audit trails — all achievable through well-designed evaluations. She recommends getting agents in front of 10 real users as early as possible, noting this yields more behavioral insight than 500 automated tests.