Voice AI: when is the "Her" moment? — Neil Zeghidour, CEO, Gradium AI

Neil Zeghidour, CEO of Gradium AI and co-creator of Moshi — the first full-duplex voice model — delivers a candid technical assessment of why voice AI has repeatedly failed to reach the so-called "Her moment" and what must change architecturally and scientifically before it does.

Zeghidour opens by framing the dominant paradigm: cascaded systems that chain streaming speech-to-text, an LLM, and streaming text-to-speech together. He acknowledges these systems are practical and improving, but argues they are architecturally incapable of feeling like natural human conversation. The core latency problem is stark: TTS alone currently exceeds 200ms in even the fastest models, yet human conversation requires the entire loop — understanding, reasoning, and speech production — to complete in roughly 200ms. Furthermore, when any real tool call enters the stack, latency balloons to 500ms–4 seconds depending on LLM routing and API overhead. This renders the marginal gains from TTS optimization largely irrelevant when tool calls dominate end-to-end latency. As a partial mitigation, Zeghidour demonstrates a live-coded travel agent that uses LLM-generated filler speech — contextually relevant commentary about a destination — to mask tool-call delays and keep the conversation flowing naturally.

The talk then dissects the difference between speech-to-speech models and true full-duplex systems. Most leading speech-to-speech models, including what Zeghidour characterizes as the best in the industry, are still half-duplex: the model is either listening or speaking, never both simultaneously. This breaks backchanneling — the "mhm," nods, and overlap that constitute up to 20% of spoken interaction time in some cultures, such as Japanese. Zeghidour demonstrates this failure mode live with a prominent half-duplex model, showing how it interprets every natural backchannel as an interruption and derails the exchange. Moshi, built by his team at a non-profit lab funded by Eric Schmidt and Xavier Niel, solved full-duplex natively: in a recorded demo, two participants talk over each other continuously and the model correctly handles the overlap. However, Zeghidour is candid that Moshi's usefulness was severely limited — no tool-call capability, no observability, no moderation layer, and no real paralinguistic exploitation despite the information being technically present in the audio stream.

Paralinguistic understanding emerges as what Zeghidour calls the most underrated open problem. He illustrates it with a clip from the film Her, where the AI correctly infers emotional discomfort purely from vocal tone. Any speech-to-speech model retains this information — it is not discarded as it is in STT transcription — but models trained on the audio equivalent of instruction-following datasets have no incentive to learn these cues. Building models that genuinely exploit tone, hesitation, and cultural vocal signals requires new training approaches, not prompt engineering.

Scalability and cost represent the final barrier. TTS at production scale is the primary cost driver in consumer voice stacks — far outpacing LLM inference, STT, or diarization. Zeghidour states he has observed startups exhaust their fundraising on TTS API bills before acquiring a sustainable user base. To address this, Gradium is releasing Phonon, an on-device TTS model under 100 million parameters that runs on a smartphone CPU (not a GPU), offers voice cloning, and matches or surpasses all existing on-device TTS models. A Rick and Morty-style demo illustrates voice cloning quality at zero API cost. Phonon is entering private beta and targets consumer app developers who need to scale voice usage without incurring per-call cloud fees. Zeghidour closes by rejecting the emerging industry consensus that voice is now a commodity, arguing the hardest technical problems — full-duplex intelligence, paralinguistic understanding, and cost-efficient deployment — remain unsolved science and engineering challenges.