Text Diffusion — Brendon Dillon, Google DeepMind

June 4, 202628:036,576 views

Summary

Brendon Dillon, Research Scientist at Google DeepMind, presents a technical overview of text diffusion models with a focus on Gemini Diffusion — a research preview variant of Gemini that replaces autoregressive next-token generation with iterative denoising. The talk covers the mechanics, hardware efficiency, unique capabilities, and current limitations of text diffusion, illustrated with live demos.

Text diffusion works by first corrupting a clean sequence of tokens with random noise during training, and then training a neural network to denoise the corrupted sequence. At inference time, the model initializes with pure random discrete tokens and iteratively refines the entire block over a series of denoising steps. Unlike autoregressive generation which produces one token at a time, diffusion processes a full output block (e.g., 256 tokens) in multiple forward passes — but crucially far fewer passes than there are tokens.

The core latency advantage is grounded in GPU/TPU memory bandwidth constraints. These chips are memory-bound: streaming model weights and KV cache from HBM to tensor cores is the bottleneck, not raw FLOP capacity. Autoregressive models must stream all weights once per token; text diffusion can generate 256 tokens in approximately 24 forward passes, resulting in roughly 10x fewer memory transfers and a corresponding ~10x latency improvement. The Gemini Diffusion research preview, released to ~100,000 users, consistently achieved 2,000 raw tokens per second in a web browser. This compared favorably against Gemini 2.0 Flash Lite at the time, with similar overall quality, a slight advantage in code, and a slight disadvantage in some other areas.

The main disadvantage of text diffusion is lower throughput at large batch sizes. Because multiple forward passes are applied to the same data, the model hits a compute threshold earlier than autoregressive alternatives. This makes it more expensive to serve at scale, which is why it has not been adopted for flagship production models like Gemini or Claude.

The talk highlights three capabilities that go beyond raw speed. First, bidirectional attention enables self-correcting generation: the model can attend to future tokens it has already tentatively generated, then revise earlier outputs. A live demo shows the model solving a multi-step arithmetic problem where it guesses 60 after one forward pass, revises to 49 after two, and correctly outputs 39 after three — having spotted and fixed its earlier error by examining downstream reasoning. By contrast, GPT-4o answered 40 (initially) and Gemini 2.5 Flash answered 42 and failed to correct itself, baking the error into subsequent reasoning. Second, adaptive computation allows the model to determine how many denoising steps to spend on a problem. Generating the first 100 digits of pi required just 4 steps; FizzBuzz required 18; explaining quantum mechanics required 31. Harder evals (GPQA Diamond) consistently required more steps; easier ones (MBPP) required fewer. Third, in-place editing lets the model make targeted changes within a generated sequence — inserting a missing middle paragraph in a story, adding code documentation, or fixing a bug — because bidirectional attention lets every token attend to the full context.

Demos shown internally at Google IO include a fake Wikipedia where every page is generated on the fly (HTML and content), a Reddit-clone generating AI comments and images on demand (images supplied by a separate Imagen-class model), an operating system where every click generates the next screen, and a 15-second voice-driven todo app build demonstrated by an external developer using the Gemini Diffusion API. Dillon concludes by noting that text diffusion is already deployed in on-device and robotics contexts within the Alphabet ecosystem — settings where low latency matters more than batch throughput — and that a next generation of the model is forthcoming.

Key Takeaways

•Text diffusion generates a full output block in ~24 forward passes vs ~256 sequential passes for autoregressive models, yielding roughly 10x fewer memory transfers and proportionally lower latency on memory-bound hardware.
•Gemini Diffusion achieved 2,000 raw tokens per second in a web browser during its 100k-user research preview — a dramatic speed advantage over comparable autoregressive models.
•The critical current limitation is lower throughput at large batch sizes, which makes text diffusion more expensive to serve at scale and explains why it has not yet replaced autoregressive generation in production.
•Bidirectional attention is architecturally unique to diffusion: the model can see future tentative tokens, catch logical errors in earlier outputs, and self-correct — something autoregressive models fundamentally cannot do without separate revision steps.
•GPT-4o and Gemini 2.5 Flash (both larger models) failed to self-correct on a multi-step arithmetic problem where Gemini Diffusion (considerably smaller) arrived at the correct answer of 39 by revising across forward passes.
•Adaptive computation is emergent: easier prompts (first 100 digits of pi) complete in ~4 denoising steps; harder prompts (GPQA Diamond-class reasoning) take significantly more, and the model decides the budget autonomously.
•In-place editing — inserting paragraphs, patching code, adding documentation — is a natural capability of diffusion models because bidirectional attention allows coherent fills within a fixed context window.
•On-device and robotics environments (phones, robots) are the immediate sweet spot for text diffusion because low latency matters more than batch throughput in those single-user, non-batched settings.
•Larger diffusion models tend to require fewer denoising steps for the same quality output, partially offsetting higher per-step FLOP costs at scale.

Transcript

Some people are still filtering into the room. But it's mostly intro stuff for the first couple of slides so they won't miss anything. Okay, welcome everybody. My name is Brendan. I'm I'm a research scientist at DeepMind. I'm talking today about text diffusion, which is kind of a more forward-looking research area at DeepMind. So you're probably familiar with image and video diffusion, which is kind of state of the art for these modalities right now where you you know, you take ground truth, say...