← back
Text Diffusion — Brendan O’Donoghue, Google DeepMind
Takeaway
Text diffusion trades stronger parallel generation and low latency against less favorable high-volume serving economics.
Summary
- Text diffusion trains models to denoise corrupted token sequences and generates by repeatedly refining a block rather than emitting tokens only left to right.
- Bidirectional attention enables revision using future tokens, while iterative generation supports adaptive computation and in-place editing.
- The Gemini Diffusion research preview is described as achieving similar quality to its contemporary autoregressive baseline with much lower latency.
- Generating many tokens per weight transfer improves memory-bound performance, but repeated forward passes reduce large-batch throughput and increase serving cost.
text-diffusiongemini-diffusioninference-latency
Original description
GPT-4o answered 40. Gemini 2.5 Flash answered 42 and stuck to it even after working through the reasoning incorrectly. The Gemini Diffusion model, considerably smaller than both, answered 60 on the first forward pass, then 49, then corrected itself to 39 once it finished reasoning. Bidirectional attention means it can see future tokens and go back to fix mistakes. Autoregressive models cannot do that. Brendon O'Donoghue covers why text diffusion is fast (24 denoising steps to generate 256 tokens means roughly 10x fewer memory transfers than autoregressive generation), what the tradeoff is (lower throughput at large batch sizes makes it expensive to serve at scale today), and what gets unlocked when latency drops to 2,000 tokens per second. The demos include a fake Wikipedia generated on the fly, a Reddit clone with AI generated comments and images, an operating system where every click generates the next screen, and a todo app built in 15 seconds by voice. Speaker info: https://x.com/bodonoghue85 https://bodono.github.io/ / bodono Timestamps: 0:00 Introduction to Text Diffusion 1:02 How Text Diffusion Works (Training and Inference) 2:06 Gemini Diffusion Research Preview 3:04 Difference Between Autoregressive and Diffusion Models 4:02 Pros and Cons of Text Diffusion 6:13 Hardware Efficiency: Why Text Diffusion is Faster 8:47 Bidirectional Reasoning and Self-Correction 12:00 Dynamic and Adaptive Computation 14:26 In-place Text Editing 16:09 Low Latency Applications and Demos 20:05 Q&A Session