Editorial illustration for Flow Matching and Diffusion Models Show Theoretical Equivalence
Flow Matching and Diffusion Models Show Theoretical...
A team from Apple's machine learning research group, including Oscar Davis, Anastasiia Filippova, and Marco Cuturi, has pushed continuous flow matching for language generation past the scale where anyone had actually tested it. Their paper, "Scaling Categorical Flow Maps," trains a 1.7-billion-parameter base model on 2.1 trillion tokens, then distills it into what they call a Categorical Flow Map, a system that generates text in as few as 4 inference steps.
Autoregressive models still dominate text generation, producing one token at a time. Diffusion and flow matching offer a different route: they've already sped up image and audio generation, and researchers have started asking whether the same tricks work for discrete data like language. Earlier attempts at flow matching for text, mapping a Gaussian distribution onto one-hot encoded tokens, showed promise but only at scales under 1 billion parameters. Nobody had shown whether the approach would hold up when scaled to sizes comparable with real language models.
The Apple team's paper also introduces a likelihood bound for scoring these models on standard benchmarks, and lays out what breaks when you push this architecture past its previous limits.
Recently, several works have demonstrated the possibility of generating discrete data continuously by a simple flow matching process between a Gaussian and the one-hot encoded data distribution. They have further shown the feasibility of accelerated sampling via Categorical Flow Maps (CFMs), resulting in competitive sample quality in the few-step regime.
Why this matters
The equivalence between flow matching and diffusion under Gaussian assumptions isn't just theoretical housekeeping. If the two frameworks really do collapse into the same mathematical object, then techniques built for one, like score distillation, should carry over to the other without a full rebuild. That's the practical bet Davis, Cuturi, Béthune and their co-authors are making with Scaling Categorical Flow Maps and the related CAR-Flow work.
For researchers, it's a signal that progress on diffusion-based language modeling doesn't have to start from zero each time someone picks flow matching instead. For teams building non-autoregressive LMs, the open question is whether distillation transfers cleanly to categorical, discrete-token settings, not just continuous ones, since language is discrete and Gaussian assumptions don't map neatly onto tokens. We'd treat this as early-stage plumbing work rather than a finished result.
The real test is whether these scaled categorical flow maps produce language models that are faster or cheaper than autoregressive baselines without sacrificing quality. Until that benchmark shows up, this is groundwork worth watching, not a result to build a roadmap on.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives Agent Teams With Real-Time Coordination Beat Claude Opus on Coding
From Archives OpenAI partners with psychologists to address AI's mental health shortcomings
Further Reading
- Score Distillation of Flow Matching Models - Apple Machine Learning Research
- Scaling Categorical Flow Maps - Apple Machine Learning Research
- An Introduction to Flow Matching and Diffusion Models - arXiv
- Exploring Diffusion and Flow Matching Under Generator Matching - arXiv
- Diff2Flow: Training Flow Matching Models via Diffusion ... - arXiv
