NVIDIA GPU cluster running DiffusionGemma for high-performance text generation, showcasing AI-powered text-to-image and langu

Editorial illustration for Run DiffusionGemma on NVIDIA GPUs for high‑throughput text generation

Run DiffusionGemma on NVIDIA GPUs for high‑throughput...

Run DiffusionGemma on NVIDIA GPUs for high‑throughput text generation

By AI Daily Post Edited by Brian Petersen, Editor-in-Chief

June 10, 2026 • 2 min read

Developers building real‑time AI—chat assistants, copilots, agentic workflows—still hit a wall when it comes to token‑by‑token generation speed. That bottleneck hurts responsiveness, drives up serving costs and makes fluid, interactive experiences hard to deliver. Here’s the thing: DiffusionGemma, a model from Google DeepMind, flips the script by generating tokens in parallel instead of one at a time.

While the tech is impressive, its impact is practical. The diffusion‑based denoising engine spits out 256 tokens per step, reaching up to 1,000 tokens per second on a single NVIDIA H100 Tensor Core GPU and about 150 tokens per second on an NVIDIA DGX Spark. The fastest local numbers show up on an NVIDIA DGX Station.

Built on the Gemma 4 26B A4B MoE architecture, DiffusionGemma is tuned for low‑latency, memory‑bound inference across NVIDIA’s data‑center and client GPUs—from GeForce RTX 5090 to RTX PRO. For developers, the promise is lower serving costs, higher concurrency and snappier user experiences, all without sacrificing model quality. The model is accessible through Hugging Face Transformers and can be scaled with vLLM following the provided playbooks.

In addition to NVIDIA data center GPUs, developers can enjoy optimal performance on a variety of client GPUs and systems. Build and prototype on NVIDIA Access DiffusionGemma through Hugging Face Transformers for initial testing and prototyping on NVIDIA GeForce RTX 5090 or DGX Spark. For higher throughput or concurrent multi-user serving on DGX Spark, DGX Station, and RTX PRO, use vLLM by following our playbooks in Table 2.

With Day 0 support across NVIDIA hardware and software--from local prototyping to production deployment--developers can quickly move from experimentation to real-world applications. NVIDIA GPU-accelerated endpoints Start building with DiffusionGemma with free access for prototyping to GPU-accelerated endpoints on build.nvidia.com as part of the NVIDIA Developer Program.

Run DiffusionGemma on NVIDIA for Developer-Ready, High-Throughput Text Generation - NVIDIA Developer Blog

Why this matters

Can we finally break the token‑by‑token bottleneck? DiffusionGemma, a model from Google DeepMind, claims to generate text in parallel, and it runs efficiently on NVIDIA data‑center and client GPUs such as the RTX 5090 or DGX Spark. For developers, that means we can prototype through Hugging Face Transformers without waiting for each token, potentially shaving latency and reducing serving costs.

Yet the article offers no data on output quality or how parallelism interacts with complex prompting, leaving open the question of trade‑offs. Founders may see a path to more fluid chat assistants or copilots, but integration effort and hardware availability could temper enthusiasm. Researchers gain a new testbed for high‑throughput generation, though it remains unclear whether the approach scales to large‑scale deployments beyond the showcased hardware.

In short, the tool expands our options, but we should monitor real‑world performance before assuming it solves the responsiveness problem entirely. We’ll need to benchmark it across diverse workloads and keep an eye on any latency spikes that could surface under heavy traffic.