NVIDIA Blackwell Ultra Shatters MLPerf Inference Records

This is what happens when you stop thinking in terms of individual chips and start engineering entire systems as a single machine. 288 GPUs, working in lockstep, turning tokens into torrents. The record isn’t the headline, it’s the proof.

Proof that co-design scales: from the silicon up through the rack, the network, the software stack, all the way to the data center’s cooling grid. The workloads themselves keep sprinting. Larger models, longer contexts, more modalities.

NVIDIA met that pace again, not just on one benchmark, but across the entire MLPerf catalog. From LLMs to vision to recommenders, the same architecture that processes millions of tokens per second for a chatbot can serve a generative ad system in the next breath. That versatility matters.

Real-world AI doesn’t live on a single benchmark; it lives in a messy, heterogeneous data center. So what comes next? The Endpoints track hints at it.

Inference is no longer a batch job, it’s a continuous, low-latency service. And as models grow, the only way to deliver that service at scale is to repeat this exercise: rethink the rack, rebalance the interconnect, recompile the kernel. Blackwell Ultra is a snapshot of that philosophy in motion.

The next one will be faster.