Editorial illustration for NVIDIA Vera Rubin NVL72 Leads MLPerf Endpoints v1.0 Agentic Inference
NVIDIA Vera Rubin Tops MLPerf Agentic Inference Test
MLCommons published MLPerf Endpoints v1.0 results on November 9, 2026, and the numbers point to a new bottleneck in AI infrastructure: agentic workloads don't behave like simple chatbots. An agent juggling tool calls, growing conversation history, and multi-step reasoning puts different strain on an inference system than a single question-and-answer exchange. MLPerf Endpoints v1.0 was built to measure exactly that strain, running coding and enterprise workflows through actual API endpoints and checking generated tool calls and workflow intent against reference answers. A separate SWE-bench Verified evaluation checks whether systems tuned for speed still solve tasks correctly end to end.
NVIDIA submitted provisional results for its next-generation Vera Rubin NVL72 system under this benchmark, running the Kimi K3 model in the Standardized division's Available category. The comparison point is NVIDIA's current GB300 NVL72, tested under the same agentic combined workload. Both systems were measured on throughput per kilowatt of provisioned power at a shared floor of 50 tokens per second per user in end-to-end interactivity, a threshold meant to keep responsiveness consistent across comparisons.
In MLPerf Endpoints v1.0, provisional NVIDIA Vera Rubin NVL72 results show over 10x the throughput per kilowatt of provisioned power of NVIDIA GB300 NVL72 at a shared minimum end-to-end interactivity of 50 tokens per second per user.1
Why this matters
MLPerf Endpoints v1.0 is the first time we've seen a standardized benchmark actually model what agentic inference looks like in production: long context accumulation, shared tool definitions, reused business rules across turns, not just single-shot prompt throughput. That distinction matters for anyone building agent systems right now. NVIDIA's Vera Rubin NVL72 topping these results tells us where the hardware roadmap is pointed, but the more useful signal for developers and founders is the benchmark's methodology itself.
If you're evaluating inference providers for a coding assistant or enterprise workflow tool, raw tokens-per-second numbers from older benchmarks won't tell you much about how a system handles a 50-turn conversation with accumulating context. We'd push back gently on treating any single vendor's win here as settled, though. Benchmarks sponsored by the hardware maker being measured deserve the usual scrutiny, and checking generated tool calls against reference outputs is only as good as how representative those references are of real workloads.
Worth watching: whether other inference providers publish comparable numbers on this same dataset, and whether the context-length findings in Figure 1 hold up across different agent architectures, not just NVIDIA's own stack.
Common Questions Answered
How does agentic inference differ from traditional chatbot inference according to MLPerf Endpoints v1.0?
Agentic inference involves agents juggling tool calls, growing conversation history, and multi-step reasoning, which puts different strain on an inference system compared to simple question-and-answer exchanges. MLPerf Endpoints v1.0 was specifically built to measure this distinct type of workload by running coding and enterprise workflows through actual API endpoints, capturing the complexity of production agent systems.
What performance advantage does the NVIDIA Vera Rubin NVL72 demonstrate over the GB300 NVL72 in MLPerf Endpoints v1.0?
The NVIDIA Vera Rubin NVL72 shows over 10x the throughput per kilowatt of provisioned power compared to the NVIDIA GB300 NVL72, while maintaining a shared minimum end-to-end interactivity of 50 tokens per second per user. This significant efficiency improvement represents a major advancement in agentic inference performance.
What key characteristics does MLPerf Endpoints v1.0 measure that weren't captured in previous benchmarks?
MLPerf Endpoints v1.0 models production agentic inference through long context accumulation, shared tool definitions, and reused business rules across turns, rather than just measuring single-shot prompt throughput. This standardized benchmark approach provides a more accurate representation of how agent systems actually behave in real-world applications.
Why is the NVIDIA Vera Rubin NVL72 leading performance in MLPerf Endpoints v1.0 significant for developers building agent systems?
The Vera Rubin NVL72's top performance indicates where the hardware roadmap is pointed and provides developers and organizations with crucial signals about which infrastructure investments will best support agentic workloads. This information helps guide decisions for building and deploying agent systems that require sustained performance across complex, multi-step reasoning tasks.
Further Reading
- Papers with Code Benchmarks - Papers with Code
- Chatbot Arena Leaderboard - LMSYS