Editorial illustration for Nvidia's Nemotron 3 Debuts with Mamba Hybrid, Activating Just 3B of 31.6B Parameters
Nvidia Nemotron 3: Mamba Hybrid LLM Breakthrough
Nvidia's Nemotron 3 uses Mamba hybrid, 31.6B params, 3B active per step
Nvidia built a language model where 90% of it is asleep. That’s the trick. Their new Nemotron 3 has 31.6 billion total parameters, but only three billion are active at any one time. The rest are just sitting there, waiting for a specific reason to wake up.
This is what happens when you move past pure Transformer architecture. Nvidia swapped it for a hybrid that incorporates Mamba, a state-space model known for efficient long-context reasoning. The result is a model that thinks less to do more, or at least does it faster.
On standard benchmarks, Nemotron 3's accuracy matches other open-source models like GPT-oss-20B and Qwen3-30B. But its token throughput blows past them.
The efficiency comes at a cost. A single test run for the model requires 160 million tokens. That's 50 million more than its nearest competitor. It’s a hungry machine, built for speed over frugality.
Hybrid architecture boosts efficiency The Nano model has 31.6 billion total parameters, but only 3 billion are active per processing step. On the Artificial Analysis Index benchmark, the open-source model rivals gpt-oss-20B and Qwen3-30B in accuracy but delivers significantly higher token throughput. However, according to Artificial Analysis, it requires 160 million tokens for a test run - far more than runner-up Qwen3-VL at 110 million.
Nvidia introduces two architectural changes for the larger Super and Ultra models. The first, LatentMoE, addresses the memory bandwidth cost of routing tokens directly to expert networks in standard MoE models. The new method projects tokens into a compressed, latent representation before processing.
Nvidia says this drastically increases expert count and active experts per token without slowing inference. The larger models also use multi-token prediction (MTP), where models predict several future tokens simultaneously during training rather than just the next one.
For the larger Super and Ultra versions, Nvidia adds two more architectural gambits. LatentMoE compresses the decision of where to send data into a bottleneck. This lets the model consult far more specialized sub-networks, or experts, without bogging down.
The second is multi-token prediction, which trains the model to guess several words ahead at once instead of just the next one. The goal isn't raw power. It's surgical speed.
The entire project is a recalibration of efficiency, prioritizing throughput over every other metric. It suggests the next phase of AI competition won't be about who has the biggest brain, but who has the most disciplined one.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic NVIDIA and Google Cloud let developers scale AI from prototype to production
Market Trends
Related Topic NVIDIA NeMo powers telco reasoning model for autonomous network workflows
Industry Applications
From Archives Senators question AI toys that suggest knives; Mattel drops OpenAI toy for 2025
From Archives UC San Diego Lab Uses NVIDIA DGX B200 to Pursue Low-Latency LLM Serving
From Archives OpenUSD and NVIDIA Halos Enhance Robotaxi Safety with Synthetic Data, SimReady
Research & Benchmarks
From Archives NVIDIA appoints CloudThat as first Indian education services partner
Business & Startups
Common Questions Answered
How does Nvidia's Nemotron 3 achieve computational efficiency with its hybrid architecture?
Nemotron 3 uses a unique approach where only 3 billion parameters are actively processed out of its total 31.6 billion parameters per step. This dynamic parameter activation allows the model to maintain high performance while significantly reducing computational overhead.
How does Nemotron 3 compare to other open-source AI models in terms of performance?
On the Artificial Analysis Index benchmark, Nemotron 3 rivals models like gpt-oss-20B and Qwen3-30B in accuracy while delivering higher token throughput. However, it requires 160 million tokens for a test run, which is more than some competitor models.
What makes the Mamba hybrid architecture in Nemotron 3 significant for AI model design?
The Mamba hybrid architecture allows for strategic parameter activation, enabling the model to engage only a fraction of its total parameters during processing. This approach potentially represents a breakthrough in improving computational efficiency and processing speed for large language models.
Further Reading
- Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning — NVIDIA Research
- NVIDIA Nemotron 3: Efficient and Open Intelligence — NVIDIA Research
- SGLang Adds Day-0 Support for the Highly Efficient, Open Nemotron 3 Nano — LMSYS Org
- Not enough good American open models? Nvidia wants to help — The Register
