Editorial illustration for EAGLE3 Training on AMD MI350X Lifts Per-GPU Throughput Up to 1.77x
EAGLE3 Training on AMD MI350X Lifts Per-GPU Throughput...
AMD's Quark team has trained an EAGLE3 speculative decoding draft model entirely on Instinct MI350X GPUs, using a ROCm-native stack built from ROCm, vLLM, and TorchSpec. The target model is amd/MiniMax-M3-MXFP4, a large mixture-of-experts LLM, and the draft itself uses EAGLE3.1, a refinement of the original EAGLE3 method. This follows a previous blog from AMD that covered the serving side of EAGLE3 on Instinct GPUs, including ROCm and vLLM backend work and Quark FP8 quantization for draft models.
That post handled only half the problem. A serving stack, no matter how well tuned, can't speed up a target model beyond what its draft model allows, so draft quality sets the ceiling.
This piece covers the training side: how the pipeline was built, what ROCm enablement work made it possible, which training decisions mattered most, and how draft quality was measured using acceptance length on the SPEED-Bench benchmark. The result is a repeatable process for training EAGLE3 drafts for newly released LLMs directly on AMD Instinct hardware, closing the loop between training and serving.
Together, these pieces make the train-to-serve loop a single ROCm-native workflow: AMD Quark for the quantized target, ROCm and vLLM for extraction and serving, and TorchSpec for the training core.
Why this matters
The MI350X numbers matter less as a benchmark win and more as proof that AMD's stack can now handle the full speculative decoding pipeline, not just serving. Quark getting used to train the draft model, not just quantize it for inference, closes a gap that made ROCm deployments feel like second-class citizens compared to CUDA-based EAGLE3 setups. For teams running inference at scale, the concurrency curve is the real story: 1.77x at low concurrency sounds great in a slide deck, but the drop to 1.40x at concurrency 128 is the number that should shape capacity planning. Speculative decoding gains shrink exactly when you need them most, under heavy load, and that's worth remembering before anyone oversells the headline multiplier.
Publishing amd/MiniMax-M3-E as an open draft model is the practical takeaway here. It gives developers something to actually test against their own target models rather than taking AMD's word for it. We'd want to see independent verification of these throughput numbers on real workloads, and some clarity on how this draft model generalizes beyond MiniMax, before treating MI350X as a settled alternative for speculative decoding infrastructure.
Related Reading
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google to End Free Access to Gemini AI for Non-Subscribers
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
From Archives DeepMind Genie 4 Modernizes Pre-LLM Agents Using Gemini Omni
From Archives CoRover Lists Three BharatGPT Models on Government-Backed AIKosh Platform
Further Reading
- Training EAGLE3 Speculative Decoding Draft Models on AMD Instinct MI350X with AMD Quark - AMD ROCm Blog
- Accelerating Large-Scale LLM Inference on AMD Instinct MI350X with EAGLE3 Speculative Decoding - AMD ROCm Blog
- EAGLE-3 Large-Model Best Recipe - AMD Quark Documentation
- Speculative Decoding (EAGLE-3) on AMD Instinct - AMD Quark Documentation
- MiniMax-M3-EAGLE3.1 - Hugging Face
