Editorial illustration for AMD builds Llama 3.1 8B pretraining benchmark for MLPerf, using random weights
AMD builds Llama 3.1 8B pretraining benchmark for...
AMD has submitted a training benchmark for a model that doesn't learn. For the latest MLPerf results, the company pretrained Meta's Llama 3.1 8B architecture using entirely random weights. No data, no real gradients. Just frozen noise moving through the system to measure pure hardware throughput.
It is an elegant hack. Benchmarks are supposed to represent real work, and training a model is the most expensive real work in AI. But the actual learning process introduces chaotic variables.
Data quality, model convergence, and loss curves can obscure the thing AMD wants to measure: whether its software stack and Instinct MI300X accelerators can simply move a realistic training workload at scale. Random weights eliminate the learning variable. The benchmark becomes a pure, if synthetic, stress test of the pipeline itself.
Ensure that $LOGDIR has write access for the results to be written by running sudo chmod -R 777 $LOGDIR , In this example the folder /data/mlperf_llama31_8b/results is used as the results directory, so please make sure to create this directory.
The submission is a provocation dressed as engineering. It asks what we are really benchmarking. Model quality or system performance?
AMD's answer is clear. For proving a platform's raw capability to handle the structure and scale of modern training, a perfect, repeatable, meaningless workload is more useful than a real one. The success condition is not a low loss value but a completed log file in a directory with the correct permissions.
This is the unglamorous foundation. Before you can train a smart model, you must first prove the machine will do exactly what you tell it to, even when the instructions are nonsense.
Related Reading
Browse all Meta news →
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
Related Topic OpenAI hires 630 ex-Meta staff as ChatGPT memory may turn data into ads
Business & Startups
Related Topic Meta AI Update Pulls From Your Calendar for Daily Briefings
LLMs & Generative AI
From Archives AMD's MI355X CDNA4 GPU Shows Competitive Training Times in MLPerf v6.0
From Archives NVIDIA Blackwell Leads MLPerf Training 6.0 with Full‑Stack Scale
From Archives Meta to tighten AI token use with budgets, allocations and new AI Gateway
Market Trends
From Archives Meta launches Hatch AI agent, its first paid product, priced up to USD 200/month
Open Source
Common Questions Answered
Why did AMD submit a Llama 3.1 8B pretraining benchmark using random weights for MLPerf?
AMD submitted this benchmark to demonstrate raw system performance capabilities rather than model quality. By using random weights with no real data or gradients, AMD created a perfect, repeatable, and meaningless workload that isolates platform performance from model learning outcomes. This approach allows for consistent benchmarking of infrastructure capabilities independent of training data quality.
What is the key difference between AMD's random weights benchmark and traditional model pretraining?
Traditional pretraining involves learning from real data with meaningful gradients to improve model quality and achieve low loss values. AMD's benchmark uses entirely random weights with no actual learning, focusing instead on system performance metrics like completed log files and correct permissions rather than model accuracy or loss reduction.
What does AMD argue is the real purpose of their MLPerf benchmark submission?
AMD argues that their benchmark reveals what MLPerf is truly measuring: system performance and infrastructure capability rather than model quality. The submission demonstrates that for proving a platform's ability to handle the structure and scale of modern training workloads, a perfect and repeatable benchmark is more useful than attempting to train a real, learning model.
How does AMD's success condition differ from conventional model training benchmarks?
In conventional training benchmarks, success is measured by achieving a low loss value, indicating the model has learned effectively from data. In AMD's submission, the success condition is simply completing the benchmark and generating a log file with correct permissions, focusing purely on system execution rather than learning outcomes.
Further Reading
- Technical Dive into AMD MLPerf Training v5.1 Submission — AMD ROCm Blog
- Reproducing AMD MLPerf Training v5.1 Submission Result — AMD ROCm Blog
- MLPerf Training Adds Llama 3.1 8B Benchmark — MLCommons
- MLCommons Releases MLPerf Training v5.1 Results — GlobeNewswire
- MLPerf Training Expands with Llama 3.1 405B — MLCommons
