Editorial illustration for ZAYA1-8B (<1B params) matches/exceeds DeepSeek-R1-0528 on math, coding tests
ZAYA1-8B (<1B params) matches/exceeds DeepSeek-R1-0528...
Size is not destiny. With fewer than one billion active parameters, ZAYA1-8B has pulled even with, or outright surpassed, DeepSeek-R1-0528 on rigorous math and coding benchmarks. That is not a fluke.
This model was built for reasoning from the ground up, embedding reasoning data into its pretraining diet via an answer-preserving trimming scheme. A four-stage RL cascade then sharpens its edge: reasoning warmup, a 400-task curriculum, targeted math and code reinforcement with test-time compute traces, and behavioral RL for chat. The result holds its own against far larger open-weight reasoning models.
But ZAYA1-8B also introduces a new test-time compute method: Markovian RSA. It recursively aggregates parallel reasoning traces while carrying forward only a bounded 4K-token tail. The payoff is stark: 91.9% on AIME'25, 89.6% on HMMT'25.
Those scores close the gap to titans like Gemini-2.5 Pro, DeepSeek-V3.2, and GPT-5-High. Efficiency, not scale, is the new frontier.
With under 1B active parameters, ZAYA1-8B matches or exceeds DeepSeek-R1-0528 on several challenging mathematics and coding benchmarks, and remains competitive with substantially larger open-weight reasoning models. ZAYA1-8B was trained from scratch for reasoning, with reasoning data included from pretraining onward using an answer-preserving trimming scheme. Post-training uses a four-stage RL cascade: reasoning warmup on math and puzzles; a 400-task RLVE-Gym curriculum; math and code RL with test-time compute traces and synthetic code environments built from competitive-programming references; and behavioral RL for chat and instruction following.
We also introduce Markovian RSA, a test-time compute method that recursively aggregates parallel reasoning traces while carrying forward only bounded-length reasoning tails between rounds. In TTC evaluation, Markovian RSA raises ZAYA1-8B to 91.9\% on AIME'25 and 89.6\% on HMMT'25 while carrying forward only a 4K-token tail, narrowing the gap to much larger reasoning models including Gemini-2.5 Pro, DeepSeek-V3.2, and GPT-5-High.
The numbers demand attention. A model with fewer than one billion active parameters, less than a rounding error in most frontier systems, now stands toe-to-toe with giants. That is not a curiosity; it is a signal.
ZAYA1-8B proves that reasoning capability is not a simple function of scale. Its performance on AIME’25, HMMT’25, and competitive coding benchmarks rewrites the assumption that you need a trillion parameters to think deeply. What makes this leap possible is a deliberate architecture of learning: reasoning embedded from the very first pretraining step, a cascade of reinforcement learning that hones mathematical and coding instincts, and a test-time method, Markovian RSA, that squeezes extraordinary gains from modest memory.
The 4K-token tail carries the essence of prior reasoning forward, and the result is a model that, for the first time, lets a sub-billion-parameter system challenge the top tier of open and closed reasoning engines. This is efficiency as strategy. It opens a door: specialized, high-performance reasoning on commodity hardware; applications where latency and cost matter more than raw parameter count; a path that does not require endless scaling.
ZAYA1-8B is not just a benchmark, it is a proof that the future of reasoning can be lean, sharp, and accessible. The gap to the giants is narrowing, and from here, it may vanish faster than anyone expected.
Related Reading
Open Source Trump cracks down on Anthropic after Amazon tip; staff largely foreign
Open Source SDOF Adds Two Defensive Layers via Intent Router and StateAwareDisp
Open Source D&B rebuilds 642 million‑business database after AI agents hit limits
From Archives SpaceX's Texas AI-chip 'Terafab' plant faces USD 55 billion plan, USD 119 billion total cost
From Archives Zyphra launches ZAYA1-8B MoE: 8.4B params, 760M active, cuts compute
Common Questions Answered
How does ZAYA1-8B achieve competitive performance with fewer than one billion parameters compared to DeepSeek-R1-0528?
ZAYA1-8B was specifically built for reasoning from the ground up by embedding reasoning data into its pretraining through an answer-preserving trimming scheme. The model then undergoes a four-stage reinforcement learning cascade including reasoning warmup, a 400-task curriculum, and targeted math and code reinforcement with test-time compute optimization to sharpen its reasoning capabilities.
What specific benchmarks demonstrate that ZAYA1-8B matches or exceeds DeepSeek-R1-0528?
ZAYA1-8B demonstrates superior or equivalent performance on rigorous math and coding benchmarks including AIME'25, HMMT'25, and competitive coding tests. These results prove that the model's reasoning capability is not simply a function of parameter scale, challenging the assumption that trillion-parameter systems are required for deep thinking.
What is the significance of ZAYA1-8B's performance relative to frontier AI systems?
ZAYA1-8B's achievement is significant because it proves that reasoning capability does not require massive scale, with fewer than one billion active parameters standing toe-to-toe with much larger frontier systems. This challenges the prevailing assumption in AI development that deeper reasoning requires trillion-parameter models, suggesting that deliberate architecture and training methodology can be more important than raw parameter count.
How does the answer-preserving trimming scheme contribute to ZAYA1-8B's reasoning capabilities?
The answer-preserving trimming scheme embeds reasoning data directly into ZAYA1-8B's pretraining diet, ensuring that the model learns reasoning patterns from its initial training phase. This foundational approach to reasoning, combined with the subsequent four-stage reinforcement learning cascade, creates a model optimized for mathematical and coding problem-solving from the ground up.
Further Reading
- ZAYA1-8B: Frontier intelligence density, trained on AMD - Zyphra — Zyphra
- Zyphra Releases ZAYA1-8B, a Reasoning Model trained on AMD and Optimized for Maximum Intelligence Density per Parameter — PR Newswire
- ZAYA1-8B: The Efficient MoE Reasoning Model Explained (2026) — Build Fast With AI
- Zyphra Releases ZAYA1-8B Reasoning Model — HPC Wire
