Editorial illustration for OpenMythos: 770M‑parameter PyTorch clone matches 1.3B Claude model, reasoning
Open-Source Claude Clone Matches 1.3B Model Performance
OpenMythos: 770M‑parameter PyTorch clone matches 1.3B Claude model, reasoning
OpenMythos arrives as a 770‑million‑parameter PyTorch reconstruction of Anthropic’s Claude Mythos, yet its performance lines up with the original 1.3‑billion‑parameter transformer on standard benchmarks. The open‑source effort promises to democratize a model that was previously locked behind proprietary walls, and the developers highlight a surprising strength: handling multi‑step reasoning without inflating model size. While most transformers stumble when the chain of inference doubles—from five hops to ten—the new codebase claims to keep its footing.
That claim matters because scaling reasoning depth has been a persistent bottleneck for language models, often requiring larger architectures or costly prompting tricks. If a lean model can preserve accuracy across longer logical sequences, the implications ripple through research and deployment alike. The authors point to a specific mechanism that lets the network entertain several possible continuations at once, effectively broadening its search through the problem space in a single pass.
*Continuous latent thoughts can also encode multiple alternative next steps simultaneously, enabling something closer to breadth‑first search over the reasoning space within a single forward pass. A standard transformer trained on 5‑hop reasoning chains fails when tested on 10‑hop chains at inference.*
Continuous latent thoughts can also encode multiple alternative next steps simultaneously, enabling something closer to breadth-first search over the reasoning space within a single forward pass. A standard transformer trained on 5-hop reasoning chains fails when tested on 10-hop chains at inference time -- it has no mechanism to extend its depth beyond what it saw during training. A Recurrent-Depth Transformer handles this naturally: running more inference-time loops extends the reasoning chain without any retraining.
Harder problems receive more compute; simpler ones exit early. Solving the Stability Problem Training looped models has historically been brittle. The hidden state ht can grow unboundedly across iterations -- a failure mode called residual explosion.
OpenMythos addresses this using a Linear Time-Invariant (LTI) injection constraint borrowed from the Parcae architecture (Prairie et al., 2026): the spectral radius of A, denoted ρ(A), is enforced to be less than 1 by construction, guaranteeing stability regardless of learning rate or gradient noise. A second failure mode also exists at the other extreme: beyond a certain loop depth, excessive recurrence degrades predictions -- the hidden state drifts past the solution and into noise. Adaptive Computation Time (ACT) halting addresses it with a learned scalar per position that dynamically decides when to stop looping.
Positions that are harder to process receive more computation; tokens that have already converged halt early. Finally, Depth-Wise LoRA adapters introduce a small rank-r adaptation matrix at each iteration depth, giving each loop step slightly distinct behavior without adding substantial parameters -- bridging the gap between pure weight-tying and fully distinct layers. Why Parameter Efficiency Matters The Parcae paper (Prairie et al., 2026) provides empirical grounding for the efficiency claim.
At 770M parameters, an RDT matches a 1.3B standard transformer trained on identical data -- roughly half the parameters for equivalent downstream quality.
OpenMythos arrives as a bold experiment. Built in PyTorch, the 770‑million‑parameter model claims to replicate the architecture that Anthropic has kept private. Its creators argue that the reconstruction follows first‑principles theory and peer‑reviewed research, not a leaked checkpoint or a distilled version.
The code is publicly available on GitHub, inviting scrutiny from the community. According to the project’s description, continuous latent thoughts can encode multiple alternative next steps simultaneously, which the authors say enables something akin to breadth‑first search over the reasoning space within a single forward pass. A standard transformer trained on five‑hop reasoning chains, they note, fails when tested on ten‑hop chains at inference, suggesting a potential advantage for the new design.
However, the model’s performance relative to Anthropic’s 1.3 B‑parameter Claude remains unverified outside the authors’ own benchmarks. It's unclear whether the theoretical reconstruction captures the nuances of the proprietary system or merely approximates its scale. The open‑source effort provides a concrete reference point, yet its claims await independent validation.
Further Reading
- OpenMythos Recasts Claude Mythos as Looped MoE Transformer - Awesome Agents
- Papers with Code - Latest NLP Research - Papers with Code
- Hugging Face Daily Papers - Hugging Face
- ArXiv CS.CL (Computation and Language) - ArXiv
Related Reading
Browse all Anthropic news →
Open Source UK PM vows action on Grok's deepfake scandal, Starmer condemns X
Open Source GPT-5 helps mathematicians offload tedious tasks, says Timothy Gowers
Open Source India proposes licensing and royalty rules for AI training by Google, OpenAI
Related Topic Google launches AI chips with 4× boost, lands Anthropic multibillion deal
Business & Startups
Related Topic Anthropic's Claude also citing Elon Musk's Grokipedia, reports say
LLMs & Generative AI
Same Week 91% of businesses now use video marketing — AI cut the cost of keeping up by 91% too
Market Trends
From Archives NVIDIA launches Ising, the first open quantum AI model family for hybrid systems
From Archives I Vibe codes tool to analyze call sentiment and topics from recordings
From Archives Anthropic's Claude Opus 4.7 lifts coding benchmark 13% and solves four new tasks
LLMs & Generative AI
From Archives Schematik ‘Cursor for Hardware’ secures USD 4.6M Lightspeed; Anthropic wants in
LLMs & Generative AI
Common Questions Answered
How does OpenMythos achieve comparable performance to the 1.3B Claude model with only 770 million parameters?
OpenMythos leverages a novel architectural approach that focuses on multi-step reasoning capabilities without increasing model size. The model uses continuous latent thoughts that can encode multiple alternative reasoning paths simultaneously, allowing it to handle complex inference tasks more efficiently than traditional transformers.
What makes the Recurrent-Depth Transformer unique in handling reasoning chains?
The Recurrent-Depth Transformer can naturally extend reasoning depth beyond its training constraints by running additional inference-time loops. Unlike standard transformers that fail when tested on reasoning chains longer than their training depth, this approach allows for more flexible and dynamic reasoning across multiple inference steps.
Why is the OpenMythos project significant for machine learning research?
OpenMythos democratizes access to a sophisticated transformer architecture previously kept private by Anthropic by providing an open-source PyTorch reconstruction. The project demonstrates that high-performance language models can be developed using first-principles theory and peer-reviewed research, potentially accelerating collaborative AI development.