Editorial illustration for Arcee releases Trinity-Large-TrueBase, a raw 10‑trillion‑token checkpoint
Trinity Large: First US-Built 10T Token Open Model
Arcee releases Trinity-Large-TrueBase, a raw 10‑trillion‑token checkpoint
Arcee’s newest model, Trinity‑Large, lands with a raw checkpoint that clocks in at ten trillion tokens—a scale most open‑source projects have never shown in an unmodified state. The release is labeled “U.S.-made, open source,” a phrase that hints at both technical ambition and a political angle. While many recent checkpoints arrive already tuned for instruction following or reinforced through costly feedback loops, Arcee is opting to share the model before those layers are applied.
That choice opens a window for researchers who want to study the base representation without the distortions introduced later in the pipeline. It also raises questions about how “open” a model can be when its underlying data and training decisions remain opaque. In a community where most contributions are filtered through proprietary fine‑tuning, a truly raw artifact could become a reference point for future work.
Sovereignty and the "TrueBase" philosophy...
Sovereignty and the "TrueBase" philosophy The most significant contribution of this release to the research community is Trinity-Large-TrueBase--a raw, 10-trillion-token checkpoint. Unlike nearly every other "open" release, which arrives after being "warped" by instruction tuning and reinforcement learning, TrueBase offers a rare, unspoiled look at foundational intelligence. In the rush to make models helpful, most labs apply supervised fine-tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) before the weights are released.
While this makes the model a better conversationalist, it can mask underlying knowledge distributions. TrueBase provides an "OG base model" that has not yet undergone the learning rate anneals or the phase two and three pre-training where instruction data is typically introduced. For researchers and enterprises in highly regulated industries, starting from TrueBase allows for authentic audits and custom alignment.
As Lucas Atkins, Arcee's CTO, noted in a video call with VentureBeat: "It's interesting like that checkpoint itself is already one of the best performing base models in the world". Technology: engineering through constraint The creation of Trinity Large was not a product of infinite resources, but rather what Atkins calls "engineering through constraint". Trained for approximately $20 million over just 33 days, the model represents a masterclass in capital efficiency.
Could this raw checkpoint finally give researchers a truly unaltered view of large‑scale language model behavior? Arcee’s decision to publish Trinity‑Large‑TrueBase—an untouched, 10‑trillion‑token checkpoint—offers exactly that, sidestepping the usual post‑training instruction tuning and reinforcement steps that obscure underlying patterns. The model itself, described as a 400‑something‑parameter architecture, is the lab’s largest publicly available offering, extending the company’s track record of building U.S.‑based LLMs from the ground up and sharing them under open or partially open licenses.
For solo developers and midsize enterprises, free access to such a sizable, raw model could lower entry barriers and enable deeper customisation. Yet the practical impact remains uncertain; without the typical fine‑tuning pipelines, it is unclear how many teams will adopt the checkpoint as‑is versus invest effort to reshape it for specific tasks. Moreover, the broader research community has yet to assess whether the “TrueBase” philosophy will translate into measurable advances or simply add another large artifact to an already crowded repository of open models.
Further Reading
- Trinity Large: An Open 400B Sparse MoE Model - Arcee AI - Arcee AI
- Trinity Large 400B: US $20M AI Strikes Back - ByteIota
- Arcee AI Launches Trinity Large, a 400B Sparse MoE AI Model - MLQ.ai
- Arcee AI goes all-in on open models built in the U.S. - Interconnects AI
Related Reading
Open Source UK PM vows action on Grok's deepfake scandal, Starmer condemns X
Open Source GPT-5 helps mathematicians offload tedious tasks, says Timothy Gowers
Open Source India proposes licensing and royalty rules for AI training by Google, OpenAI
From Archives Host Your Portfolio for Free on Hugging Face Spaces via GitHub
From Archives Nvidia’s China AI Chip Push Succeeds as Trump‑Era Logic Gains Backing
Common Questions Answered
What makes Trinity Large-TrueBase unique in the open-source AI model landscape?
Trinity Large-TrueBase is a raw 10-trillion-token checkpoint that provides an unmodified view of foundational language model intelligence. Unlike most open-source releases that undergo instruction tuning and reinforcement learning, this checkpoint offers researchers an unaltered look at the model's base capabilities.
How does the Trinity Large model's architecture differ from other open-source AI models?
Trinity Large is a 400B parameter sparse Mixture of Experts (MoE) model with 13B active parameters per token, using 256 experts with only 4 experts active per token. This unique architecture allows for extremely efficient training and inference, with Arcee claiming roughly 2-3x faster performance compared to peer models in the same weight class.
What is Arcee's motivation behind releasing a 'TrueBase' checkpoint?
Arcee aims to provide researchers with a truly unmodified view of large-scale language model behavior before typical post-training modifications like supervised fine-tuning and reinforcement learning. By releasing an untouched checkpoint, they hope to offer insights into the fundamental capabilities of AI models without the layers of subsequent refinement.