Editorial illustration for Anthropic Says Claude Leads Research, But Its AI Judge Could Repeat Errors
Anthropic Claims Claude Leads AI Research, Raises Concerns
Anthropic Says Claude Leads Research, But Its AI Judge Could Repeat Errors
Anthropic published a self-assessment this week claiming Claude "leads" 26 percent of the company's internal research work on future models, up from under one percent in February. The company frames the numbers as transparency in service of CEO Dario Amodei's push to slow AI development at the frontier in a coordinated way, arguing the public needs more visibility into how these systems actually get built, not just what they can do on benchmarks.
The disclosure rests on an autonomy scale borrowed from Epoch AI, running from AL0, meaning no AI involvement, up to AL5, full autonomy. Anthropic says more than 90 percent of its development work now clears AL3, and Claude never touches AL5 anywhere in the pipeline. The 26 percent figure that made the headline corresponds to AL4, which Epoch AI itself labels "AI leads." That word choice is doing a lot of work, and the scoring behind it comes from Claude judging its own contributions. Before anyone reads too much into a fraction that sounds like a machine running its own research, it's worth seeing exactly what the company means, and doesn't mean, by that label.
Anthropic is disclosing metrics on how fast it's building its own AI. According to the numbers, Claude "leads" 26 percent of the work on future models. But the underlying scale is fuzzy, the scoring comes from Claude itself, and "lead" means less than it sounds.
Why this matters
Anthropic is grading its own homework and asking us to trust the transcript. A Claude model reads Slack logs and internal docs, then another Claude model decides whether that counts as "leading" or merely "collaborating," and the company itself admits the judge can inherit the same blind spots as the system under review. For developers and researchers, that's the part worth sitting with: a self-referential scoring loop dressed up as a metric.
The 26 percent figure sounds precise, but precision means nothing if the categories underneath it are soft and self-assigned. Anthropic's own cross-check apparently found gaps, which is the most useful admission in the whole disclosure. If you're building on Claude, or benchmarking against it, treat this number as a marketing signal about internal velocity, not a verified capability claim.
The real story isn't that AI might be doing a quarter of Anthropic's research. It's that nobody outside the company can currently check that number against anything except more Claude output. Watch for whether Anthropic lets outside auditors near this process next, or whether "self-graded" becomes the industry norm.
Related Reading
Browse all Anthropic news →
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google tests visual 'magazine-style' UI for Gemini 3 Pro users
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic Claude gains shared context in Excel, PowerPoint; Microsoft adds Copilot Cowork
AI Tools & Apps
Related Topic Trump cracks down on Anthropic after Amazon tip; staff largely foreign
Open Source
From Archives Vercel Engineer Uses OpenAI's ChatGPT Luna 5.6 to Review Commands
From Archives WGA East Executive Director Condemns Tech Oligarchs Pushing AI
From Archives Anthropic's Claude Hacked OpenAI Systems in 72 Hours Via Forum Flaw
From Archives Claude Code Projects Coordinate Multiple AI Agents in the Cloud
Business & Startups
Common Questions Answered
What does Anthropic claim about Claude's role in its internal research work?
Anthropic claims that Claude "leads" 26 percent of the company's internal research work on future models, a significant increase from under one percent in February. The company frames this disclosure as transparency to support CEO Dario Amodei's push for slowing AI development at the frontier in a coordinated way.
What is the problem with Anthropic's autonomy scale and scoring methodology?
The underlying autonomy scale used to measure Claude's research contribution is described as fuzzy and imprecise. Additionally, the scoring itself comes from Claude models reading Slack logs and internal documents, creating a self-referential loop where Claude judges its own performance, which Anthropic admits can inherit the same blind spots as the system under review.
Why is the 26 percent figure potentially misleading according to the article?
The term "lead" in Anthropic's disclosure means less than it sounds, as it encompasses various levels of contribution that are not clearly defined. The scoring methodology relies on Claude models evaluating other Claude models' work, making the metric self-referential and potentially unreliable rather than an objective measure of research contribution.
What concern does the article raise about Anthropic grading its own research metrics?
The article highlights that Anthropic is essentially grading its own homework and asking the public to trust the results, which represents a problematic self-referential scoring loop dressed up as a metric. This approach raises questions about transparency and accountability, as the company uses its own AI systems to evaluate their own performance without independent verification.
Further Reading
- Anthropic says Claude leads 26% of its AI R&D work - Quartz
- Anthropic says Claude now leads a quarter of work building its next AI models - Reuters
- Anthropic says Claude now leads over a quarter of its AI R&D work - Bloomberg
- Claude Leads 26% of Anthropic's AI R&D - 36Kr
- Agentic Misalignment in Summer 2026 - Anthropic Alignment Science Blog
