Editorial illustration for OpenAI's Ted Sanders says model names can't be reduced to a single IQ score
Model Performance Isn't One IQ Score, OpenAI Says
OpenAI's Ted Sanders says model names can't be reduced to a single IQ score
Picking the right GPT model used to mean checking a benchmark table and moving on. Ted Sanders, who works on OpenAI's API team, says that approach misses how the company actually builds and names its models. In a recent breakdown of GPT-6.1 Terra, Sanders argued that performance isn't a single number you can rank models by. A model tuned for math problems won't necessarily write a better joke, and a cheaper model can beat a pricier one on specific tasks depending on training data and architecture, not just size.
Sanders pointed to three separate variables that shape how a model performs: its size, how much reasoning effort it applies, and whether it's running as part of a multi-agent system. Collapsing those into one "IQ score," he said, would hide more than it reveals. That's part of why OpenAI's naming scheme, with models like Terra, Astra, Sol, and Luna, can look confusing from the outside.
Size still explains most of the gap between models, Sanders said, but not all of it. His comments give some shape to how buyers might think about trade-offs between cost and capability when a new model lands.
Newer, smaller models can outperform older, larger ones through better training data or improved architectures. GPT-6.1 Terra is likely better than the larger Astra at technical communication and roughly matches it in image processing and computer use at a much lower price point, but falls short in most other areas, Sanders says.
Why this matters
For developers picking a model off OpenAI's shelf, Sanders's comments are a warning against shortcuts. A single score sounds convenient, but it hides the actual tradeoffs: size versus reasoning effort versus whether a task benefits from multi-agent setups. Teams that pick a model by "biggest number wins" will keep getting burned when a smaller, newer model beats an older flagship on their specific task, simply because of better training data.
The practical takeaway is that benchmarking needs to happen per use case, not per model name. If your product leans on math-heavy reasoning, test that directly rather than assuming a model's general reputation carries over to jokes, code, or summarization. Founders scoping costs should also note that reasoning effort and multi-agent orchestration are separate knobs from raw parameter count, meaning cheaper models can sometimes match pricier ones if you tune the right dimension.
It's also a reminder that naming conventions from OpenAI, or any lab, are marketing shorthand first and technical description second. Treat GPT-6.1 Terra's name as a label, not a spec sheet, until you've run your own tests.
Related Reading
Browse all OpenAI news →
LLMs & Generative AI ChatGPT's 'Nerdy' tweak rewards goblin metaphors in answers, study finds
LLMs & Generative AI Google to End Free Access to Gemini AI for Non-Subscribers
LLMs & Generative AI AI Engineers Face Rising Costs, Need New Strategies for Efficiency
Related Topic OpenAI's GPT Transcribe Cuts Error Rate to 3.31%, Improving on GPT-4o
Open Source
Related Topic OpenAI Developer: AI Agent "Swarms" Waste Tokens, Create "Coordination Tax
AI Tools & Apps
From Archives E2E Networks to Add 5.4 MW NVIDIA Blackwell AI Capacity in India
From Archives Claude Code vs Cursor: Field Test Shows No Single Winner in Real Use
From Archives OpenAI turned coding tools into dots, an always-on personal assistant
Business & Startups
From Archives GPT-6.1 Terra Halves OpenAI API Prices, Gains on Larger Models
Common Questions Answered
Why does Ted Sanders argue that model names can't be reduced to a single IQ score?
Sanders explains that performance isn't a single number you can rank models by because different models excel at different tasks depending on their training data and architecture. A model tuned for math problems won't necessarily write a better joke, and a cheaper model can beat a pricier one on specific tasks, making it impossible to rank them with one metric.
How does GPT-6.1 Terra compare to the larger Astra model according to Sanders?
GPT-6.1 Terra is likely better than the larger Astra at technical communication and roughly matches it in image processing and computer use, but at a much lower price point. However, Terra falls short compared to Astra in most other areas, demonstrating that newer, smaller models can outperform older, larger ones through better training data or improved architectures.
What are the main tradeoffs developers should consider when picking a GPT model?
Developers should evaluate tradeoffs including size versus reasoning effort versus whether a task benefits from multi-agent setups, rather than simply choosing the model with the biggest benchmark number. Picking a model by 'biggest number wins' often leads to poor results when a smaller, newer model actually beats an older flagship on a specific task due to superior training data.
Why do teams often make mistakes when selecting OpenAI models based on benchmark rankings?
Teams that rely on single benchmark scores as a shortcut miss the actual performance tradeoffs between different models. A single score sounds convenient but hides how different models are optimized for different capabilities, causing developers to get burned when a smaller, newer model unexpectedly outperforms an older flagship on their specific use case.
Further Reading
- Papers with Code - Latest NLP Research - Papers with Code
- Hugging Face Daily Papers - Hugging Face
- ArXiv CS.CL (Computation and Language) - ArXiv
