Editorial illustration for AI Agents Solved Tasks but Couldn't Track Time Accurately
AI Agents Fail at Time Estimation, Study Finds
AI Agents Solved Tasks but Couldn't Track Time Accurately
Ask Claude Code how long a coding job will take, and it will guess. Ask again after the work is done, and the answer barely changes, even when the real number is off by a factor of six or more. That gap shows up in a new study from two independent researchers working through the MATS program, who set out to test whether AI coding assistants have any working sense of time passing while they operate.
The researchers ran Anthropic's Claude Code and OpenAI's Codex through 200 tasks pulled from ProgramBench, then through 18 benchmarks they built themselves. Each agent had to predict how long a task would take before starting, then estimate afterward how much time had actually elapsed. Neither model treated the two questions as connected to real-world duration. On ProgramBench, both systems clustered their pre-task guesses around 90 minutes almost regardless of what the task actually demanded, easy or hard.
The errors got worse, not better, once the agents looked back at their own work. Short jobs produced the widest misses. Only tasks stretching across multiple hours drew estimates that landed anywhere close to accurate.
A new study finds that popular coding assistants can't predict how long a task will take, and they can't reliably tell how long they've already been working. That's a problem for long-running jobs.
Why this matters
For anyone deploying coding agents on tasks that run longer than a quick autocomplete, this is worth sitting with. If a model can't accurately judge how long it's been working, it can't reliably decide when to check in, ask for help, or flag that it's stuck. That's a real problem for founders building autonomous agent products marketed around "long-running" tasks: the marketing promise of an agent that grinds away for hours unsupervised runs into a model that has no internal clock to know whether it's five minutes or five hours into the job.
Researchers should treat this as a benchmark gap worth closing, not a side note. ProgramBench and the 18-task suite used here are small compared to production workloads, so we'd want to see this replicated at scale before drawing firm conclusions about specific models. But the direction is clear enough: time-awareness isn't something these systems pick up for free from being good at the underlying task.
Until that changes, anyone building on top of these assistants should assume the agent has no idea how long it's been running, and design timeouts and checkpoints accordingly rather than trusting the model's own sense of elapsed time.
Related Reading
Research & Benchmarks Google's FACTS benchmark shows 70% factuality ceiling across four tests
Research & Benchmarks Databricks finds multi-step agents beat single-turn RAG by 21% to 38% on STaRK
Research & Benchmarks Nvidia's DLSS 4.5 beta adds 6x Multi Frame Generation for RTX 50 GPUs
From Archives Anthropic's Claude Code limit change: a raise on paper, a cut in practice
From Archives Google's WikiSkill gives AI agents a persistent memory to avoid past mistakes
Common Questions Answered
What did the MATS study find about Claude Code's ability to estimate task duration?
The study found that Claude Code cannot accurately estimate how long coding tasks will take, and when asked again after completion, its estimates barely change even when the actual time differs by a factor of six or more. This demonstrates that the AI assistant has no reliable internal sense of time passing during task execution.
Which AI coding assistants were tested in this time-tracking study?
The researchers tested Anthropic's Claude Code and OpenAI's Codex through 200 tasks pulled from programming benchmarks. Both assistants were evaluated by independent researchers working through the MATS program to assess their ability to track elapsed time.
Why is the inability to track time a problem for long-running autonomous agent tasks?
If a coding agent cannot accurately judge how long it has been working, it cannot reliably decide when to check in with users, ask for help, or flag that it's stuck in a loop. This creates significant challenges for founders building autonomous agent products marketed around handling long-running tasks unsupervised, as the agents lack the internal clock needed to make these critical decisions.
What are the practical implications of AI agents not being aware of their time-tracking limitations?
The lack of awareness about their poor time-tracking abilities makes the problem worse, as agents cannot self-correct or compensate for their temporal blindness. This means they will confidently provide inaccurate time estimates and continue working without proper checkpoints or escalation protocols, which is particularly problematic for long-duration tasks that require human oversight.
Further Reading
- AI Coding Agent Benchmarks & Leaderboard - Artificial Analysis
- Measuring AI Ability to Complete Long Software Tasks - METR
- Establishing Best Practices for Building Rigorous Agentic Benchmarks - arXiv
- Benchmarking AI Coding Agents on End-to-End Project Development - arXiv
- The Missing Infrastructure for AI Agent Evaluation - arXiv
