Within Task Horizons
Why AI Task Horizons Do Not Measure Running Time
A task horizon measures the human effort represented by a task, not how long the AI remains active while completing it.
On this page
- How human expert completion times set the scale
- Why AI agents may finish much faster than people
- Common mistakes when interpreting horizon figures
Page outline Jump by section
Introduction
When people first encounter AI task-horizon figures, one of the most common misunderstandings is to assume they measure how long an AI runs for. They do not. A task horizon measures the length of the task in human work time, not the AI’s wall-clock running time.
This distinction matters because task horizons are designed to answer a capability question rather than a speed question. Researchers ask: “What is the longest task, measured by how long it normally takes a skilled human, that an AI can reliably complete?” They are not asking how many minutes or hours the computer spends processing. An AI may finish a task in a fraction of the time taken by a human expert, or occasionally take longer because it repeatedly retries, but neither outcome changes the task’s measured horizon.[Metr]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
Within debates about AI doom and existential risk, this distinction is especially important. Concerns about dangerous autonomy depend primarily on what kinds of projects an AI can finish without human intervention, not on whether its processors remain busy for five minutes or five hours.
How human expert completion times set the scale
Task-horizon research begins by assigning a human timescale to each evaluation task.[aipatternbook.com]aipatternbook.comTask HorizonTask Horizon
Researchers recruit skilled human participants to complete well-defined technical tasks under comparable conditions. From these observations they estimate how long a competent human would normally require. AI agents are then evaluated on the same tasks, and researchers estimate the probability that the agent succeeds as task duration increases. The reported “50% time horizon”, for example, is the human task length at which the AI succeeds about half the time.[Metr]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
This means that a statement such as:
> “The model has a two-hour task horizon”[aipatternbook.com]aipatternbook.comTask HorizonTask Horizon
should be read as:
“The model can reliably complete tasks that would normally occupy a skilled human for around two hours.”
It should not be interpreted as:
“The model needs two hours of computation.”
Human time provides a common unit that readers can intuitively understand. Saying that an AI can complete “tasks equivalent to several hours of expert work” is generally more informative than reporting abstract benchmark percentages.
Why AI agents may finish much faster than people
One reason this metric is easy to misunderstand is that modern AI systems often execute successful tasks considerably faster than humans.
For software engineering tasks, an AI can frequently:
- generate large amounts of code almost instantly;
- inspect documentation rapidly;
- search files automatically;
- avoid the physical typing speed that constrains human programmers;
- perform some operations in parallel through software tools.
As a result, a task requiring several hours of concentrated human work may sometimes be completed by an AI agent in tens of minutes or less. METR explicitly notes that successful AI agents are typically several times faster than humans on the tasks they solve, although exact runtimes vary substantially depending on the model, inference provider and agent setup, so the project deliberately does not publish a single runtime figure.[Metr]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
This is analogous to measuring the lifting ability of different machines. A forklift and a crane may both lift a one-tonne load, but one may do so much faster than the other. The capability being measured is the size of the load, not the duration of the lift.
Why faster execution does not imply greater autonomy
Speed and autonomy are related but distinct properties.
An AI that answers every prompt almost instantly may still fail whenever a task requires:
- maintaining a plan across many decisions;
- recovering from mistakes;
- coordinating multiple tools;
- adapting when intermediate results are unexpected;
- remembering previous work over long sequences.
Conversely, an AI might spend considerable wall-clock time repeatedly attempting a difficult task yet still fail to complete it.
Task horizons therefore treat elapsed runtime as largely incidental. The metric is intended to reflect task difficulty as experienced by humans, because longer human tasks generally require more sustained planning, coordination and error recovery. Those are the abilities that become increasingly relevant when considering whether an AI could independently carry out substantial projects.[arXiv]arxiv.orgarXiv Measuring AI Ability to Complete Long TasksarXiv Measuring AI Ability to Complete Long Tasks
Why runtime varies so much
Even if two AI systems achieve identical task-horizon scores, their actual execution times may differ dramatically.
Several factors influence runtime:
- Inference speed. Different providers deliver tokens at different rates.
- Agent design. Some agents make many cautious intermediate checks, while others attempt larger solutions in fewer steps.
- Hardware. Faster processors reduce elapsed time without changing the underlying capability.
- Tool configuration. External search, testing and compilation introduce variable delays.
- Retry strategies. One agent may repeatedly revise failed attempts before succeeding.
Because these engineering choices change over time and differ between deployments, runtime is an unstable measurement. Human task duration is comparatively stable, making it a more useful capability scale. METR therefore separates capability measurement from implementation-specific execution speed.[Metr]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
Common mistakes when interpreting horizon figures
Several misconceptions appear repeatedly in public discussions.
Mistake: A four-hour horizon means the AI runs continuously for four hours.
No. It means the AI succeeds on tasks that typically require about four hours of skilled human work.
Mistake: Faster completion automatically means a higher horizon.
No. A model may solve two-hour human tasks in ten minutes, while another solves the same tasks in five minutes. Both may have essentially the same task horizon because they complete tasks of similar difficulty.
Mistake: Runtime improvements alone increase task horizons.
Making inference hardware faster or reducing latency does not necessarily enable an AI to solve harder tasks. A higher task horizon requires improved capability and reliability, not merely quicker execution.
Mistake: Human time assumes humans and AIs think in the same way.
The comparison is purely a measurement convention. Researchers are not claiming that AI reasoning resembles human reasoning, only that human completion time provides a familiar scale for comparing task difficulty.[Metr]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
Why this distinction matters in AI doom debates
Arguments about AI doom generally depend on whether advanced systems could autonomously complete increasingly long and complex chains of work without requiring continual human supervision.
Suppose two hypothetical systems each finish a task that would occupy a human software engineer for eight hours. One completes it in fifteen minutes, while the other takes three hours. For existential-risk analysis, the more important observation is usually that both systems successfully completed the full project. Their different runtimes may matter economically or operationally, but the key question about autonomy is whether they can sustain competent action across the entire task.
This is why task-horizon researchers deliberately frame their measurements around human task length rather than AI runtime. The metric is intended to estimate how much coherent work an AI can carry through to completion, because that is more closely connected to questions about dangerous autonomy, loss of human control and increasingly capable AI agents than simple execution speed. At the same time, researchers emphasise that these measurements come from specific benchmark distributions—primarily software engineering, machine learning and cybersecurity—and should not be interpreted as universal measures of autonomy across every real-world domain.[metr.org]metr.orgTask-Completion Time Horizons of Frontier AI ModelsTask-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026…
Amazon book picks
Further Reading
Books and field guides related to Why AI Task Horizons Do Not Measure Running Time. Use these as the next step if you want deeper reading beyond the article.
Human Compatible
A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...
Artificial Intelligence: A Modern Approach
Rating: 4.5/5 from 10 Google Books ratings
Artificial intelligence: A Modern Approach, 3e,is ideal for one or two-semester, undergraduate or graduate-level courses in Artificial In...
Superintelligence
This profoundly ambitious and original book picks its way carefully through a vast tract of forbiddingly difficult intellectual terrain.
Artificial Intelligence
Rating: 4.5/5 from 10 Google Books ratings
Artificial intelligence: A Modern Approach, 3e,is ideal for one or two-semester, undergraduate or graduate-level courses in Artificial In...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Selected fromtechnology art print oneBay.co.uk.
Endnotes
1.
Source: metr.org
Title: Task-Completion [Time Horizons]({{ ‘time-horizons/’ | relative_url }}) of Frontier AI Models
Link:https://metr.org/time-horizons/
Source snippet
Task-Completion Time Horizons of Frontier AI Models - METRMay 8, 2026...
Published: May 8, 2026
2.
Source: arxiv.org
Title: arXiv Measuring AI Ability to Complete Long Tasks
Link:https://arxiv.org/abs/2503.14499
3.
Source: metr.org
Title: ics of Agent Ability
Link:https://metr.org/notes/2026-07-24-metrics-of-model-ability/
Source snippet
We now discuss a set of metrics which are calibrated against human effort. An advantage of human-grounded metrics is their interpretabili...
4.
Source: metr.org
Title: Impact of modelling assumptions on time horizon results
Link:https://metr.org/notes/2026-03-20-impact-of-modelling-assumptions-on-time-horizon-results/
5.
Source: metr.org
Title: Time Horizon 1.1
Link:https://metr.org/blog/2026-1-29-time-horizon-1-1/?%3F%3F%3Futm_source=content
6.
Source: metr.org
Title: Clarifying limitations of time horizon
Link:https://metr.org/notes/2026-01-22-time-horizon-limitations/
7.
Source: metr.org
Title: How Does Time Horizon Vary Across Domains?
Link:https://metr.org/blog/2025-07-14-how-does-time-horizon-vary-across-domains/?_bhlid=6457d4cebb55c805dae1ad15c6cced22a1838d0d
8.
Source: metr.org
Link:https://metr.org/horizon-chart-embed
9.
Source: evals.alignment.org
Title: Evals Task-Completion Time Horizons of Frontier AI Models
Link:https://evals.alignment.org/time-horizons/
Source snippet
Task-Completion Time Horizons of Frontier AI Models - METR...
10.
Source: evals.alignment.org
Title: Evals How Does Time Horizon Vary Across Domains?
Link:https://evals.alignment.org/blog/2025-07-14-how-does-time-horizon-vary-across-domains/
Source snippet
How Does Time Horizon Vary Across Domains? - METR...
11.
Source: aiwiki.ai
Title: MET R | AI Wiki
Link:https://aiwiki.ai/wiki/metr
Source snippet
METR's most widely discussed research output is a paper titled "Measuring AI Ability to Complete Long [Software Tasks]({{ 'software-bias/' | relative_url }})," submitted to arXiv...
12.
Source: evals.alignment.org
Title: 2026 05 19 frontier risk report
Link:https://evals.alignment.org/blog/2026-05-19-frontier-risk-report/
13.
Source: evals.alignment.org
Link:https://evals.alignment.org/
14.
Source: aipatternbook.com
Title: Task Horizon
Link:https://aipatternbook.com/task-horizon
Additional References
15.
Source: americandefault.org
Title: HOURS OF HUMAN WORK AI CAN COMPLETE AU
Link:https://americandefault.org/indicators/the-horizon/
Source snippet
AI Task Horizon (METR, April 2026): 1044.8 hoursJune 25, 2026 — THE HORIZON Doubling roughly every four months since 2023; AI can now aut...
Published: June 25, 2026
16.
Source: blog.redwoodresearch.org
Title: estimating no cot task completion
Link:https://blog.redwoodresearch.org/p/estimating-no-cot-task-completion
Source snippet
No-CoT Task-Completion Time Horizons of Frontier AI ModelsJune 10, 2026 — ESTIMATING NO-COT TASK-COMPLETION TIME HORIZONS OF FRONTIER AI...
Published: June 10, 2026
17.
Source: youtube.com
Title: The AI Progress Chart Everyone Is Misreading — Beth Barnes & David Rein
Link:https://www.youtube.com/watch?v=zSAGzfspuDE
Source snippet
The Most Important Graph in AI Right Now | Beth Barnes, CEO of METR...
18.
Source: youtube.com
Title: Why Agent Hype can fall short of reality – Joel Becker, METR
Link:https://www.youtube.com/watch?v=RhfqQKe22ZA
Source snippet
METR time horizons human expert time 47 - David Rein on METR Time Horizons AXRP...
19.
Source: youtube.com
Title: The Most Important Graph in AI Right Now | Beth Barnes, CEO of METR
Link:https://www.youtube.com/watch?v=jXtk68Kzmms
Source snippet
Measuring AI Progress: The Task Completion Time Horizon...
20.
Source: youtube.com
Title: Measuring AI Progress: The Task Completion Time Horizon
Link:https://www.youtube.com/watch?v=xoDw7UFfj8A
Source snippet
Why Agent Hype can fall short of reality – Joel Becker, METR...
21.
Source: github.com
Link:https://github.com/METR/eval-analysis-public
Source snippet
GitHub - METR/eval-analysis-public: Public repository containing METR's DVC pipeline for eval data analysis · GitHub...
22.
Source: youtube.com
Link:https://www.youtube.com/watch?v=WaJhhD7Qgac
Source snippet
The AI Progress Chart Everyone Is Misreading — Beth Barnes & David Rein...
23.
Source: metavert.io
Link:https://www.metavert.io/metr-benchmarking
24.
Source: epoch.ai
Link:https://epoch.ai/benchmarks/metr-time-horizons