Within Alignment & Governance
Can Compute Controls Actually Slow an AI Race?
Limits on the biggest AI training runs are often proposed as a way to slow dangerous capability races.
On this page
- How compute governance is supposed to work
- Why critics doubt enforcement and coordination
- What warning signs would justify tighter controls
Page outline Jump by section
Introduction
One reason some researchers assign a lower p(doom)—the estimated probability that advanced AI causes existential catastrophe—is that they believe governments could slow or shape the development of the most powerful systems through compute governance. In this context, “compute” means the specialised computing power needed to train and run frontier AI models. Because the largest AI systems require enormous quantities of advanced chips, electricity, data-centre capacity, and capital, some analysts argue that compute is one of the few practical choke points available to policymakers. [cdn.governance.ai]cdn.governance.aiComputing Power and the Governance of ArtificialFebruary 13, 2024 — 14 Feb 2024 — Computing power, or "compute," is crucial for the development and deployment of artificial intelligence…
The core idea is straightforward. If the most dangerous capabilities emerge only after very large training runs, then monitoring and regulating those runs could provide warning time, enable safety evaluations, reduce competitive pressure, and make it harder for reckless or malicious actors to develop frontier systems. Under that view, effective compute governance could lower p(doom) by reducing the chances of an uncontrolled AI race. Critics, however, argue that enforcement may be difficult, international coordination may fail, and technical changes could weaken compute-based controls over time. Institute for Law & AI [Default]Lawfareto govern ai we must govern computeDefaultTo Govern AI, We Must Govern Computeby L Heim · 2024 · Cited by 2 — Compute governance can support AI policy goals in multiple way…
How Compute Governance Is Supposed to Work
Unlike software algorithms, advanced AI chips are physical objects produced through a highly concentrated global supply chain. A relatively small number of companies manufacture the most advanced semiconductors, and frontier training runs typically require large clusters of those chips operating in specialised data centres. This concentration is one reason compute governance attracts attention among AI-risk researchers. Compute is often described as more detectable, quantifiable, and controllable than other AI inputs such as data or scientific knowledge. [cdn.governance.ai]cdn.governance.aiComputing Power and the Governance of ArtificialFebruary 13, 2024 — 14 Feb 2024 — Computing power, or "compute," is crucial for the development and deployment of artificial intelligence…
The most commonly discussed measures include:
- Mandatory reporting of exceptionally large training runs.
- Licensing requirements above specified compute thresholds.
- Independent safety evaluations before deployment.
- Monitoring of large chip clusters and data centres.
- Export controls on advanced AI chips.
- Hardware security features that help verify compliance with regulations. [ai-safety-atlas.com]ai-safety-atlas.comCompute GovernanceChapter 4Regulations have already begun using compute thresholds to trigger oversight mechanisms. The U.S. Executive Order on AI requires… [Effective Altruism Forum]forum.effectivealtruism.orgEffective Altruism Forum12 tentative ideas for US AI policy (Luke Muehlhauser)19 Apr 2023 — Security features on chips can be leveraged f…
Many proposals rely on compute thresholds. Instead of attempting to regulate every AI model, governments would focus on systems trained using exceptionally large amounts of computing power. Advocates argue that training compute correlates with frontier capabilities strongly enough to serve as a practical trigger for additional oversight. [arXiv]arxiv.orgarXivTraining Compute Thresholds: Features and Functions in…Aug 6, 2024 — We argue that training compute currently is the most suitabl… [Institute]law-ai.orgInstitute for Law & AIThe Role of Compute Thresholds for AI GovernanceThis article discusses the role of training compute thresholds, whi… for Law & AI
From an AI-doom perspective, the attraction is not merely regulation for its own sake. The hope is that oversight of the largest training runs could create opportunities to identify dangerous capabilities before they become widely deployed.
Why Doom Researchers See Compute as a Strategic Bottleneck
Many AI doom arguments involve some form of rapid capability growth. The concern is that highly capable systems could emerge before alignment and control methods are sufficiently reliable. If development proceeds at full speed under intense commercial and geopolitical competition, safety work may struggle to keep pace.
[Compute Governance]ai-safety-atlas.comChapter 4Regulations have already begun using compute thresholds to trigger oversight mechanisms. The U.S. Executive Order on AI requires… aims to alter that dynamic in several ways.
Slowing Capability Races
One common concern is that AI developers may feel pressured to deploy increasingly capable systems quickly because competitors are doing the same. If governments require reporting, testing, or licensing for the largest training runs, the pace of frontier development could become more predictable and less dependent on racing behaviour. [Institute for Law & AI]law-ai.orgInstitute for Law & AIThe Role of Compute Thresholds for AI GovernanceThis article discusses the role of training compute thresholds, whi…
For people worried about AI doom, even modest delays can matter. A six-month or one-year delay may not sound dramatic, but if alignment techniques, interpretability tools, and safety evaluations improve during that period, the resulting reduction in risk could be significant.
Increasing Visibility Into Frontier Development
Another argument is that compute governance creates visibility. Governments currently have limited insight into many aspects of advanced AI development. Large compute clusters, however, are harder to hide than software research alone.
Proponents argue that monitoring major training runs could help authorities answer questions such as:
- Who is building frontier systems?
- How rapidly are capabilities advancing?
- Which organisations possess the resources to train the next generation of models?
- When should additional safety measures be triggered? Blog - Lennart Heim+2Default [blog.heim.xyz]blog.heim.xyzCompute offers unique governance capacities for AIBlog - Lennart HeimCompute and the Governance of AI - TalkNov 5, 2023 — This talk explores the role of computational resources, or "compu…
For lower p(doom) forecasters, greater visibility reduces the chance that transformative capabilities arrive unexpectedly.
Limiting Proliferation
Some AI-risk arguments focus less on a single dangerous model and more on widespread proliferation. If frontier AI becomes cheap and easily available, more actors gain access to potentially dangerous capabilities.
Several governance proposals therefore seek to restrict access to the largest quantities of cutting-edge compute. Supporters argue that limiting proliferation could reduce the number of organisations capable of conducting frontier training runs and make coordinated safety measures easier to maintain. [Machine Intelligence Research Institute]intelligence.orgMachine Intelligence Research Institute AI Governance to Avoid ExtinctionMachine Intelligence Research InstituteAI Governance to Avoid ExtinctionMay 1, 2025 — by P Barnett · 2025 · Cited by 7 — Proliferation of…
Real-World Examples of Compute-Based Governance
Although comprehensive compute governance does not yet exist, elements of the approach have already appeared in policy.
The United States introduced reporting requirements for exceptionally large AI training runs through executive action, while the European Union’s AI framework incorporates compute-related thresholds for identifying especially capable models that may require additional scrutiny and risk-management measures. [ai-safety-atlas.com]ai-safety-atlas.comCompute GovernanceChapter 4Regulations have already begun using compute thresholds to trigger oversight mechanisms. The U.S. Executive Order on AI requires…
Export controls on advanced AI chips represent another example. While often discussed in geopolitical terms, some AI-safety advocates view export controls as a form of compute governance because they influence who can access the hardware needed for frontier AI development. [arXiv]arxiv.orgarXivTraining Compute Thresholds: Features and Functions in…Aug 6, 2024 — We argue that training compute currently is the most suitabl…
These measures are far from the comprehensive global monitoring systems envisioned by some researchers. Nevertheless, supporters see them as early demonstrations that compute can be used as a policy lever.
Why Critics Doubt Enforcement and Coordination
The strongest objections to compute governance do not usually concern the importance of compute itself. Instead, they concern whether governments can actually enforce restrictions in practice.
International Competition May Override Safety Goals
A recurring criticism is that countries may hesitate to slow domestic AI development if they believe rivals will continue advancing.
In this view, frontier AI resembles a strategic technology competition. If one nation imposes strict limits while others do not, political leaders may fear falling behind economically or militarily. Critics therefore argue that effective compute governance would require unusually high levels of international coordination. [Georgetown Journal]gjia.georgetown.eduJournal A Playbook for Winning the AI Race: Compete, CounterGeorgetown JournalA Playbook for Winning the AI Race: Compete, Counter…November 18, 2025 — 18 Nov 2025 — The challenge facing policyma…
This concern is particularly important for p(doom) estimates because many optimistic governance scenarios assume substantial cooperation among major AI powers.
Technical Progress Could Circumvent Thresholds
Another challenge is that compute thresholds are not fixed targets. Researchers continuously discover more efficient algorithms and training methods.
A model that requires enormous resources today might be trainable with much less compute in the future. Some analysts therefore worry that regulations tied to current compute levels could become outdated quickly. [Epoch AI]epoch.aithree issues undermining compute based ai policiesThree challenges facing compute-based AI policiesSep 11, 2025 — 'Training compute' is constantly evolving, and compute-based AI policies…
This does not necessarily invalidate compute governance, but it implies that thresholds may require frequent revision.
Distributed Training Could Make Monitoring Harder
Many proposals assume that frontier training requires large, visible data centres. Recent research has explored whether increasingly distributed training methods could weaken that assumption.
If developers can spread training across many locations or clusters, detection may become more difficult. Researchers examining this possibility generally conclude that countermeasures may exist, but they also highlight distributed training as a genuine challenge for long-term enforcement. [arXiv]arxiv.orgarXivTraining Compute Thresholds: Features and Functions in…Aug 6, 2024 — We argue that training compute currently is the most suitabl…
Hardware Verification Remains Immature
Some of the most ambitious proposals rely on hardware-level monitoring, cryptographic verification, or built-in chip security features.
While these ideas receive growing attention, several analyses note that many proposed verification mechanisms remain technically immature. The systems needed for robust international monitoring may take years to develop and standardise. [arXiv]arxiv.orgarXivTraining Compute Thresholds: Features and Functions in…Aug 6, 2024 — We argue that training compute currently is the most suitabl…
For sceptics, this creates a gap between theoretical governance proposals and currently deployable policy tools.
What Warning Signs Would Justify Tighter Controls?
Even people who support compute governance disagree about when stronger intervention should occur.
Several developments are frequently cited as potential triggers for tighter controls:
- Frontier models demonstrating dangerous autonomous capabilities.
- Evidence of successful deception during evaluations.
- Rapid increases in AI-driven scientific or engineering performance.
- Models showing the ability to accelerate AI research itself.
- Increasing concentration of frontier capability in a small number of training runs.
- Repeated failures of voluntary safety commitments. [GOV.UK]GOV.UKNovember 3, 2023 — 28 Apr 2025 — The risks posed by future Frontier AI will include the risks we see today, but with potential for larger… [AI Security Institute]aisi.gov.ukfrontier ai trends report“Training Compute of Frontier AI Models Grows by 4-5x per Year.Read more…
From an AI-doom perspective, the most important warning sign would be evidence that capability growth is outpacing humanity’s ability to understand, evaluate, or control advanced systems.
Supporters of compute governance argue that once such warning signs appear, it may be too late to design monitoring systems from scratch. They therefore favour building reporting mechanisms, data-centre visibility, and regulatory capacity before the most dangerous capabilities emerge. Critics counter that governments risk creating expensive and intrusive regulatory structures based on uncertain forecasts. [Default]Lawfareto govern ai we must govern computeDefaultTo Govern AI, We Must Govern Computeby L Heim · 2024 · Cited by 2 — Compute governance can support AI policy goals in multiple way… [Default]Lawfareto govern ai we must govern computeDefaultTo Govern AI, We Must Govern Computeby L Heim · 2024 · Cited by 2 — Compute governance can support AI policy goals in multiple way…
Why Compute Governance Changes Some p(doom) Estimates
Ultimately, compute governance affects p(doom) because it changes assumptions about whether society can intervene before frontier AI development becomes uncontrollable.
People with higher p(doom) estimates often assume that competitive pressures, enforcement failures, and international rivalry will overwhelm attempts at coordination. Under that view, compute controls either arrive too late or prove too weak to matter.
People with lower p(doom) estimates tend to believe that compute represents a rare governance advantage: a measurable, concentrated, physical input that can be monitored more easily than algorithms or knowledge. If governments can track frontier training runs, require evaluations, and coordinate responses to warning signs, then dangerous capability races may become more manageable. [cdn.governance.ai]cdn.governance.aiComputing Power and the Governance of ArtificialFebruary 13, 2024 — 14 Feb 2024 — Computing power, or "compute," is crucial for the development and deployment of artificial intelligence… [Default]Lawfareto govern ai we must govern computeDefaultTo Govern AI, We Must Govern Computeby L Heim · 2024 · Cited by 2 — Compute governance can support AI policy goals in multiple way…
The disagreement is therefore not mainly about whether compute matters. It is about whether institutions can successfully govern it before the most consequential AI systems arrive. For many debates about AI doom, that question remains one of the largest unresolved uncertainties. [Institute for Law & AI]law-ai.orgInstitute for Law & AIAdvanced AI Governance: A Literature Review of ProblemsNovember 1, 2023 — This literature review provides an overview and taxonomy of past and recent research in the emerging field of advanced…
Amazon book picks
Further Reading
Books and field guides related to Can Compute Controls Actually Slow an AI Race?. Use these as the next step if you want deeper reading beyond the article.
The Alignment Problem
Examines technical and policy challenges in aligning AI systems.
The Coming Wave
Covers regulation, control mechanisms, and governance responses to advanced AI.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: cdn.governance.ai
Title: Computing Power and the Governance of Artificial
Link: https://cdn.governance.ai/Computing_Power_and_the_Governance_of_AI.pdfSource snippet
February 13, 2024 — 14 Feb 2024 — Computing power, or "compute," is crucial for the development and [deployment]({{ 'release-gates/' | relative_url }}) of artificial intelligence...
Published: February 13, 2024
-
Source: law-ai.org
Link: https://law-ai.org/the-role-of-compute-thresholds-for-ai-governance/Source snippet
Institute for Law & AIThe Role of Compute Thresholds for AI GovernanceThis article discusses the role of training compute thresholds, whi...
-
Source: epoch.ai
Title: three issues undermining compute based ai policies
Link: https://epoch.ai/gradient-updates/three-issues-undermining-compute-based-ai-policiesSource snippet
Three challenges facing compute-based AI policiesSep 11, 2025 — 'Training compute' is constantly evolving, and compute-based AI policies...
-
Source: ai-safety-atlas.com
Title: Compute Governance
Link: https://ai-safety-atlas.com/chapters/v1/governance/compute-governanceSource snippet
Chapter 4Regulations have already begun using compute thresholds to trigger oversight mechanisms. The U.S. Executive Order on AI requires...
-
Source: arxiv.org
Link: https://arxiv.org/html/2405.10799v2Source snippet
arXivTraining Compute Thresholds: Features and Functions in...Aug 6, 2024 — We argue that training compute currently is the most suitabl...
-
Source: law-ai.org
Title: Institute for Law & AIAdvanced AI Governance: A Literature Review of Problems
Link: https://law-ai.org/advanced-ai-gov-litrev/Source snippet
November 1, 2023 — This literature review provides an overview and taxonomy of past and recent research in the emerging field of advanced...
Published: November 1, 2023
-
Source: blog.heim.xyz
Title: Compute offers unique governance capacities for AI
Link: https://blog.heim.xyz/compute-and-the-governance-of-ai-talk/Source snippet
Blog - Lennart HeimCompute and the Governance of AI - TalkNov 5, 2023 — This talk explores the role of computational resources, or "compu...
-
Source: intelligence.org
Title: Machine Intelligence Research Institute AI Governance to Avoid Extinction
Link: https://intelligence.org/wp-content/uploads/2025/05/AI-Governance-to-Avoid-Extinction.pdfSource snippet
Machine Intelligence Research InstituteAI Governance to Avoid ExtinctionMay 1, 2025 — by P Barnett · 2025 · Cited by 7 — Proliferation of...
Published: May 1, 2025
-
Source: arxiv.org
Link: https://arxiv.org/html/2506.20530v1Source snippet
arXivToward a Global Regime for Compute Governance25 Jun 2025 — Applied to frontier AI, a global export control regime would restrict the...
-
Source: gjia.georgetown.edu
Title: Journal A Playbook for Winning the AI Race: Compete, Counter
Link: https://gjia.georgetown.edu/science-technology/a-playbook-for-winning-the-ai-race-compete-counter-cooperate/Source snippet
Georgetown JournalA Playbook for Winning the AI Race: Compete, Counter...November 18, 2025 — 18 Nov 2025 — The challenge facing policyma...
Published: November 18, 2025
-
Source: governance.ai
Title: inference scaling and ai governance
Link: https://www.governance.ai/research-paper/inference-scaling-and-ai-governanceSource snippet
AI governance measures that rely on training-compute thresholds. Rapid scaling of inference-during-training would have more ambiguous eff...
-
Source: arxiv.org
Title: arXiv Does Distributed Training Undermine Compute Governance?
Link: https://arxiv.org/abs/2605.29359Source snippet
arXivDoes Distributed Training Undermine Compute Governance?May 28, 2026...
Published: May 28, 2026
-
Source: arxiv.org
Link: https://arxiv.org/abs/2604.04712 -
Source: GOV.UK
Link: https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper/future-risks-of-frontier-ai-annex-aSource snippet
November 3, 2023 — 28 Apr 2025 — The risks posed by future Frontier AI will include the risks we see today, but with potential for larger...
Published: November 3, 2023
-
Source: aisi.gov.uk
Title: frontier ai trends report
Link: https://www.aisi.gov.uk/frontier-ai-trends-reportSource snippet
“Training Compute of Frontier AI Models Grows by 4-5x per Year.Read more...
-
Source: arxiv.org
Title: arXiv Risk thresholds for frontier AI
Link: https://arxiv.org/abs/2406.14713 -
Source: Lawfare
Title: to govern ai we must govern compute
Link: https://www.lawfaremedia.org/article/to-govern-ai-we-must-govern-computeSource snippet
DefaultTo Govern AI, We Must Govern Computeby L Heim · 2024 · Cited by 2 — Compute governance can support AI policy goals in multiple way...
-
Source: Lawfare
Title: legal challenges to compute governance
Link: https://www.lawfaremedia.org/article/legal-challenges-to-compute-governanceSource snippet
Challenges to Compute Governanceby D Bernabei · 2024 · Cited by 1 — This article assesses the emergence of compute-based AI regulation, i...
-
Source: forum.effectivealtruism.org
Link: https://forum.effectivealtruism.org/posts/iiRGCydMX7aiEjvGm/12-tentative-ideas-for-us-ai-policy-luke-muehlhauserSource snippet
Effective Altruism Forum12 tentative ideas for US AI policy (Luke Muehlhauser)19 Apr 2023 — Security features on chips can be leveraged f...
-
Source: lesswrong.com
Title: compute governance literature review
Link: https://www.lesswrong.com/posts/eLzDLCB68qNoWDRba/compute-governance-literature-reviewSource snippet
25 Jun 2024 — Compute governance is a method for monitoring and regulating AI systems by tracking the hardware required to develop fronti...
-
Source: forum.effectivealtruism.org
Title: let s think about slowing down ai 1
Link: https://forum.effectivealtruism.org/posts/vwK3v3Mekf6Jjpeep/let-s-think-about-slowing-down-ai-1Source snippet
If the AI safety community tries to slow things down, it will disproportionately slow down progress in the...Read more...
Additional References
-
Source: siliconranch.substack.com
Link: https://siliconranch.substack.com/p/export-controls-are-ai-safetySource snippet
Export Controls Are AI SafetyExport controls do reduce existential risks, and they hopefully hinder the use of AI for human rights abuses...
-
Source: ai-frontiers.org
Link: https://ai-frontiers.org/topic/policy-and-regulationSource snippet
Policy & RegulationArticles in this section explore if, when, and how to implement regulation that harnesses AI's benefits while limiting...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/380720022_Training_Compute_Thresholds_Features_and_Functions_in_AI_GovernanceSource snippet
Features and Functions in AI GovernanceCompute thresholds provide a practical starting point for identifying potentially high-risk models...
-
Source: longtermresilience.org
Link: https://www.longtermresilience.org/response-to-the-uks-future-of-compute-review-a-missed-opportunity-to-lead-in-compute-governance/Source snippet
Response to the UK's Future of Compute Review13 Mar 2023 — Requiring AI companies to report, or possibly in the future apply for a licens...
-
Source: gov.ca.gov
Title: June 17 2025 – The California Report on Frontier AI Policy
Link: https://www.gov.ca.gov/wp-content/uploads/2025/06/June-17-2025-%E2%80%93-The-California-Report-on-Frontier-AI-Policy.pdfSource snippet
CALIFORNIA REPORT ON FRONTIER AI POLICYJun 17, 2025 — Training Compute Thresholds: Features and Functions in AI Regulation.... On the Li...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=TGQ0FtvAPK4Source snippet
Beating the AI Doom CycleAI inequality explored as access to frontier models becomes scarce and selectively allocated by security, comput...
-
Source: sciencedirect.com
Link: https://www.sciencedirect.com/science/article/pii/S030859612500014XSource snippet
ScienceDirectIt's just distributed computing: Rethinking AI governanceby ML Mueller · 2025 · Cited by 38 — Attempts to control AI can hav...
-
Source: lcfi.ac.uk
Title: What Role Can “COMPUTE” Play In AI Governance?
Link: https://www.lcfi.ac.uk/news-events/blog/post/what-role-can-compute-play-in-ai-governanceSource snippet
LCFI14 Feb 2024 — A major new 19-author report argues that governing computing power ('compute') can help AI governance be more effective...
-
Source: forethought.org
Title: Inference Scaling Reshapes AI Governance
Link: https://www.forethought.org/research/inference-scaling-reshapes-ai-governanceSource snippet
ForethoughtFeb 13, 2025 — The shift from scaling up the pre-training compute of AI systems to scaling up their inference compute may have...
-
Source: smarterarticles.co.uk
Link: https://smarterarticles.co.uk/capture-by-design-how-frontier-labs-wrote-ai-rules-before-regulators-arrivedSource snippet
Capture by Design: How Frontier Labs Wrote AI Rules Before...25 Apr 2026 — “[Anthropic]({{ 'anthropic-tests/' | relative_url }})'s dispute with US government exposes deeper rifts...
Topic Tree