Within First Movers
Why waiting can feel too expensive
Competitive pressure can make extra safety testing feel like a strategic sacrifice even when every major developer says safety matters.
On this page
- The payoff from deploying first
- Why shared safety benefits become externalities
- When rational choices produce collective danger
Page outline Jump by section
Introduction
A recurring claim in AI doom discussions is that safety is not always ignored because developers are careless. Instead, safety work can become strategically expensive. When organisations believe that being first to deploy a highly capable AI system brings major rewards, every month spent on additional testing, interpretability work, red-teaming, or safeguards may feel like a competitive loss.
This creates a specific mechanism within the broader debate about first-mover advantage and premature deployment. Even if all major developers publicly support safety, each may worry that a rival will capture users, investment, influence, or strategic advantage during any delay. In that environment, waiting can look less like responsible caution and more like surrendering a lead. Critics of the AI doom argument dispute how strong these pressures really are, but many researchers studying AI competition view race incentives as a plausible reason why organisations could deploy systems before their risks are fully understood. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha…
The payoff from deploying first
The key idea is simple: the benefits of moving first are often concentrated, while the benefits of extra safety work are widely shared.
For a frontier AI developer, deploying earlier may help secure market share, attract investment, recruit talent, establish technical standards, and gather valuable real-world usage data. If powerful AI systems become economically transformative, these advantages could be enormous. As a result, even relatively short delays may appear costly from the perspective of executives, investors, or governments. [Taylor & Francis Online]tandfonline.comTaylor & Francis Online Arms Race or Innovation Race?Geopolitical AI Developmentby S Schmid · 2025 · Cited by 45 — We argue that the arms race metaphor does not capture the dynamics of globa…
In AI doom arguments, the concern is not merely that companies like success. The concern is that the expected value of being first may grow faster than confidence in safety. A developer might believe that another six months of evaluation would uncover important failure modes, yet still feel pressure to deploy because competitors are moving quickly.
Economic models of AI deployment timing suggest that competition can create a “race to the bottom” effect in which firms deploy earlier than they would if they were jointly maximising long-term welfare. The individually rational choice can become faster deployment, even when all participants recognise that additional safety work would be beneficial. [CEPR]cepr.orgCEPRDP21454 AI Safety and Competitionby JP Choi · 2026 — This paper examines how competition affects the timing of AI deployment under sa…
A useful way to think about the mechanism is that every delay has two prices:
- The direct cost of continued research and testing.
- The opportunity cost of allowing competitors to advance.
When the second cost becomes large enough, safety delays begin to resemble strategic sacrifices.
Why shared safety benefits become externalities
The central economic argument concerns externalities. An externality exists when the actor making a decision does not capture all of the resulting costs or benefits.
Extra safety testing can reduce the probability of serious failures that would affect users, competitors, governments, and society more broadly. However, the organisation paying for the testing may receive only part of that benefit. Meanwhile, the commercial gains from releasing earlier largely accrue to the organisation itself.
This creates an asymmetry:
- The developer bears most of the cost of waiting.
- Society receives much of the benefit of waiting.
As a result, the amount of safety work that is optimal for an individual organisation may be lower than the amount that would be optimal from society’s perspective. Researchers examining responsible AI development have argued that competitive pressures can encourage firms to underinvest in safety, security, and broader risk reduction measures even when leaders recognise their importance. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha…
In AI doom scenarios, this matters because the potential harms being discussed are not localised product failures. They include possibilities such as severe loss of control, dangerous autonomy, or other failures with consequences extending far beyond the organisation that made the deployment decision. If risks are globally shared but deployment benefits are concentrated, market incentives may systematically favour earlier deployment than many observers would consider prudent.
When rational choices produce collective danger
The classic comparison is a prisoner’s dilemma.
Imagine two frontier AI developers. Each would prefer a world in which both organisations conduct extensive safety testing before release. Yet each also knows that if it alone pauses while the other continues, it may lose market position, investment, talent, or strategic influence.
The resulting logic can look like this:
- Both developers recognise that additional safety work has value.
- Each fears losing ground if it pauses unilaterally.
- Both accelerate development or deployment.
- Safety precautions end up weaker than either would ideally choose in isolation.
Importantly, this mechanism does not require bad actors. It can emerge precisely because organisations are responding rationally to incentives. The collective outcome may be worse even when individual decisions make sense from the viewpoint of each participant.
Game-theoretic models of technology races have repeatedly found conditions under which actors choose lower levels of precaution than would maximise overall welfare, particularly when rewards for winning are large and timelines are short. Research specifically examining AI races reaches similar conclusions, although the magnitude of the effect remains uncertain. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha… [PMC]pmc.ncbi.nlm.nih.govintelligence development races in heterogeneous…by T Cimpeanu · 2022 · Cited by 54 — Here we investigate how different interaction str…
What evidence do doomers point to?
One challenge is that no AI system has yet produced the kind of catastrophic loss-of-control event that AI doom researchers worry about. Evidence therefore comes mostly from observed incentives, organisational behaviour, historical technology races, and formal modelling rather than direct proof.
Several strands of evidence are commonly cited.
First, simulation and gaming exercises exploring AI race dynamics repeatedly find competitive pressures pushing participants towards acceleration. Researchers analysing dozens of “Intelligence Rising” simulations reported recurring patterns in which race dynamics emerged, safety concerns were deprioritised, and cooperation proved difficult to sustain under pressure. [ScienceDirect]sciencedirect.comScienceDirectStrategic insights from simulation gaming of AI race dynamicsby R Gruetzemacher · 2025 · Cited by 14 — Our analysis reveals…
Second, AI governance researchers have long warned that collective-action problems could cause underinvestment in safety. The argument is not that companies dislike safety, but that competition can make safety expenditures harder to justify when rivals are moving quickly. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha…
Third, observers point to real-world debates within frontier AI companies. In recent years, several major labs have publicly discussed the tension between maintaining safety commitments and remaining competitive. Critics interpret such debates as evidence that race pressures are not merely theoretical. Supporters of the companies involved argue that adapting safety frameworks can reflect changing circumstances rather than abandonment of safety goals. [Business Insider]businessinsider.comThe company will no longer unilaterally pause or delay new AI model deployments when safety mechanisms lag, citing increased competition…
None of these examples demonstrates that an existential catastrophe will occur. They are instead used to support a narrower claim: competitive incentives capable of reducing safety margins appear to exist.
Why this matters for AI doom
Race incentives matter in AI doom arguments because many proposed safeguards require time.
Interpretability research, alignment testing, capability evaluations, adversarial red-teaming, monitoring systems, and emergency-response procedures all involve delays, costs, or deployment restrictions. If organisations believe that competitive advantage depends heavily on speed, these measures may become increasingly difficult to maintain.
The concern is especially acute for scenarios involving rapid capability advances. If AI systems become dramatically more capable over short periods, the gap between “technically possible” and “safely understood” could widen. In that world, competition might encourage deployment before researchers have established reliable ways to predict or control system behaviour.
The International AI Safety Report highlights the broader challenge that capabilities can advance faster than understanding and risk-management techniques. For doom-focused researchers, race dynamics are one reason that this gap could persist rather than naturally correcting itself. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha…
The strongest objections
Not everyone accepts that race incentives are likely to produce dangerous underinvestment in safety.
One objection is that safety itself may become a competitive advantage. A company that releases unreliable or dangerous systems could lose customers, face legal liability, suffer reputational damage, or trigger regulatory intervention. From this perspective, markets may create incentives for at least some forms of safety.
Another objection is that the “AI race” metaphor can be overstated. Some researchers argue that AI development resembles a complex innovation ecosystem rather than a simple winner-takes-all contest. Collaboration, shared research, supply-chain interdependence, and regulatory constraints may weaken pure race dynamics. [Taylor & Francis Online]tandfonline.comTaylor & Francis Online Arms Race or Innovation Race?Geopolitical AI Developmentby S Schmid · 2025 · Cited by 45 — We argue that the arms race metaphor does not capture the dynamics of globa…
A third objection is empirical uncertainty. The strongest doom claims often depend on future systems becoming extraordinarily powerful and difficult to control. If such systems never emerge, or if alignment techniques improve quickly, then competitive deployment pressure may prove far less consequential than doomers expect.
These objections do not eliminate the race-incentive argument. They instead highlight a central uncertainty: nobody knows how powerful future AI systems will become, how large first-mover advantages will be, or how effectively safety and competition can be balanced.
Why coordination is often proposed as the solution
The importance of race incentives explains why coordination appears so frequently in AI-risk discussions.
If safety delays are costly because competitors continue advancing, then the obvious way to reduce the cost is to ensure that competitors face similar constraints. Shared testing standards, coordinated evaluations, reporting requirements, compute monitoring, international agreements, and industry-wide safety commitments are all attempts to reduce the penalty for caution.
The underlying goal is not necessarily to stop competition. It is to prevent a situation in which every actor believes that slowing down would be responsible but also believes that slowing down alone would be self-destructive.
For AI doom advocates, race incentives are therefore significant not because they guarantee catastrophe, but because they provide a mechanism through which intelligent, well-intentioned organisations could collectively create more risk than any of them would deliberately choose. [arXiv]arxiv.orgarXiv The Role of Cooperation in Responsible AI DevelopmentarXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha… [ScienceDirect]sciencedirect.comScienceDirectStrategic insights from simulation gaming of AI race dynamicsby R Gruetzemacher · 2025 · Cited by 14 — Our analysis reveals…
Amazon book picks
Further Reading
Books and field guides related to Why waiting can feel too expensive. Use these as the next step if you want deeper reading beyond the article.
The Logic of Collective Action
Rating: 4.5/5 from 6 Google Books ratings
Foundational explanation of shared-benefit coordination failures.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: arxiv.org
Title: arXiv The Role of Cooperation in Responsible AI Development
Link: https://arxiv.org/pdf/1907.04534Source snippet
arXivThe Role of Cooperation in Responsible AI DevelopmentJuly 10, 2019 — by A Askell · 2019 · Cited by 110 — In this paper, we argue tha...
Published: July 10, 2019
-
Source: cepr.org
Link: https://cepr.org/publications/dp21454Source snippet
CEPRDP21454 AI Safety and Competitionby JP Choi · 2026 — This paper examines how competition affects the timing of AI deployment under sa...
-
Source: arxiv.org
Link: https://arxiv.org/abs/1907.12393Source snippet
arXivTo regulate or not: a social dynamics analysis of the race for AI supremacyJuly 26, 2019...
Published: July 26, 2019
-
Source: pmc.ncbi.nlm.nih.gov
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC8810789/Source snippet
intelligence development races in heterogeneous...by T Cimpeanu · 2022 · Cited by 54 — Here we investigate how different interaction str...
-
Source: sciencedirect.com
Link: https://www.sciencedirect.com/science/article/pii/S0016328725000254Source snippet
ScienceDirectStrategic insights from simulation gaming of AI race dynamicsby R Gruetzemacher · 2025 · Cited by 14 — Our analysis reveals...
-
Source: arxiv.org
Title: arXiv Strategic Insights from Simulation Gaming of AI Race Dynamics
Link: https://arxiv.org/abs/2410.03092Source snippet
Strategic Insights from Simulation Gaming of AI Race...by R Gruetzemacher · 2024 · Cited by 14 — Our analysis reveals key strategic cons...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2602.21012Source snippet
arXiv[2602.21012] International AI Safety Report 2026by Y Bengio · 2026 · Cited by 74 — The International AI Safety Report 2026 synthesis...
-
Source: themoonlight.io
Link: https://www.themoonlight.io/en/review/strategic-insights-from-simulation-gaming-of-ai-race-dynamics -
Source: sciencedirect.com
Link: https://www.sciencedirect.com/journal/futures/vol/167/suppl/CSource snippet
Futures | Vol 167, March 2025Strategic insights from simulation gaming of AI race dynamics. Ross Gruetzemacher, Shahar Avin, James Fox, A...
-
Source: tandfonline.com
Title: Taylor & Francis Online Arms Race or Innovation Race?
Link: https://www.tandfonline.com/doi/full/10.1080/14650045.2025.2456019Source snippet
[Geopolitical]({{ 'geopolitics/' | relative_url }}) AI Developmentby S Schmid · 2025 · Cited by 45 — We argue that the arms race metaphor does not capture the dynamics of globa...
-
Source: businessinsider.com
Link: https://www.businessinsider.com/anthropic-changing-safety-policy-2026-2Source snippet
The company will no longer unilaterally pause or delay new AI model deployments when safety mechanisms lag, citing increased competition...
-
Source: s-rsa.com
Link: https://s-rsa.com/index.php/agi/article/view/16439/11861Source snippet
A Race dynamics risk safe and responsible AI. B Why collaborate? C Forms of collaboration. III Antitrust...Read more...
Additional References
-
Source: linkedin.com
Link: https://www.linkedin.com/pulse/3rd-edition-ai-race-competition-dynamics-alyssa-christensen-nqxweSource snippet
3rd Edition: The AI Race and Competition DynamicsThe Center for AI Safety defines the AI Race as a dynamic in which competitive incentive...
-
Source: intelligencerising.org
Link: https://www.intelligencerising.org/insights-and-resources-2-2Source snippet
Our publicationsStrategic Insights from Simulation Gaming of AI Race Dynamics. AI Future: Insights from 43 Intelligence Rising Games. Ros...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/334388570_The_Role_of_Cooperation_in_Responsible_AI_DevelopmentSource snippet
The Role of Cooperation in Responsible AI DevelopmentIn this paper, we argue that competitive pressures could incentivize AI companies to...
-
Source: axios.com
Link: https://www.axios.com/2026/03/03/ai-race-safety-guardrailSource snippet
Even traditionally cautious firms, like Anthropic, have recently revised their internal guidelines, narrowing criteria for delaying risky...
-
Source: techuk.org
Link: https://www.techuk.org/resource/the-release-of-the-international-ai-safety-report-2026-navigating-rapid-ai-advancement-and-emerging-risks.htmlSource snippet
The release of the international AI safety report 20263 Feb 2026 — The International AI Safety Report 2026 has been released today on 3 F...
-
Source: institute.global
Link: https://institute.global/insights/tech-and-digitalisation/europe-in-the-age-of-ai-how-technology-leadership-can-boost-competitiveness-and-securitySource snippet
Europe in the Age of AI: How Technology Leadership Can...17 Nov 2025 — Competitiveness in the AI era depends on affordable, sustainable...
-
Source: linkedin.com
Title: welker international ai safety report 2026 activity 7424732745643380736 o3XA
Link: https://www.linkedin.com/posts/welker_international-ai-safety-report-2026-activity-7424732745643380736-o3XASource snippet
2026 AI Safety Report: Emerging Risks from General...The 2026 AI Safety Report underscores how quickly the landscape is evolving from ra...
-
Source: aikido.dev
Title: international ai safety report aikido security analysis
Link: https://www.aikido.dev/blog/international-ai-safety-report-aikido-security-analysisSource snippet
International AI Safety Report 2026: Aikido Security Analysis9 Feb 2026 — The International AI Safety Report 2026 is one of the most comp...
-
Source: researchgate.net
Title: Strategic Insights from Simulation Gaming of AI Race
Link: https://www.researchgate.net/publication/389032252_Strategic_Insights_from_Simulation_Gaming_of_AI_Race_DynamicsSource snippet
AI. This paper assesses the potential risks of the AI race narrative and of an actual competitive race to develop AI, such as incentivisi...
-
Source: semanticscholar.org
Link: https://www.semanticscholar.org/paper/Strategic-Insights-from-Simulation-Gaming-of-AI-Gruetzemacher-Avin/a19fb918a8e9b56d26cfa07f9b7ba5ad458ec97bSource snippet
ents, challenges to the robustness of such agreements, the critical role of...
Topic Tree