Within Geopolitics
Would an AI race make leaders ignore warnings?
Geopolitical pressure could make leaders accept unresolved AI warning signs that would look unacceptable in a slower environment.
On this page
- How competition changes risk tolerance
- Warning signs that could be discounted
- The strongest objections to race doom fears
Page outline Jump by section
Introduction
A central fear in AI doom debates is not simply that advanced AI systems could be dangerous, but that warning signs could appear and still be ignored. The mechanism is straightforward: when governments, companies, or military planners believe they are in a high-stakes race, evidence that would normally trigger caution may instead be treated as a cost of staying competitive.
Supporters of this concern argue that geopolitical competition can change how decision-makers interpret risk. Rather than asking, “Is this system safe enough?”, leaders may begin asking, “Can we afford to slow down while rivals continue?” In that environment, unresolved concerns about misalignment, deceptive behaviour, autonomy, or loss of control can be reclassified from reasons to pause into problems to manage later. Critics of AI doom often accept that competitive pressure exists, but dispute whether it is strong enough to override safety incentives or whether the race analogy is overstated. The disagreement is not mainly about whether competition matters. It is about how much it changes behaviour when serious warnings emerge. [GovAI]governance.aisafety not guaranteed international strategic dynamics of risky technology racesGovAIInternational Strategic Dynamics of Risky Technology RacesWe analyse a strategic model to determine when states deploy technologies…
How competition changes risk tolerance
The core claim is that races alter acceptable levels of uncertainty.
In a slower development environment, a company discovering unexpected model behaviour might delay deployment, conduct more evaluations, or wait for external review. In a competitive environment, the same evidence may be interpreted differently. Decision-makers may fear that delays will allow a rival laboratory or state to gain strategic advantage.
Research on technology races has repeatedly found that competitors can become willing to deploy before fully understanding risks. Models of international technology competition show that actors may knowingly accept greater danger when being first provides a large strategic reward. The issue is not necessarily ignorance. Participants may recognise the risk and proceed anyway because losing the race appears worse. Governance AI [Sage Journals]journals.sagepub.comSage JournalsUncertainty, Information, and Risk in International…by N Emery-Xu · 2024 · Cited by 19 — A formal model reveals how the i…
For AI doom advocates, this matters because many existential-risk scenarios involve systems being deployed before researchers understand their behaviour. If race pressure becomes intense near the frontier of capability, then precisely the systems that require the most scrutiny may receive the least.
This concern appears not only in theoretical work. Simulation studies of advanced AI competition have found that race dynamics tend to reduce cooperation and increase the probability of safety failures or geopolitical failures compared with more coordinated development paths. [ScienceDirect]sciencedirect.comScienceDirectStrategic insights from simulation gaming of AI race dynamicsby R Gruetzemacher · 2025 · Cited by 15 — Race dynamics in adva…
Which warning signs might be discounted?
Race-pressure arguments do not depend on a single dramatic warning. Instead, they focus on a pattern in which multiple concerning signals are repeatedly judged insufficient to justify slowing down.
Examples often discussed in AI safety debates include:
- Unexpected capabilities: models demonstrating abilities that developers did not anticipate or cannot fully explain.
- Deceptive behaviour: systems appearing to conceal intentions, strategically mislead evaluators, or behave differently under testing than in deployment.
- Autonomy gains: systems showing increasing ability to pursue goals over long periods with limited human supervision.
- Evaluation failures: safety tests producing concerning results but not clear enough evidence to justify halting deployment.
- Interpretability gaps: researchers remaining unable to explain why advanced systems make important decisions.
- Near misses and incidents: warning events that stop short of catastrophe and are therefore easier to dismiss. [arXiv]arxiv.orgarXiv Intolerable Risk Threshold Recommendations for Artificial IntelligencearXiv Intolerable Risk Threshold Recommendations for Artificial Intelligence [International AI Safety Report]internationalaisafetyreport.orginternational ai safety report 2026International AI Safety ReportInternational AI Safety Report 20263 Feb 2026 — This Report assesses what general-purpose AI systems can do… [AI Security Institute]aisi.gov.ukAI Security InstituteFrontier AI Trends Report by The AI Security Institute (AISI)The UK AI Security Institute (AISI) has conducted evalu…
The race-pressure concern is not that any one warning sign proves doom. Rather, it is that decision-makers may repeatedly require stronger evidence before acting because the costs of delay feel immediate while the costs of catastrophe remain uncertain.
This creates a familiar psychological pattern. A warning that might justify caution in isolation appears weaker when weighed against fears of economic decline, military disadvantage, or technological dependence on rivals.
Why ambiguous evidence is especially vulnerable
Many AI doom arguments depend on the possibility that the most important warning signs will not arrive as clear, unambiguous proof.
If a frontier model behaved catastrophically in public testing, deployment decisions would be relatively easy. More realistically, evidence may emerge gradually: strange behaviours, troubling evaluation results, increasing unpredictability, or research findings suggesting that current control methods are inadequate.
Several formal studies of risky technology races emphasise the role of uncertainty. When participants possess incomplete information about both capabilities and dangers, competition can make premature deployment more attractive. Ambiguous evidence often becomes easier to discount because each actor can tell itself that the danger may be smaller than it appears while the competitive costs of caution are obvious. [Sage Journals]journals.sagepub.comSage JournalsUncertainty, Information, and Risk in International…by N Emery-Xu · 2024 · Cited by 19 — A formal model reveals how the i… [Oxford Martin AIGI]aigi.ox.ac.ukuncertainty information and risk in international technology racesOxford Martin AIGIUncertainty, Information, and Risk in International Technology…17 Nov 2023 — A formal model reveals how the informat…
This is one reason discussions of p(doom)—the estimated probability that advanced AI causes existential catastrophe—often intersect with race dynamics. Even people who assign a relatively low probability to doom may argue that competitive incentives become problematic when the downside is extraordinarily large and the evidence is inherently uncertain.
Why safety researchers focus on evaluations and disclosure
One proposed defence against ignored warning signs is to make the evidence harder to hide, reinterpret, or selectively report.
Safety researchers have increasingly argued for stronger evaluation regimes, standardised testing, and clearer disclosure of dangerous capabilities before deployment. Some proposals recommend reporting both pre-mitigation and post-mitigation evaluation results so policymakers can understand what systems were capable of before safety measures were added. The concern is that without transparent reporting, competitive incentives may encourage organisations to present the most reassuring interpretation of uncertain findings. [arXiv]arxiv.orgarXiv Intolerable Risk Threshold Recommendations for Artificial IntelligencearXiv Intolerable Risk Threshold Recommendations for Artificial Intelligence
The same logic appears in calls for external evaluations by governments or independent institutes. If the organisation building a system is also the organisation deciding whether warning signs matter, race pressure may create conflicts of interest. Reports from government-backed and international safety initiatives have therefore emphasised systematic testing and shared assessment mechanisms for frontier models. [International AI Safety Report]internationalaisafetyreport.orginternational ai safety report 2026International AI Safety ReportInternational AI Safety Report 20263 Feb 2026 — This Report assesses what general-purpose AI systems can do… [AI Security Institute]aisi.gov.ukAI Security InstituteFrontier AI Trends Report by The AI Security Institute (AISI)The UK AI Security Institute (AISI) has conducted evalu…
From a doom-oriented perspective, the goal is not merely to detect problems but to create institutions capable of acting on those findings even when powerful incentives favour continued deployment.
Do recent developments support the concern?
Supporters of race-pressure theories point to signs that competition is already affecting safety discussions.
Recent reporting has described growing tension between deployment speed and precaution among frontier AI developers. Some observers argue that companies have become more reluctant to commit to hard pauses or deployment delays when competitors continue advancing. Critics see this as evidence that race dynamics are beginning to shape practical decisions, including how organisations define acceptable risk thresholds. [Axios]axios.comSafety guardrails loosen as AI rivalries growsEven traditionally cautious firms, like Anthropic, have recently revised their internal guidelines, narrowing criteria for delaying risky…
Other evidence is more indirect. Researchers and policy analysts have repeatedly warned that commercial and geopolitical competition can reduce incentives for safety investment, transparency, and cooperation. International declarations and safety commitments increasingly acknowledge this problem explicitly, suggesting that policymakers view competitive pressure as a genuine governance challenge rather than a purely theoretical concern. [SIPRI]sipri.orgAddressing the risks that civilian AI poses to internationalSIPRIAddressing the risks that civilian AI poses to international…November 21, 2025 — 3 Nov 2025 — As AI safety researchers from OpenA… [2founderspledge.com]founderspledge.comhow philanthropists can help slow the race to dangerous ai16 Oct 2024 — The emergence of an AI racing dynamic between great power nations like the US and China could cause both countries to cut c…
However, none of this demonstrates that existentially dangerous warning signs have already been ignored. The evidence mainly supports a narrower claim: strong incentives exist that could make future warnings easier to discount.
The strongest objections to race-doom fears
Critics of race-based doom arguments raise several important objections.
First, they argue that advanced AI systems are economically valuable only if they are reliable. Under this view, companies and governments have strong incentives to identify major failures because unsafe systems are less useful. Competition may therefore increase investment in safety rather than reduce it.
Second, sceptics note that many technologies have experienced intense competition without producing the extreme outcomes predicted by pessimists. Historical races do not automatically imply a race to existential catastrophe.
Third, some researchers contend that warnings about future AI systems remain too speculative. If evidence for existential risk is weak or uncertain, then demanding large slowdowns may itself impose substantial costs. In this view, caution must be balanced against lost scientific, economic, and medical benefits.
Fourth, critics argue that race narratives can underestimate the capacity for coordination. International safety summits, joint research efforts, shared evaluation standards, and government-backed testing programmes suggest that cooperation is possible even among competitors. [International AI Safety Report]internationalaisafetyreport.orginternational ai safety report 2026International AI Safety ReportInternational AI Safety Report 20263 Feb 2026 — This Report assesses what general-purpose AI systems can do… [AI Security Institute]aisi.gov.ukAI Security InstituteFrontier AI Trends Report by The AI Security Institute (AISI)The UK AI Security Institute (AISI) has conducted evalu…
These objections do not eliminate the race-pressure concern, but they challenge the stronger claim that competition inevitably leads to reckless deployment.
What would count as evidence that warnings are being ignored?
The most persuasive evidence would probably not be a single decision or incident. Instead, it would be a recurring pattern.
Examples might include:
- Frontier systems repeatedly failing safety evaluations without triggering meaningful deployment restrictions.
- Organisations weakening previously announced safety commitments after competitive pressure increases.
- Governments declining to act on identified risks because rivals are perceived to be moving faster.
- Independent evaluators repeatedly highlighting serious concerns that receive little operational response.
- Growing capability gains accompanied by stagnant investment in interpretability, control, monitoring, or incident response. [AI Security & Safety Directory]aisecurityandsafety.orgai race dynamicsanizations under competitive pressure may allocate more resources…Read more… [Governor of California]gov.ca.govJune 17 2025 – The California Report on Frontier AI PolicyGovernor of CaliforniaTHE CALIFORNIA REPORT ON FRONTIER AI POLICY17 Jun 2025 — In cases where competitive pressures may disincentivize pa…
For AI doom advocates, the deepest concern is not that humanity will fail to notice warning signs. It is that warning signs may become visible while competitive incentives steadily raise the threshold for taking them seriously. In that scenario, the danger comes less from ignorance than from a collective decision that slowing down feels harder than pressing ahead. [GovAI]governance.aiinformation hazards in races for advanced artificial intelligenceInformation Hazards in Races for Advanced Artificial…We study how the information environment affects races to implement a powerful ne… [Governance AI]governance.aisafety not guaranteed international strategic dynamics of risky technology racesGovAIInternational Strategic Dynamics of Risky Technology RacesWe analyse a strategic model to determine when states deploy technologies…
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: governance.ai
Title: safety not guaranteed international strategic dynamics of risky technology races
Link: https://www.governance.ai/research-paper/safety-not-guaranteed-international-strategic-dynamics-of-risky-technology-racesSource snippet
GovAIInternational Strategic Dynamics of Risky Technology RacesWe analyse a strategic model to determine when states deploy technologies...
-
Source: sciencedirect.com
Link: https://www.sciencedirect.com/science/article/pii/S0016328725000254Source snippet
ScienceDirectStrategic insights from simulation gaming of AI race dynamicsby R Gruetzemacher · 2025 · Cited by 15 — Race dynamics in adva...
-
Source: cdn.governance.ai
Title: AIInternational Races for Risky Technologies
Link: https://cdn.governance.ai/International_Races_for_Risky_Technologies_DRAFT_NOV_2022.pdfSource snippet
Governance AIInternational Races for Risky TechnologiesNovember 24, 2022 — by E Stafford · 2022 · Cited by 3 — We examine how three facto...
Published: November 24, 2022
-
Source: governance.ai
Title: information hazards in races for advanced [artificial]({{ ‘artificial-goals/’ | relative_url }}) intelligence
Link: https://www.governance.ai/research-paper/information-hazards-in-races-for-advanced-artificial-intelligenceSource snippet
Information Hazards in Races for Advanced Artificial...We study how the information environment affects races to implement a powerful ne...
-
Source: arxiv.org
Title: arXiv Intolerable Risk Threshold Recommendations for Artificial Intelligence
Link: https://arxiv.org/abs/2503.05812 -
Source: arxiv.org
Title: arXiv Emerging Practices in Frontier AI Safety Frameworks
Link: https://arxiv.org/abs/2503.04746 -
Source: aigi.ox.ac.uk
Title: uncertainty information and risk in international technology races
Link: https://aigi.ox.ac.uk/publications/uncertainty-information-and-risk-in-international-technology-races/Source snippet
Oxford Martin AIGIUncertainty, Information, and Risk in International Technology...17 Nov 2023 — A formal model reveals how the informat...
-
Source: arxiv.org
Title: arXiv AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations
Link: https://arxiv.org/abs/2503.17388Source snippet
arXivAI Companies Should Report Pre- and Post-Mitigation Safety EvaluationsMarch 17, 2025...
Published: March 17, 2025
-
Source: axios.com
Title: Safety guardrails loosen as AI rivalries grows
Link: https://www.axios.com/2026/03/03/ai-race-safety-guardrailSource snippet
Even traditionally cautious firms, like [Anthropic]({{ 'anthropic-tests/' | relative_url }}), have recently revised their internal guidelines, narrowing criteria for delaying risky...
-
Source: sipri.org
Title: Addressing the risks that civilian AI poses to international
Link: https://www.sipri.org/sites/default/files/2025-11/1125_civilian_ai.pdfSource snippet
SIPRIAddressing the risks that civilian AI poses to international...November 21, 2025 — 3 Nov 2025 — As AI safety researchers from OpenA...
Published: November 21, 2025
-
Source: founderspledge.com
Title: how philanthropists can help slow the race to dangerous ai
Link: https://www.founderspledge.com/research/how-philanthropists-can-help-slow-the-race-to-dangerous-aiSource snippet
16 Oct 2024 — The emergence of an AI racing dynamic between great power nations like the US and China could cause both countries to cut c...
-
Source: arxiv.org
Title: arXiv Enabling Frontier Lab Collaboration to Mitigate AI Safety Risks
Link: https://arxiv.org/abs/2511.08631 -
Source: arxiv.org
Link: https://arxiv.org/html/2603.10015v2Source snippet
The Coordination Gap in Frontier AI Safety Policies20 May 2026 — Frontier AI Safety Policies concentrate on prevention: capability evalua...
Published: May 2026
-
Source: arxiv.org
Link: https://arxiv.org/html/2505.00174v1Source snippet
Real-World Gaps in AI Governance Research AI safety and...30 Apr 2025 — Significant research gaps exist in high-risk deployment domains...
-
Source: journals.sagepub.com
Link: https://journals.sagepub.com/doi/10.1177/00220027231214996Source snippet
Sage JournalsUncertainty, Information, and Risk in International...by N Emery-Xu · 2024 · Cited by 19 — A formal model reveals how the i...
-
Source: internationalaisafetyreport.org
Title: international ai safety report 2026
Link: https://internationalaisafetyreport.org/publication/international-ai-safety-report-2026Source snippet
International AI Safety ReportInternational AI Safety Report 20263 Feb 2026 — This Report assesses what general-purpose AI systems can do...
-
Source: aisi.gov.uk
Link: https://www.aisi.gov.uk/frontier-ai-trends-reportSource snippet
AI Security InstituteFrontier AI Trends Report by The AI Security Institute (AISI)The UK AI Security Institute (AISI) has conducted evalu...
-
Source: gov.ca.gov
Title: June 17 2025 – The California Report on Frontier AI Policy
Link: https://www.gov.ca.gov/wp-content/uploads/2025/06/June-17-2025-%E2%80%93-The-California-Report-on-Frontier-AI-Policy.pdfSource snippet
Governor of CaliforniaTHE CALIFORNIA REPORT ON FRONTIER AI POLICY17 Jun 2025 — In cases where competitive pressures may disincentivize pa...
-
Source: aisecurityandsafety.org
Title: ai race dynamics
Link: https://aisecurityandsafety.org/fr/glossary/ai-race-dynamics/Source snippet
anizations under competitive pressure may allocate more resources...Read more...
Additional References
-
Source: atlasinstitute.org
Link: https://atlasinstitute.org/ai-driven-political-risk-challenges-for-britains-national-security/Source snippet
AI-Driven Political Risk: Challenges for Britain's National...7 days ago — This article critically analyzes how AI-driven geopolitical r...
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/coenvanmourik_aipolicy-aisafety-techethics-activity-7456238391483863040-yaycSource snippet
Coen Van Mourik's PostUnderstanding the Global Arms Race in AI Technology · AI Safety Governance Framework · Global AI Safety Collaborati...
-
Source: globaia.org
Link: https://globaia.org/ai-risks/Source snippet
Faster Than [Oversight]({{ 'oversight-evasion/' | relative_url }}) — AI Risks ObservatoryAI risks are growing faster than our ability to govern them. 30+ interactive visualizations t...
-
Source: medium.com
Link: https://medium.com/%40ZombieCodeKill/the-plan-to-pause-ai-before-its-too-late-68c6208c94b4Source snippet
The Plan to Pause AI Before It's Too LateKevin: People decided to ignore the warning signs because that pointed against their business in...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=-idQtT8WIr8Source snippet
Why the AI Race Undermines Safety (with Steven Adler)Stephen Adler is a former safety researcher at OpenAI. He joins the podcast to discu...
-
Source: aigi.ox.ac.uk
Title: Uncertainty, Information, and Risk in International Technology Races
Link: https://aigi.ox.ac.uk/press-reports/uncertainty-information-and-risk-in-international-technology-races/Source snippet
ox.ac.ukUncertainty, Information, and Risk in International Technology...International AI Governance · Social Impact of Emerging Technol...
-
Source: facebook.com
Link: https://www.facebook.com/PCMag/posts/as-the-guardian-reports-there-was-a-fivefold-rise-in-ai-misbehavior-between-octo/1312975630702233/Source snippet
n October and March 2025, including AI models deleting emails and...Read more...
Published: March 2025
-
Source: aisafetybook.com
Title: This could result in increased risks of large-scale wars.Read more
Link: https://www.aisafetybook.com/textbook/ai-raceSource snippet
1.3: AI Race | AI Safety, Ethics, and Society TextbookCompetitive pressures may lead militaries and corporations to hand over excessive p...
-
Source: businessinsider.com
Title: anthropic changing safety policy 2026 2
Link: https://www.businessinsider.com/anthropic-changing-safety-policy-2026-2Source snippet
The company will no longer unilaterally pause or delay new AI model deployments when safety mechanisms lag, citing increased competition...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=2Pel_1BPm_kSource snippet
AI: Existential Risk or Power Grab? - YouTube AI: Existential Risk or Power Grab? - YouTube...
Topic Tree