Within Alignment & Governance
Why Experts With Similar p(doom) Numbers Still Disagree
Expert surveys show substantial concern about catastrophic AI risk but deep disagreement about its likely causes.
On this page
- What major AI risk surveys have found
- How survey framing changes reported risk levels
- Why similar forecasts can hide different assumptions
Page outline Jump by section
Introduction
When people cite expert surveys to support a particular p(doom) estimate, they often leave out the most important finding: AI researchers show substantial concern about catastrophic AI risk, but they disagree deeply about why such a catastrophe might happen and how likely different pathways are.
Across several large surveys of AI researchers, the median estimate for AI causing human extinction or a similarly severe and permanent loss of human control has often landed around 5%. At the same time, large minorities of respondents give probabilities above 10%, while others assign risks close to zero. The headline number therefore hides a remarkably wide distribution of views. [AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or… [AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
The larger 2023 survey of 2,778 AI researchers found a similar result. Researchers again gave a median estimate of roughly 5% for human extinction from advanced AI, and large fractions of respondents assigned probabilities above 10%. Depending on the exact question wording, roughly 38% to 51% of respondents placed at least a 10% chance on outcomes as bad as human extinction. [Business Insider]businessinsider.comApproximately 58% of surveyed researchers believe there is at least a 5% chance of AI causing catastrophic outcomes. The survey highlight… [3AI Impacts 3AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
At the same time, surveys of AI researchers conducted for other purposes sometimes produce a different picture. Some studies asking researchers about their most immediate concerns find that issues such as misinformation, labour-market disruption, concentration of power, and bias rank above extinction scenarios. In one UK-based survey, only a small minority identified existential risk as their single greatest concern. [Responsible AI]rai.ac.ukounding existential risk–which did not land amongst the top ten concerns that emerged in responses.Read more…
This is not necessarily a contradiction. A researcher can believe there is a 5% chance of extinction while still spending most of their attention on more immediate and probable harms.
How survey framing changes reported risk levels
One of the strongest lessons from the evidence is that survey wording matters.
Researchers have been asked slightly different questions, including:
- The probability that AI causes human extinction. [aiimpacts.org]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or… * The probability that AI causes extinction or severe permanent disempowerment. [aiimpacts.org]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
- The probability that humanity fails to control advanced AI systems.
- The probability that advanced AI leads to outcomes as bad as extinction.
These are related but not identical concepts. A respondent may think literal extinction is unlikely while believing that humanity could lose meaningful control over its future. Others may view misuse by governments or malicious actors as more plausible than a takeover by autonomous AI systems. [AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
Survey timing also matters. Researchers answering before the release of powerful frontier systems may have had different intuitions from those responding after seeing rapid advances in large language models and multimodal systems. Several surveys have found that expectations about AI timelines shifted significantly after recent breakthroughs, and shorter timelines can indirectly increase perceived catastrophe risks because there appears to be less time to solve safety problems. [AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or… [AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
Another complication is the difference between median responses and distributions. A median p(doom) of 5% can sound modest, but it may conceal a population split between people assigning near-zero risk and people assigning very high probabilities. Looking only at the median therefore understates how much disagreement exists inside the field. AI Impacts [2blog.aiimpacts.org]blog.aiimpacts.org2023 ai survey of 2778 six thingsSurvey of 2778 AI authors: six parts in picturesJan 4, 2024 — Median respondents put 5% or more on advanced AI leading to human extinctio…
Why similar forecasts can hide different assumptions
The most important limitation of p(doom) surveys is that they usually measure conclusions rather than reasoning.
Recent research examining disagreement among AI experts suggests that experts often cluster around fundamentally different mental models of advanced AI. One broad camp tends to view future systems as powerful but controllable tools. Another is more sympathetic to the idea that advanced AI could become an autonomous strategic actor capable of escaping human control. These worldviews can produce dramatically different risk estimates even when participants share similar technical backgrounds. [arXiv]arxiv.orgarXivWhy do Experts Disagree on Existential Risk and P(doom)? A Survey of AI ExpertsJanuary 25, 2025…
Three researchers could each report a 5% p(doom) while holding very different beliefs:
The alignment-focused forecaster worries that highly capable systems will pursue objectives that diverge from human intentions and eventually resist correction.
The governance-focused forecaster believes technical alignment may be possible but expects competitive pressures between companies or states to undermine safety precautions.
The misuse-focused forecaster thinks the greatest danger comes from humans using powerful AI systems for destructive purposes, including large-scale biological, cyber, or military attacks.
The headline survey result does not distinguish between these pathways. Yet the policy implications differ dramatically depending on which explanation is driving the estimate.
What surveys reveal about alignment and governance assumptions
Because this page sits within the broader question of how alignment and governance assumptions shape p(doom), the key finding is not simply that experts are worried. It is that survey responses often reflect hidden assumptions about whether future problems are technically solvable and socially manageable.
Researchers who expect major breakthroughs in interpretability, monitoring, evaluations, and control methods generally tend toward lower catastrophe estimates. They may still acknowledge serious risks but expect engineering solutions to reduce them over time.
Researchers with higher p(doom) estimates often believe that capability gains could outpace understanding, creating systems that are difficult to monitor or control before they become strategically powerful. Others place greater weight on governance failures, arguing that even if technical solutions exist, commercial and geopolitical competition may prevent their consistent adoption. [arXiv]arxiv.orgarXivWhy do Experts Disagree on Existential Risk and P(doom)? A Survey of AI ExpertsJanuary 25, 2025… [RAND]rand.orgRRA3034 1RAND CorporationOn the Extinction Risk from Artificial Intelligence6 May 2025 — In this report, RAND researchers examined and seriously c…
Survey numbers alone cannot tell readers which of these assumptions is correct. They can only reveal that these assumptions are major drivers of disagreement.
What expert surveys do not tell us
Expert surveys are useful evidence, but they are not measurements in the way that climate observations or engineering tests are measurements.
Several limitations deserve attention:
- AI researchers are forecasting a future that has not yet happened.
- Researchers often disagree about what counts as artificial general intelligence or transformative AI.
- The field has a mixed record on long-range forecasting.
- Survey participants may be influenced by different definitions of catastrophe.
- Response rates and sampling methods can affect results. [Vox]vox.comThousands of AI experts are torn about what they'veVoxJanuary 10, 2024 — 10 Jan 2024 — The 2016 AI impacts survey was immediately controversial. In 2016, barely anyone was talking about th… [Nature For this reason]nature.comAI & robotics briefing: There's a 5% risk that AI will wipe out…Jan 16, 2024 — In a survey of 2700 AI experts, a majority said there w…, a survey showing a median 5% extinction risk should not be interpreted as proving there is exactly a 5% chance of doom. Equally, it should not be dismissed as meaningless. The important information is that many people closest to AI research consider existential catastrophe plausible enough to assign non-trivial probabilities to it.
The main takeaway
Expert surveys provide evidence that concern about AI doom is widespread within the AI research community, but they do not reveal a consensus theory of how doom would occur.
The most robust finding is not the exact median p(doom) figure. It is the coexistence of two facts: many AI researchers assign meaningful probabilities to existential catastrophe, and those researchers often disagree profoundly about the underlying mechanism. Some fear misalignment, some fear loss of control, some fear governance failure, and some fear catastrophic misuse. Similar p(doom) numbers can therefore mask very different assumptions about the future of advanced AI. [arXiv]arxiv.orgarXivWhy do Experts Disagree on Existential Risk and P(doom)? A Survey of AI ExpertsJanuary 25, 2025… [3AI Impacts 3AI Impacts]aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or…
Amazon book picks
Further Reading
Books and field guides related to Why Experts With Similar p(doom) Numbers Still Disagree. Use these as the next step if you want deeper reading beyond the article.
The Alignment Problem
Shows how similar risk estimates can arise from different assumptions.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: aiimpacts.org
Link: https://aiimpacts.org/wp-content/uploads/2024/01/EMBARGOED_-AI-Impacts-Survey-Release-Google-Docs.pdfSource snippet
AI ImpactsMedian AI expert says 5% chance of human extinction from...4 Jan 2024 — BERKELEY, CALIFORNIA: In a new survey of 2,778 AI expe...
-
Source: aiimpacts.org
Title: 2022 expert survey on progress in ai
Link: https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/Source snippet
AI Impacts2022 Expert Survey on Progress in AIAug 3, 2022 — What probability do you put on future AI advances causing human extinction or...
-
Source: aiimpacts.org
Title: Thousands of AI authors on the future of AI
Link: https://aiimpacts.org/wp-content/uploads/2023/04/Thousands_of_AI_authors_on_the_future_of_AI.pdfSource snippet
More than half suggested that “...Read more...
-
Source: rai.ac.uk
Link: https://rai.ac.uk/hed-are-ai-researchers-concerned-about-the-existential-threat-of-ai/Source snippet
ounding existential risk–which did not land amongst the top ten concerns that emerged in responses.Read more...
-
Source: aiimpacts.org
Link: https://aiimpacts.org/2024/Source snippet
2024 – AI ImpactsThe Expert Survey on Progress in AI (ESPAI) is a large survey of AI researchers about the future of AI, conducted in 201...
-
Source: time.com
Title: When Might AI Outsmart Us?
Link: https://time.com/6556168/when-ai-outsmart-humans/Source snippet
It Depends Who You AskExperts have varied opinions on when artificial general intelligence (AGI) will be developed, with some predicting...
-
Source: blog.aiimpacts.org
Title: 2023 ai survey of 2778 six things
Link: https://blog.aiimpacts.org/p/2023-ai-survey-of-2778-six-thingsSource snippet
Survey of 2778 AI authors: six parts in picturesJan 4, 2024 — Median respondents put 5% or more on advanced AI leading to human extinctio...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2502.14870Source snippet
arXivWhy do Experts Disagree on Existential Risk and P(doom)? A Survey of AI ExpertsJanuary 25, 2025...
Published: January 25, 2025
-
Source: arxiv.org
Link: https://arxiv.org/html/2502.14870v1Source snippet
arXivWhy do Experts Disagree on Existential Risk and P(doom)...23 Feb 2025 — According to AI Impacts, roughly 40% of respondents indicat...
-
Source: rand.org
Title: RRA3034 1
Link: https://www.rand.org/pubs/research_reports/RRA3034-1.htmlSource snippet
RAND CorporationOn the Extinction Risk from Artificial Intelligence6 May 2025 — In this report, RAND researchers examined and seriously c...
Published: May 2025
-
Source: vox.com
Title: Thousands of AI experts are torn about what they’ve
Link: https://www.vox.com/future-perfect/2024/1/10/24032987/ai-impacts-survey-artificial-intelligence-chatgpt-openai-existential-risk-superintelligenceSource snippet
VoxJanuary 10, 2024 — 10 Jan 2024 — The 2016 AI impacts survey was immediately controversial. In 2016, barely anyone was talking about th...
Published: January 10, 2024
-
Source: nature.com
Link: https://www.nature.com/articles/d41586-024-00147-zSource snippet
AI & robotics briefing: There's a 5% risk that AI will wipe out...Jan 16, 2024 — In a survey of 2700 AI experts, a majority said there w...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2401.02843Source snippet
More than half...Read more...
-
Source: businessinsider.com
Link: https://www.businessinsider.com/ai-researchers-chance-tech-making-humans-extinct-2024-1Source snippet
Approximately 58% of surveyed researchers believe there is at least a 5% chance of AI causing catastrophic outcomes. The survey highlight...
-
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/P%28doom%29Source snippet
P(doom)In AI safety, P(doom) is the probability of existentially catastrophic outcomes (so-called "doomsday scenarios") as a result of...
Additional References
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/simontorrance_why-do-experts-disagree-on-existential-risk-activity-7301886948334333952-VwQSSource snippet
AI experts divided on AI safety and controlA new survey reveals that 'AI experts' cluster into two viewpoints - "AI as controllable tool"...
-
Source: wsj.com
Link: https://www.wsj.com/articles/a-godfather-of-ai-remains-concerned-as-ever-about-human-extinction-ec0fe932Source snippet
Despite his earlier calls for a moratorium on AI development to prioritize safety, the industry has surged ahead, investing massively in...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/2024/dec/27/godfather-of-ai-raises-odds-of-the-technology-wiping-out-humanity-over-next-30-yearsSource snippet
'Godfather of AI' shortens odds of the technology wiping out...28 Dec 2024 — Geoffrey Hinton says there is 10% to 20% chance AI will lea...
-
Source: machine.news
Title: welcome to the x risk games which ai models pose the greatest threat to humanity
Link: https://www.machine.news/welcome-to-the-x-risk-games-which-ai-models-pose-the-greatest-threat-to-humanity/Source snippet
Which AI model poses the worst existential apocalypse risk?17 Feb 2026 — Chinese academics have identified the AI model that currently po...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=BCA7ZTafHc8Source snippet
most AI researchers are REALLY worriedThis Yudkowskian Has A 99.999% P(Doom). Doom Debates•19K · 1:09:03. Go... AI Expert Exposes...
-
Source: thebulletin.org
Title: three key misconceptions in the debate about ai and existential risk
Link: https://thebulletin.org/2024/07/three-key-misconceptions-in-the-debate-about-ai-and-existential-risk/Source snippet
Three key misconceptions in the debate about AI and...15 Jul 2024 — 59 percent of US adults support prioritizing mitigating the risk of...
-
Source: brookings.edu
Title: are ai existential risks real and what should we do about them
Link: https://www.brookings.edu/articles/are-ai-existential-risks-real-and-what-should-we-do-about-them/Source snippet
Are AI existential risks real—and what should we do about...Jul 11, 2025 — The challenges of existential risk from highly capable AI sys...
-
Source: nickbostrom.com
Link: https://nickbostrom.com/papers/existential-risks/Source snippet
al hazards, this paper analyzes a recently emerging category: that of existential...Read more...
-
Source: shop.freiheit.org
Title: FNF A4 Policy Paper Existential Risk from AI EN web final
Link: https://shop.freiheit.org/download/P2%401939/989395/FNF_A4_Policy%20Paper_Existential%20Risk%20from%20AI_EN_web_final.pdfSource snippet
RISK FROM AIby C Nauer · 2025 — A 2024 survey of 2,778 AI researchers estimated a 10% chance of AGI by 2027 and a 50% chance by 2047, ref...
-
Source: klover.ai
Title: pdoom ai risk fridmans perspective on existential threat
Link: https://www.klover.ai/pdoom-ai-risk-fridmans-perspective-on-existential-threat/Source snippet
P(doom) & AI Risk: Fridman's Perspective on Existential...23 Jun 2025 — Fridman openly estimated the probability of AI creating an exist...
Topic Tree