Within P Doom
Do Expert Surveys Really Measure AI Extinction Risk?
Researcher surveys show that existential concern is widespread, but they measure beliefs rather than the true probability of catastrophe.
On this page
- What large researcher surveys establish
- Selection, expertise and response biases
- Why survey concern is not a scientific consensus probability
Page outline Jump by section
Introduction
Expert surveys are one of the most frequently cited pieces of evidence in debates about AI doom and p(doom), the estimated probability that advanced AI could cause human extinction or an equally irreversible loss of humanity’s future. They are valuable because they reveal what qualified researchers currently believe about highly uncertain questions. They are not valuable because they somehow measure the true probability of catastrophe.
That distinction matters. Surveys can establish that existential concern is widespread among machine-learning researchers, that opinions vary enormously even among experts, and that many researchers assign a non-trivial chance to extremely bad outcomes. They cannot demonstrate that those probabilities are objectively correct, nor do they create a scientific consensus comparable to measuring the speed of light or estimating the efficacy of a medicine through repeated experiments.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…
What large researcher surveys actually establish
The strongest recent evidence comes from the large Expert Survey on Progress in AI conducted by researchers at AI Impacts and collaborating institutions. The 2023 survey collected responses from 2,778 researchers who had published at leading AI conferences, making it the largest published survey of expert opinion in this area.[arXiv]arxiv.orgarXiv Thousands of AI Authors on the Future of AIarXiv Thousands of AI Authors on the Future of AI
Across different versions of the existential-risk question, several findings consistently appear.
- A substantial minority of AI researchers assign meaningful probabilities to existential catastrophe.
- The median response is generally around 5% for human extinction or similarly permanent human disempowerment.
- Around 38% to 51% of respondents assign at least a 10% probability depending on the exact wording.
- Most researchers nevertheless expect beneficial outcomes from advanced AI to be more likely overall than catastrophic ones. Even among optimistic respondents, many still assign a non-negligible chance of disaster.[arXiv]arxiv.orgarXiv Thousands of AI Authors on the Future of AIarXiv Thousands of AI Authors on the Future of AI
This combination is often misunderstood. Someone can believe there is a 90% chance advanced AI benefits humanity while simultaneously believing there is a 10% chance of existential catastrophe. That does not make them an “AI doomer”; it reflects the unusually high stakes involved.
The surveys therefore demonstrate something important but limited: existential concern is not confined to a handful of campaigners or philosophers. It exists within the research community itself, although to very different degrees.
Why these surveys are informative
Expert surveys provide useful evidence for several reasons.
First, they aggregate judgements from people who spend their careers studying AI systems rather than from the general public.
Second, they reveal the distribution of opinion rather than highlighting only famous individuals. Public discussion often centres on a few well-known figures, whereas surveys show thousands of researchers ranging from highly sceptical to deeply concerned.
Third, repeated surveys allow researchers to observe broad changes in opinion over time. Recent surveys suggest expectations about AI progress have accelerated, while substantial concern about catastrophic outcomes has remained present across multiple survey rounds.[arXiv]arxiv.orgarXiv Thousands of AI Authors on the Future of AIarXiv Thousands of AI Authors on the Future of AI
Importantly, these surveys also expose disagreement instead of hiding it. Wide variation in answers signals that the underlying uncertainty remains extremely large.
Selection, expertise and response biases
The main limitation is that no expert survey perfectly represents “what AI experts think.”
Who counts as an expert?
Different surveys use different sampling methods.
Some recruit authors from prestigious machine-learning conferences. Others survey members of professional societies or selected research communities. These groups overlap but are not identical.
Conference authors are generally appropriate respondents for questions about AI capabilities, but they are not necessarily specialists in AI alignment, forecasting, economics or geopolitical risk. A machine-learning researcher may understand current models extremely well while having relatively little expertise in forecasting long-term civilisation-scale outcomes.
Conversely, some researchers who specialise in AI safety or existential risk publish less frequently in mainstream machine-learning venues, meaning they may be underrepresented.
Non-response bias
Another concern is that people with stronger views may be more likely to complete voluntary surveys.
The organisers of the 2023 survey investigated this possibility in detail. They found little evidence that respondents dropped out when existential-risk questions appeared, and demographic analyses suggested that measured response biases were unlikely to explain the overall results. They also identified some detectable biases—for example, differences in participation by gender or educational background—but found no evidence that correcting these would eliminate substantial concern about catastrophic AI risk.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…
That strengthens confidence that the survey reflects genuine diversity of opinion rather than merely attracting the most worried researchers. It does not eliminate all sampling uncertainty.
Expertise does not guarantee calibration
Even genuine experts can make poor probability forecasts about unprecedented events.
Forecasting transformative AI differs from forecasting familiar engineering projects because there are no previous cases of artificial superintelligence, no historical frequencies to analyse and no controlled experiments that reveal the true probability of extinction.
Experts therefore rely heavily on judgement, analogy and theoretical reasoning rather than empirical measurement.
Why wording changes the answers
Perhaps the most striking lesson from these surveys is how sensitive responses are to question wording.
Researchers give different answers depending on whether they are asked about:
- human extinction alone;
- extinction or permanent human disempowerment;
- “extremely bad outcomes”;
- failures specifically caused by loss of control;
- events within the next century rather than eventually.
The same underlying researcher may reasonably answer each question differently because each refers to a different event.
The organisers themselves highlight this as one of the principal limitations of interpreting survey results. In particular, “human extinction” and “permanent and severe disempowerment” are related but not identical scenarios, and respondents may imagine very different futures when answering.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…
This is one reason why headlines such as “experts think there is a 5% chance AI will wipe out humanity” oversimplify the underlying evidence.
Why survey concern is not a scientific consensus probability
Readers often assume that if thousands of experts report a median estimate of 5%, then science has established a 5% probability of AI extinction.
That inference is not justified.
Unlike probabilities derived from repeated observations—such as weather forecasting models calibrated against decades of measurements—AI existential-risk estimates are subjective probability judgements. They combine beliefs about future technical progress, incentives, governance, human behaviour and unknown scientific discoveries.
Experts can agree that current evidence is worrying while still disagreeing profoundly about:
- whether artificial general intelligence will be developed this century;
- whether highly capable systems will become autonomous;
- whether alignment techniques will scale;
- whether governments will successfully regulate deployment;
- whether catastrophic failures would become irreversible.
Because these assumptions differ, two equally informed researchers can honestly arrive at estimates that differ by an order of magnitude.
The surveys therefore measure the distribution of informed judgement rather than discovering an objective physical quantity.
What surveys cannot prove
Expert surveys are sometimes presented as stronger evidence than they really are.
They cannot prove:
- that AI doom is likely;
- that AI doom is unlikely;
- that the median estimate represents the “correct” probability;
- that experts have reached a settled scientific consensus;
- that disagreement will narrow as AI research progresses.
Nor can surveys identify which underlying argument for AI doom is correct. A respondent assigning a 20% existential risk might primarily fear deceptive alignment, while another with the same numerical estimate might worry about geopolitical racing, autonomous cyber capabilities or deliberate misuse. The identical probability conceals different causal stories.
Likewise, surveys do not validate specific technical theories about loss of control. They simply reveal that many researchers consider such scenarios plausible enough to assign them non-zero probabilities.
How these surveys should influence p(doom)
For readers trying to interpret p(doom), expert surveys are best viewed as one piece of evidence rather than the final answer.
They are stronger evidence than social-media polls, opinion pieces or isolated quotations because they systematically measure a large population of researchers. They are weaker evidence than direct empirical tests because no experiment can yet measure the probability of AI-caused existential catastrophe.
A balanced interpretation is therefore:
- the surveys provide credible evidence that existential concern is widespread within the AI research community;
- they show that informed opinion spans an exceptionally wide range rather than converging on a single estimate;
- they indicate that assigning a non-trivial probability to AI doom is not an eccentric or fringe position;
- they do not establish that any particular p(doom) value represents the true probability of humanity’s future.
For estimating AI extinction risk, the most defensible approach is to treat expert surveys as evidence about expert beliefs—not as measurements of reality itself. Those beliefs deserve attention because they come from people closest to the technology, but they remain informed judgements about an uncertain future rather than experimentally verified probabilities.[arxiv.org]arxiv.orgarXiv Thousands of AI Authors on the Future of AIarXiv Thousands of AI Authors on the Future of AI
Amazon book picks
Further Reading
Books and field guides related to Do Expert Surveys Really Measure AI Extinction Risk?. Use these as the next step if you want deeper reading beyond the article.
Human Compatible
A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...
The Alignment Problem
Finalist for the Los Angeles Times Book Prize A jaw-dropping exploration of everything that goes wrong when we build AI systems and the m...
Superforecasting
The international bestseller 'A manual for thinking clearly in an uncertain world. Read it.' Daniel Kahneman, author of Thinking, Fast an...
The Precipice
What existential threats does humanity face? And how can we secure our future?'The Precipice is a powerful book . . . Ord's love for huma...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Selected fromartificial intelligence poster oneBay.co.uk.
Current eBay listing
A. I. Artificial Intelligence. Jude Law. Original UK Video Poster.
Current eBay listing
A.I. Artificial Intelligence - Jude Law - One Sheet Cinema Poster
Endnotes
1.
Source: blog.aiimpacts.org
Title: AI Impacts Blog FAQ: Expert Survey on Progress in AI methodology
Link:https://blog.aiimpacts.org/p/faq-expert-survey-on-progress-in
Source snippet
AI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025...
Published: October 31, 2025
2.
Source: arxiv.org
Title: arXiv Thousands of AI Authors on the Future of AI
Link:https://arxiv.org/abs/2401.02843
3.
Source: wiki.aiimpacts.org
Title: 2023 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai
Source snippet
AI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]...
4.
Source: aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/
Source snippet
AI Impacts2022 Expert Survey on Progress in AI – AI Impacts...
5.
Source: blog.aiimpacts.org
Title: 2023 [ai survey]({{ ‘survey-bias/’ | relative_url }}) of 2778 six things
Link:https://blog.aiimpacts.org/p/2023-ai-survey-of-2778-six-things?action=share
6.
Source: wiki.aiimpacts.org
Title: ai risk surveys
Link:https://wiki.aiimpacts.org/uncategorized/ai_risk_surveys
7.
Source: aiimpacts.org
Title: How bad a future do ML researchers expect? – AI Impacts
Link:https://aiimpacts.org/how-bad-a-future-do-ml-researchers-expect/
8.
Source: wiki.aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2022_expert_survey_on_progress_in_ai
Additional References
9.
Source: lesswrong.com
Title: FA Q: Expert Survey on Progress in AI methodology — Less Wrong
Link:https://www.lesswrong.com/posts/BEg5dct7nr2Yq3dJY/faq-expert-survey-on-progress-in-ai-methodology
Source snippet
No. Respondents answered nearly all the normal questions they saw (excluding demographics, free response, and conditionally-asked questio...
10.
Source: doi.org
Link:https://doi.org/10.1007/s00146-024-02134-4
Source snippet
Depending on the phrasing of the question, between 38% and 51% gave at least 10% probabil...
11.
Source: spectrum.ieee.org
Title: ai existential risk survey 2667013207
Link:https://spectrum.ieee.org/amp/ai-existential-risk-survey-2667013207
Source snippet
the Prophecies of AI Doom - IEEE SpectrumJanuary 25, 2024 — WEIGHING THE PROPHECIES OF AI DOOM Researchers surveyed point to AI’s existen...
Published: January 25, 2024
12.
Source: aiwiki.ai
Title: Existential risk from AI | AI Wiki
Link:https://aiwiki.ai/wiki/ai_existential_risk
Source snippet
SURVEY DATA The 2023 Expert Survey on Progress in AI (ESPAI), conducted by AI Impacts, collected responses from 2,778 AI researchers who...
13.
Source: lesswrong.com
Title: I’ve found this
Link:https://www.lesswrong.com/posts/RkegCmCgjGhskiFvm/ai-impacts-2023-expert-survey-on-progress-in-ai
Source snippet
AI Impacts 2023 Expert Survey on Progress in AI — LessWrongJanuary 5, 2024 — AI IMPACTS 2023 EXPERT SURVEY ON PROGRESS IN AI by habryka 5...
Published: January 5, 2024
14.
Source: youtube.com
Title: Katja Grace on the Largest Survey of AI Researchers
Link:https://www.youtube.com/watch?v=nCkS-bqMdX0
Source snippet
Why A.I. Might Not Take Your Job or Supercharge the Economy...
15.
Source: youtube.com
Title: AI Impacts Survey
Link:https://www.youtube.com/watch?v=xwJx_xqZI3Q
Source snippet
Katja Grace—Slowing Down AI, Forecasting AI Risk...
16.
Source: youtube.com
Title: Katja Grace—Slowing Down AI, Forecasting AI Risk
Link:https://www.youtube.com/watch?v=rSw3UVDZge0
Source snippet
Katja Grace on the Largest Survey of AI Researchers...
17.
Source: youtube.com
Title: Why A.I. Might Not Take Your Job or Supercharge the Economy
Link:https://www.youtube.com/watch?v=-Ut3h5st6VM
Source snippet
The A.I. Dilemma - March 9, 2023...
Published: March 9, 2023
18.
Source: youtube.com
Title: The A.I. Dilemma
Link:https://www.youtube.com/watch?v=xoVJKj8lcNQ
Published: March 9, 2023