Within Survey Estimates
How Survey Design Shapes AI Doom Predictions
How the wording, timeframes, and scenario assumptions in surveys influence experts' p(doom) estimates.
On this page
- Variations in 'doom' definitions
- Time horizon effects on risk estimates
- Scenario assumptions and implicit bias
Page outline Jump by section
Introduction
When people see a headline claiming that AI researchers assign a 5%, 10%, or 20% chance to human extinction from advanced AI, it is easy to assume those numbers represent a stable expert consensus. In practice, p(doom) survey results are highly sensitive to how questions are asked. Small changes in wording, time horizon, definitions of catastrophe, or assumptions about AI control can shift responses substantially. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
This does not mean the surveys are meaningless. Rather, it means they are measuring a difficult and uncertain belief space. Researchers are asking experts to estimate unprecedented events involving future technologies, unclear timelines, and contested theories of AI alignment. The resulting numbers depend not only on what respondents believe, but also on how they interpret the survey itself. Understanding these framing effects is essential for interpreting expert forecasts about AI doom and existential risk.
Variations in What Counts as “Doom”
One of the most important design choices in AI risk surveys is defining the outcome being measured.
Some surveys ask directly about human extinction, while others ask about broader outcomes such as “human extinction or similarly permanent and severe disempowerment of the human species”. That second formulation captures scenarios where humans survive biologically but permanently lose control over civilisation, political authority, economic power, or humanity’s long-term future. [AI Impacts]lesswrong.comai impacts 2023 expert survey on progress in aiLessWrongAI Impacts 2023 Expert Survey on Progress in AI5 Jan 2024 — We asked about the likelihood that AI will cause human extinction us…
This distinction matters because many AI safety researchers do not view extinction as the only existentially catastrophic outcome. A future in which highly capable AI systems permanently dominate human decision-making may be considered an existential loss even if humans remain alive. When surveys broaden the definition in this way, respondents may include a wider range of failure scenarios in their estimates. [AI Impacts]lesswrong.comai impacts 2023 expert survey on progress in aiLessWrongAI Impacts 2023 Expert Survey on Progress in AI5 Jan 2024 — We asked about the likelihood that AI will cause human extinction us…
The wording also affects how respondents mentally construct examples:
- “Human extinction” often evokes a narrow image of everyone dying.
- “Permanent severe disempowerment” invites consideration of slower or more complex loss-of-control scenarios.
- “Existential catastrophe” may be interpreted differently by different respondents unless explicitly defined.
This creates a recurring challenge in p(doom) discussions. Two experts may both answer “10%”, but one may be thinking only about literal extinction while the other includes a broader class of irreversible outcomes.
How a Single Phrase Can Change Risk Estimates
The clearest demonstration of framing effects comes from the large 2023 AI researcher survey conducted by AI Impacts and collaborators.
Researchers deliberately asked multiple versions of an AI catastrophe question. One version asked:
What probability do you put on future AI advances causing human extinction or similarly permanent and severe disempowerment?
A second version asked:
What probability do you put on human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment?
The results shifted noticeably. The first wording produced a median estimate of 5% and a mean estimate of 16.2%. The second wording produced a median estimate of 10% and a mean estimate of 19.4%. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
The underlying event is similar, but the second question explicitly highlights loss of control. That wording directs respondents toward one of the central concerns in AI doom arguments: the possibility that advanced systems become difficult or impossible to supervise effectively.
This illustrates a broader survey principle. Questions do not simply record beliefs; they also activate particular mental models. Mentioning “inability to control” encourages respondents to think about alignment failures, deceptive behaviour, autonomous agents, or runaway optimisation. Leaving those concepts implicit may lead some respondents to focus instead on beneficial or neutral AI futures.
Time Horizons Change the Numbers
Another major source of variation is the timeframe attached to the question.
A respondent asked about catastrophe “within the next 20 years” is answering a different question from one asked about catastrophe “within the next century” or “eventually”.
The AI Impacts survey included versions that specified a 100-year horizon and versions that did not. This matters because many researchers separate two uncertainties:
- Whether extremely powerful AI will ever be developed.
- Whether such systems would be catastrophically dangerous.
Someone who expects transformative AI in the next decade may give a relatively high near-term risk estimate. Someone who expects such systems centuries from now may assign a much lower probability over the next hundred years while still believing eventual risks are serious. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
Time horizons also interact with forecasting psychology. Longer periods naturally accumulate more opportunities for failure. A respondent who thinks there is a small annual chance of catastrophe may report a much larger cumulative probability over a century than over a decade.
As a result, comparing p(doom) figures across surveys without checking the timeframe can be misleading. Two apparently contradictory numbers may simply refer to different forecasting windows.
Scenario Assumptions Hidden Inside the Question
Many AI doom surveys contain implicit assumptions that respondents may interpret differently.
Consider a question asking about risks from “future AI advances”. Some respondents may imagine incremental progress and continued human oversight. Others may imagine artificial general intelligence, recursive self-improvement, or systems vastly exceeding human capabilities.
The survey question itself may never specify which scenario is intended.
This creates what survey researchers sometimes call a hidden-reference problem. Respondents answer the same words while imagining different futures.
Several assumptions can alter responses:
- How powerful future AI becomes
- How quickly capabilities improve
- Whether governments intervene
- Whether alignment techniques succeed
- Whether dangerous systems are deployed widely
- Whether geopolitical competition encourages reckless development
The 2025 survey on expert disagreement over AI existential risk found that experts often diverge because they start from different conceptual models of AI itself. Some view advanced AI primarily as a controllable tool. Others view it as a potentially autonomous agent that could pursue goals independently of human intentions. These underlying models influence risk estimates before any numerical forecasting begins. [arXiv]arxiv.orgarXivWhy do Experts Disagree on Existential Risk and P(doom)…23 Feb 2025 — I surveyed 111 AI experts on their familiarity with AI safe…
A survey question that implicitly points respondents toward one model or the other can therefore shift aggregate results.
The Difference Between “AI Causes Doom” and “Humans Lose Control”
Framing effects become especially visible when comparing broad catastrophe questions with control-focused questions.
A general question about AI causing extinction can include many pathways:
- Misaligned superintelligence
- Autonomous weapons escalation
- Bioweapon design assistance
- Political destabilisation
- Economic collapse
- Other unknown mechanisms
A question specifically about human inability to control advanced AI systems narrows attention to alignment and control failures. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
This distinction matters because different experts find different pathways more plausible.
An AI governance specialist worried about geopolitical races may assign a higher risk when all pathways are considered. A technical alignment researcher may assign a higher risk when the survey specifically highlights control problems.
The survey wording effectively determines which danger models become most salient during estimation.
Median Versus Mean: Another Framing Trap
Survey interpretation is affected not only by question wording but also by how results are presented.
AI doom surveys often contain extremely skewed distributions. Many respondents give very low probabilities while a smaller group gives very high probabilities.
This means the median and mean can tell different stories.
In the 2023 AI researcher survey, median responses around 5–10% coexisted with substantially higher mean values. A reader shown only the median might conclude that concern is modest. A reader shown only the mean might conclude that concern is widespread and severe. Both descriptions capture part of the same dataset. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
Presentation choices therefore become a secondary framing effect. Media reports often emphasise whichever statistic produces a more striking headline.
Why Survey Designers Use Multiple Question Variants
Because framing effects are so powerful, some researchers deliberately ask several versions of the same underlying question.
The goal is not necessarily to find a single “true” p(doom) number. Instead, it is to map the range of reasonable interpretations.
Multiple-question designs help reveal:
- Whether respondents are sensitive to control-focused language.
- Whether extinction-only definitions differ from broader existential-risk definitions. [normaltech.ai]normaltech.aiai existential risk probabilitiesare too unreliable to inform policy26 Jul 2024 — On the one hand, existential risks (x-risks) are necessarily somewhat speculative: by th…
- Whether time horizons drive large changes in estimates.
- Whether respondents interpret key concepts consistently.
The AI Impacts surveys are notable partly because they expose these framing effects rather than hiding them. By publishing multiple variants, they show how much uncertainty exists in the measurement process itself. [LessWrong]lesswrong.comai impacts 2023 expert survey on progress in aiLessWrongAI Impacts 2023 Expert Survey on Progress in AI5 Jan 2024 — We asked about the likelihood that AI will cause human extinction us…
For readers, this is often more informative than a single headline number. [arxiv.org]arxiv.orgMore than half…Read more…
Does Framing Mean the Surveys Are Unreliable?
Critics sometimes argue that framing effects make AI doom surveys untrustworthy. Supporters generally draw a different conclusion.
The stronger interpretation is that the surveys are measuring a genuinely ambiguous subject. Experts are being asked to forecast unprecedented developments involving uncertain technologies, uncertain social responses, and uncertain scientific breakthroughs.
Under those conditions, framing sensitivity may reflect real uncertainty rather than survey failure.
A useful comparison is climate forecasting or pandemic forecasting. Estimates change depending on assumptions about policy responses, technological advances, and human behaviour. AI doom surveys face even greater uncertainty because there is no historical record of superhuman AI systems to calibrate against.
From this perspective, the variation between question framings is itself valuable information. It reveals which assumptions matter most to experts and where disagreement is concentrated.
What Readers Should Take From p(doom) Surveys
The most important lesson is that p(doom) figures are not objective measurements like temperatures or election vote counts.
They are structured judgements produced under uncertainty. The exact number depends partly on:
- How doom is defined.
- Which scenarios are included.
- Whether control failures are emphasised.
- What timeframe is specified.
- Which experts are surveyed. [arxiv.org]arxiv.orgarXivWhy do Experts Disagree on Existential Risk and P(doom)…23 Feb 2025 — I surveyed 111 AI experts on their familiarity with AI safe…
- How results are summarised and reported.
When one survey reports a median p(doom) of 5% and another reports 10% or higher, that does not necessarily indicate a dramatic shift in expert opinion. Sometimes it reflects a different question being asked. [AI Impacts Wiki]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc… [AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc…
For discussions of AI doom, the most informative takeaway is often not the precise percentage. It is the fact that substantial numbers of AI researchers assign non-trivial probabilities to catastrophic outcomes across a range of survey framings, while also disagreeing sharply about which assumptions justify those estimates. The wording of the question does not create that disagreement, but it often reveals where the deepest uncertainties lie.
Amazon book picks
Further Reading
Books and field guides related to How Survey Design Shapes AI Doom Predictions. Use these as the next step if you want deeper reading beyond the article.
Thinking, Fast and Slow
Explains framing effects and biases relevant to survey design.
The Art of Statistics
Helps readers understand uncertainty and interpretation of survey results.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: arxiv.org
Link: https://arxiv.org/html/2502.14870v1Source snippet
arXivWhy do Experts Disagree on Existential Risk and P(doom)...23 Feb 2025 — I surveyed 111 AI experts on their familiarity with AI safe...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2502.14870Source snippet
Why do Experts Disagree on Existential Risk and P(doom)...by S Field · 2025 · Cited by 8 — I surveyed 111 AI experts on their familiarit...
-
Source: lesswrong.com
Title: ai impacts 2023 [expert survey]({{ ‘survey-estimates/’ | relative_url }}) on progress in ai
Link: https://www.lesswrong.com/posts/RkegCmCgjGhskiFvm/ai-impacts-2023-expert-survey-on-progress-in-aiSource snippet
LessWrongAI Impacts 2023 Expert Survey on Progress in AI5 Jan 2024 — We asked about the likelihood that AI will cause human extinction us...
-
Source: arxiv.org
Link: https://arxiv.org/html/2401.02843v1Source snippet
More than half...Read more...
-
Source: lesswrong.com
Title: ai impacts survey december 2023 edition
Link: https://www.lesswrong.com/posts/NfPxAp5uwgZugwovY/ai-impacts-survey-december-2023-editionSource snippet
AI Impacts Survey: December 2023 Edition5 Jan 2024 — In Figure 13's question 3, we have 14.4% mean chance of either human extinction or s...
Published: december 2023
-
Source: wiki.aiimpacts.org
Title: 2023 expert survey on progress in ai
Link: https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_aiSource snippet
AI Impacts Wiki2023 Expert Survey on Progress in AI17 Aug 2023 — “What probability do you put on human inability to control future advanc...
-
Source: blog.aiimpacts.org
Link: https://blog.aiimpacts.org/p/faq-expert-survey-on-progress-inSource snippet
AI Impacts BlogFAQ: Expert Survey on Progress in AI methodology31 Oct 2025 — What probability do you put on future AI advances causing hu...
-
Source: aiimpacts.org
Title: Thousands of AI authors on the future of AI
Link: https://aiimpacts.org/wp-content/uploads/2023/04/Thousands_of_AI_authors_on_the_future_of_AI.pdfSource snippet
AI ImpactsTHOUSANDS OF AI AUTHORS ON THE FUTURE OF AIby K Grace · 2024 · Cited by 208 — Question 1: What probability do you put on future...
-
Source: aiimpacts.org
Title: how bad a future do ml researchers expect
Link: https://aiimpacts.org/how-bad-a-future-do-ml-researchers-expect/Source snippet
?8 Mar 2023 — To check, we added two more questions in 2022 explicitly about 'human extinction or similarly permanent and severe disempow...
-
Source: aiimpacts.org
Title: what do ml researchers think about ai in 2022
Link: https://aiimpacts.org/what-do-ml-researchers-think-about-ai-in-2022/Source snippet
?4 Aug 2022 — The median respondent's probability of x-risk from humans failing to control AI was 10%, weirdly more than median chance of...
-
Source: blog.aiimpacts.org
Title: how bad a future do ml researchers
Link: https://blog.aiimpacts.org/p/how-bad-a-future-do-ml-researchersSource snippet
bad a future do ML researchers expect?10 May 2023 — To check, we added two more questions in 2022 explicitly about 'human extinction or s...
Published: May 2023
-
Source: thezvi.substack.com
Title: ai impacts survey december 2023 edition
Link: https://thezvi.substack.com/p/ai-impacts-survey-december-2023-editionSource snippet
Impacts Survey: December 2023 Edition - Zvi MowshowitzIn Figure 13's question 3, we have 14.4% mean chance of either human extinction or...
Published: december 2023
-
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/P%28doom%29Source snippet
P(doom)In a 2023 survey, AI researchers were asked to estimate the probability that future AI advancements could lead to human extinct...
-
Source: blog.biocomm.ai
Link: https://blog.biocomm.ai/2024/02/25/ai-impacts-report-thousands-of-ai-authors-on-the-future-of-ai-38participants-put-at-least-a-10-chance-on-extremely-bad-outcomes-e-g-human-extinction-january-2024/Source snippet
biocomm.ai38% of participants put at least a 10% chance on extremely...25 Feb 2024 — AI Impacts Report January 2024 · THOUSANDS OF AI AU...
Published: January 2024
-
Source: normaltech.ai
Title: ai existential risk probabilities
Link: https://www.normaltech.ai/p/ai-existential-risk-probabilitiesSource snippet
are too unreliable to inform policy26 Jul 2024 — On the one hand, existential risks (x-risks) are necessarily somewhat speculative: by th...
Additional References
-
Source: merriam-webster.com
Link: https://www.merriam-webster.com/thesaurus/impactSource snippet
IMPACT Synonyms: 193 Similar and Opposite WordsSynonyms for IMPACT: effect, influence, consequence, repercussion, sway, importance, signi...
-
Source: impact.org.uk
Link: https://impact.org.uk/Source snippet
Impact – Improving health, preventing disabilityIMPACT's partners around the world take action to prevent and treat needless disability a...
-
Source: collinsdictionary.com
Link: https://www.collinsdictionary.com/dictionary/english/impactSource snippet
the striking of one thing against another; forceful contact; collision · 2. an impinging · 3. influence; effect · 4. an...Read more...
-
Source: forum.effectivealtruism.org
Title: the argument for near term human disempowerment through ai
Link: https://forum.effectivealtruism.org/posts/CcAuHCXDLjCH4bGyD/the-argument-for-near-term-human-disempowerment-through-aiSource snippet
argument for near-term human disempowerment...16 Apr 2024 — This paper provides an argument that AI will lead to the permanent disempowe...
-
Source: researchgate.net
Title: 396256646 Thousands of AI Authors on the Future of AI
Link: https://www.researchgate.net/publication/396256646_Thousands_of_AI_Authors_on_the_Future_of_AISource snippet
AI advances. causing human extinction or similarly permanent and severe. disempowerment of the human species? Question 2: What probabilit...
-
Source: linkedin.com
Link: https://www.linkedin.com/pulse/spoiler-only-5-chance-human-extinction-because-ai-other-stave-phd-ppmeeSource snippet
cal guardrails, and fostering international cooperation to prevent...Read more...
-
Source: forum.effectivealtruism.org
Title: survey of 2 778 ai authors six parts in pictures
Link: https://forum.effectivealtruism.org/posts/M9MSe4KHNv4HNf44f/survey-of-2-778-ai-authors-six-parts-in-picturesSource snippet
Median respondents put 5% or more on advanced AI leading to human extinction or similar, and a third to a half of participants gave 10% o...
-
Source: researchgate.net
Title: Why do experts disagree on existential risk?
Link: https://www.researchgate.net/publication/392941122_Why_do_experts_disagree_on_existential_risk_A_survey_of_AI_expertsSource snippet
A survey of AI...23 Jun 2025 — Leading AI labs and scientists have called for the global prioritization of AI safety, citing existential...
-
Source: asterisk.dynevor.org
Title: Asterisk Risk from artificial intelligence
Link: https://asterisk.dynevor.org/risk-from-artificial-intelligence.htmlSource snippet
from artificial intelligence - Asterisk - Matthew Brett1 May 2023 — 50% gave a 10% chance or higher of “human inability to control future...
Published: May 2023
-
Source: semanticscholar.org
Link: https://www.semanticscholar.org/paper/Why-do-Experts-Disagree-on-Existential-Risk-and-A-Field/38f774f51534dd13cea1137c4a347b046f741b66Source snippet
s · 7 Citations · 29 References.Read more...
Topic Tree