Why Do AI Doom Survey Numbers Change?

Introduction

Numbers from expert surveys about AI doom often appear to contradict one another. One headline may say that the typical AI researcher assigns a 5% chance of extinction, while another highlights a 10% estimate or reports concern about “extremely bad outcomes”. In many cases, these differences do not reflect sudden changes in expert opinion. They reflect changes in the question being asked.

This is one of the most important limitations of using expert surveys to estimate p(doom)—the estimated probability that advanced AI could permanently destroy humanity’s future. Small changes in wording can alter what respondents imagine, which scenarios they include, and how they interpret the probability they are being asked to estimate. As a result, survey results should be compared carefully rather than treated as directly interchangeable.

Why Do AI Doom Survey Numbers Change?

Survey researchers have long known that responses to uncertain questions depend partly on how those questions are framed. This effect becomes especially important when asking about unprecedented events such as AI-driven existential catastrophe, where there is no agreed definition or historical reference point.

In AI-risk surveys, seemingly modest wording differences can change at least four things simultaneously:

• The outcome being estimated. Is the question about literal human extinction, permanent human disempowerment, or any extremely bad outcome?
• The causal pathway. Is any AI-related catastrophe included, or only failures caused by humans losing control of advanced AI?
• The time horizon. Does the question ask about all future time or only the next century?
• The level of specificity. More detailed questions may prompt respondents to think about particular mechanisms rather than a general feeling of risk.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…Published: October 31, 2025

These are not trivial wording choices. They define different events.

How Different Questions Define Different Catastrophes

The largest recent expert surveys deliberately tested several closely related versions of existential-risk questions rather than relying on a single formulation. The results show why headlines that quote only one number can be misleading.

For example, respondents have been asked about:

• “Human extinction or similarly permanent and severe disempowerment of the human species.”
• “Human inability to control future advanced AI systems causing human extinction or similarly permanent and severe disempowerment.”
• “Extremely bad outcomes (e.g. human extinction).”
• Variants limited to the next 100 years rather than all future time.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…Published: October 31, 2025

Although these questions overlap, they are not equivalent.

A researcher might think that humanity is unlikely to become literally extinct but considerably more likely to suffer irreversible loss of control over civilisation. Another may regard “extremely bad outcomes” as including permanent authoritarian rule, irreversible economic domination by AI systems, or other civilisation-ending scenarios that stop short of extinction.

Consequently, two surveys can report different-looking numbers while measuring partially different concepts.

Why the Same Researcher May Answer Each Differently

It may seem odd that the same expert could honestly give different probabilities depending on wording, but this is exactly what survey designers expect.

Extinction versus permanent disempowerment

Literal extinction is a narrow endpoint: every human dies.

Permanent and severe disempowerment is broader. It includes futures where humans survive biologically but permanently lose meaningful control over civilisation or their long-term future. Many AI safety researchers regard such outcomes as existential because humanity’s ability to shape its own future has effectively ended, even if people remain alive.

Someone who assigns:

might reasonably answer 10% when both possibilities are included in the question.

The larger number would not represent greater pessimism. It reflects a broader definition of catastrophe.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…Published: October 31, 2025

“Extremely bad outcomes” invites wider interpretation

Another commonly used formulation asks about “extremely bad outcomes (e.g. human extinction)”.

The phrase “for example” deliberately leaves room for respondents to include catastrophes that they personally judge comparable to extinction.

Different researchers therefore imagine different reference classes:

The survey records one numerical answer but not exactly which scenario each respondent had in mind. The AI Impacts survey organisers explicitly identify this as an important limitation of interpreting the results.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…Published: October 31, 2025

Specifying a causal mechanism

Questions can also specify why catastrophe happens.

Compare:

• “future AI advances causing…”
• “human inability to control future advanced AI systems causing…”

The second wording narrows attention to one family of scenarios centred on loss of control or alignment failures.

Interestingly, the 2022 survey found a median estimate of 10% for the more specific control-failure question compared with 5% for the broader formulation. Researchers caution against reading too much into this difference because different randomly selected groups answered each version, but it illustrates that wording effects do not always move in the obvious direction.[AI Impacts]wiki.aiimpacts.org2022 expert survey on progress in aiAI Impacts2022 Expert Survey on Progress in AI [AI Impacts Wiki]…

Why Survey Designers Randomise Question Variants

Rather than treating wording differences as a flaw, recent surveys intentionally use multiple versions.

In the 2023 Expert Survey on Progress in AI, respondents were randomly assigned different variants of existential-risk questions. This allowed researchers to test whether conclusions remained broadly stable across different phrasings instead of depending entirely on one sentence.

Across the variants, the central finding remained similar:

• median responses clustered around roughly 5%, with one closely related variant producing a 10% median;
• thousands of respondents answered one or more versions of these questions rather than only a small subgroup;
• every respondent also answered another related question about extremely bad outcomes, providing an additional consistency check.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts BlogFAQ: Expert Survey on Progress in AI methodologyOctober 31, 2025…Published: October 31, 2025

This does not eliminate framing effects, but it makes the overall conclusions more robust than relying on a single wording.

How Headlines Flatten Important Distinctions

Media reporting often compresses nuanced survey questions into much simpler claims.

For example, a headline may state:

“AI experts think there is a 5% chance AI causes human extinction.”

That sounds straightforward, but the underlying survey may actually have asked about:

Each simplification removes context that affects interpretation.

Likewise, comparing two surveys can become misleading if one asks about extinction alone while another asks about broader existential catastrophe. The resulting numbers are not necessarily inconsistent—they may simply describe different events.

What This Means for Interpreting p(doom)

Because p(doom) has no universally accepted definition, survey wording inevitably shapes measured estimates.

Readers should therefore ask several questions before comparing figures:

These details matter because expert disagreement often concerns the boundaries between different catastrophic futures as much as the overall likelihood that advanced AI could permanently end humanity’s long-term prospects.

The practical lesson is not that expert surveys are unreliable. Rather, it is that their numbers should be read as estimates tied to carefully defined questions, not as universally applicable probabilities. Small wording changes can shift estimates because respondents are genuinely evaluating different futures, not merely changing their minds.

Amazon book picks

Further Reading

Books and field guides related to Why Do AI Doom Survey Numbers Change?. Use these as the next step if you want deeper reading beyond the article.

Book

Human Compatible

By Stuart Russell

A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...

See on Amazon

Book

The Alignment Problem

By Brian Christian

Finalist for the Los Angeles Times Book Prize A jaw-dropping exploration of everything that goes wrong when we build AI systems and the m...

See on Amazon

Book

Superforecasting

By Philip Tetlock, Dan Gardner

The international bestseller 'A manual for thinking clearly in an uncertain world. Read it.' Daniel Kahneman, author of Thinking, Fast an...

See on Amazon

Book

Thinking, Fast and Slow

By Daniel Kahneman

Why is there more chance we'll believe something if it's in a bold type face? Why are judges more likely to deny parole before lunch? Why...

See on Amazon

Browse more on Amazon:Human CompatibleThe Alignment ProblemSuperforecasting

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Selected fromdata science poster oneBay.co.uk.

Current eBay listing

Data Science Girl Definition Data S Framed Wall Art Poster Canvas Print Picture

GBP 29.95 | Shipping GBP 4.95NewSeller: silkframe (99.7% positive)Ships from: GB

View listing on eBay

Current eBay listing

Data Science Scientist Framed Wall Art Poster Canvas Print Picture

GBP 8.99 | Shipping GBP 4.99NewSeller: mirrashop2018 (99.8% positive)Ships from: GB

View listing on eBay

Current eBay listing

Data Science Girl Definition Data S Framed Wall Art Poster Canvas Print Picture

GBP 29.95 | Shipping GBP 4.95NewSeller: silkframe (99.7% positive)Ships from: GB

View listing on eBay

Current eBay listing

Data Science Girl Definition Data S Framed Wall Art Poster Canvas Print Picture

GBP 29.95 | Shipping GBP 4.95NewSeller: silkframe (99.7% positive)Ships from: GB

View listing on eBay

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: blog.aiimpacts.org
Title: AI Impacts Blog FAQ: Expert Survey on Progress in AI methodology
Link:https://blog.aiimpacts.org/p/faq-expert-survey-on-progress-in

Published: October 31, 2025

2. Source: wiki.aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2022_expert_survey_on_progress_in_ai

3. Source: wiki.aiimpacts.org
Title: 2023 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai

4. Source: lesswrong.com
Title: ai impacts 2023 expert survey on progress in ai
Link:https://www.lesswrong.com/posts/RkegCmCgjGhskiFvm/ai-impacts-2023-expert-survey-on-progress-in-ai

5. Source: lesswrong.com
Title: ai impacts survey december 2023 edition
Link:https://www.lesswrong.com/posts/NfPxAp5uwgZugwovY/ai-impacts-survey-december-2023-edition
Published: december 2023

6. Source: theverysoon.com
Title: ai impacts survey december 2023 edition
Link:https://theverysoon.com/ai-impacts-survey-december-2023-edition/
Published: december 2023

7. Source: blog.aiimpacts.org
Title: 2023 [ai survey]({{ ‘survey-bias/’ | relative_url }}) of 2778 six things
Link:https://blog.aiimpacts.org/p/2023-ai-survey-of-2778-six-things?action=share

8. Source: wiki.aiimpacts.org
Title: ai risk surveys
Link:https://wiki.aiimpacts.org/uncategorized/ai_risk_surveys

9. Source: grjenkin.com
Title: 2022 expert survey on progress in ai
Link:https://grjenkin.com/articles/category/machine-learning/6892881/03/20/2023/2022-expert-survey-on-progress-in-ai

10. Source: aiimpacts.org
Title: How bad a future do ML researchers expect? – AI Impacts
Link:https://aiimpacts.org/how-bad-a-future-do-ml-researchers-expect/

11. Source: aiimpacts.org
Title: what do ml researchers think about ai in 2022
Link:https://aiimpacts.org/what-do-ml-researchers-think-about-ai-in-2022/

12. Source: aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/

Additional References

13. Source: researchgate.net
Title: (PDF) Why do Experts Disagree on Existential Risk and P(doom)?
Link:https://www.researchgate.net/publication/389274220_Why_do_Experts_Disagree_on_Existential_Risk_and_Pdoom_A_Survey_of_AI_Experts

Published: January 25, 2025

14. Source: doi.org
Link:https://doi.org/10.1007/s00146-024-02134-4

15. Source: youtube.com
Link:https://www.youtube.com/watch?v=8CeplhPjVSI

16. Source: aiwiki.ai
Title: Existential risk from AI | AI Wiki
Link:https://aiwiki.ai/wiki/ai_existential_risk

17. Source: youtube.com
Link:https://www.youtube.com/watch?v=NO8WR85wqIo

18. Source: youtube.com
Title: The Catastrophic Risks of AI — and a Safer Path | Yoshua Bengio | TED
Link:https://www.youtube.com/watch?v=qe9QSCF-d88

19. Source: pmc.ncbi.nlm.nih.gov
Link:https://pmc.ncbi.nlm.nih.gov/articles/PMC12037001/

20. Source: nature.com
Link:https://www.nature.com/articles/s41598-019-50145-9

21. Source: nature.com
Link:https://www.nature.com/articles/s41562-024-02026-z

22. Source: beta.effectivealtruism.org
Title: Lxu Ku Qd69Qx5FKh NZ
Link:https://beta.effectivealtruism.org/posts/LxuKuQd69Qx5FKhNZ?commentId=9xBzwQvGyRbDcYdg8

Published: March 25, 2026