How Much AI Doom Risk Lies Within 100 Years?

Introduction

One of the most revealing findings in the 2023 Expert Survey on Progress in AI is not that researchers assigned a 5% median probability to AI causing human extinction or similarly permanent and severe disempowerment, but that this median did not change when the question was restricted to events occurring within the next 100 years. That result matters because it shows how survey wording shapes the interpretation of p(doom), the informal shorthand for the probability that advanced AI could cause an existential catastrophe. Rather than pushing the estimated risk far into the distant future, many respondents appear to believe that if such a catastrophe happens at all, a substantial share of the risk lies within roughly the next century. At the same time, the survey also found a lower average probability under the 100-year wording, suggesting that some respondents did reserve part of their risk estimate for much more distant futures.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

Unrestricted risk versus a 100-year deadline

The 2023 survey randomly assigned different versions of closely related existential-risk questions to different groups of AI researchers. One version asked about the probability that future AI advances would eventually cause human extinction or similarly permanent and severe disempowerment. Another asked the same question but limited the outcome to within the next 100 years. This experimental design was intended to measure how much wording alone affected responses rather than comparing different groups with different backgrounds.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

The headline result was striking:

Because a 100-year deadline is necessarily more restrictive than “ever”, one might have expected the second figure to be noticeably lower. Instead, the median remained identical.

This does not prove that researchers think all existential AI risk would occur within a century. Rather, it indicates that the middle respondent in the distribution did not feel the need to reduce their estimate simply because a century-long deadline had been introduced.

Why the median stayed at 5%

The unchanged median offers an important clue about how many respondents mentally picture AI existential risk.

For many researchers, concern appears to centre on the period during which advanced AI is expected to become capable enough to transform civilisation. In the same survey, aggregate forecasts for human-level machine intelligence and many transformative AI capabilities clustered within this century rather than hundreds of years ahead. If respondents believe the highest-risk transition occurs during that technological period, adding a 100-year limit may remove relatively little probability mass from their judgement.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

It is also worth remembering what the median represents. A median of 5% simply means that half of respondents gave estimates above 5% and half below. The statistic is insensitive to changes among a minority of respondents with unusually high or unusually low estimates. Consequently, some experts may indeed have lowered their estimates under the 100-year wording without changing the survey’s median.

The AI Impacts researchers explicitly designed these wording experiments to test the robustness of expert judgements across different phrasings. Their interpretation is cautious: the broadly similar medians suggest that concern about existential AI risk is reasonably stable across these closely related formulations, even though individual respondents undoubtedly interpret them somewhat differently.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts Blog FAQ: Expert Survey on Progress in AI methodology

What the lower average suggests about distant risk

Although the median remained unchanged, the average (mean) estimate for the 100-year question was somewhat lower than for the unrestricted version. That difference is informative because averages are influenced by respondents assigning very high probabilities.

The simplest interpretation is that a subset of researchers places meaningful probability on existential AI catastrophe occurring after the first century. Those respondents would naturally reduce their estimates when forced to consider only the next 100 years, lowering the average even if the typical respondent’s estimate remained unchanged.

This pattern illustrates why both median and mean deserve attention:

• The median describes the typical central judgement.
• The mean reflects the influence of respondents with especially high estimates.
• A stable median combined with a lower mean suggests that long-term risk exists in some experts’ thinking but is not the dominant driver of the overall distribution.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

Importantly, the survey was not designed to estimate exactly what proportion of total p(doom) respondents assign beyond 100 years. It simply demonstrates that introducing a century-long deadline affects some respondents more than others.

What this tells us about interpreting p(doom)

The 100-year experiment highlights an often-overlooked feature of p(doom): the number always depends on the question being asked.

A statement such as “there is a 5% chance of AI doom” can conceal several assumptions:

The survey shows that changing only one of these assumptions can alter parts of the distribution even when the headline median remains unchanged. Readers should therefore avoid comparing p(doom) figures unless the underlying wording matches closely.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

What the evidence does—and does not—show

The 100-year wording experiment provides evidence about expert judgement, not about the true probability of existential catastrophe.

Several points remain important:

• The survey records subjective forecasts from AI researchers rather than objective measurements.
• Random assignment means small differences between question variants may arise partly through ordinary sampling variation.
• The unchanged median should not be interpreted as proof that experts believe nearly all AI existential risk lies within the next century.
• The lower average indicates that at least some respondents do allocate meaningful probability to more distant futures.[AI Impacts Blog]blog.aiimpacts.orgAI Impacts Blog FAQ: Expert Survey on Progress in AI methodologyAI Impacts Blog FAQ: Expert Survey on Progress in AI methodology

Taken together, the findings suggest a nuanced conclusion. For many researchers, imposing a 100-year deadline leaves their central estimate of p(doom) essentially unchanged, implying that much of the risk they envisage already falls within that timeframe. At the same time, the modest reduction in the average estimate shows that not everyone shares this view, and that some experts still expect a non-trivial portion of any existential AI risk to emerge only over much longer horizons.[AI Impacts]wiki.aiimpacts.org2023 expert survey on progress in aiAI Impacts2023 Expert Survey on Progress in AI [AI Impacts Wiki]August 17, 2023…Published: August 17, 2023

Amazon book picks

Further Reading

Books and field guides related to How Much AI Doom Risk Lies Within 100 Years?. Use these as the next step if you want deeper reading beyond the article.

Book

Human Compatible

By Stuart Russell

A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...

See on Amazon

Book

Life 3.0

By Max Tegmark

'This is the most important conversation of our time, and Tegmark's thought-provoking book will help you join it' Stephen Hawking THE INT...

See on Amazon

Book

Superforecasting

By Philip Tetlock, Dan Gardner

The international bestseller 'A manual for thinking clearly in an uncertain world. Read it.' Daniel Kahneman, author of Thinking, Fast an...

See on Amazon

Book

The Precipice

By Toby Ord

What existential threats does humanity face? And how can we secure our future?'The Precipice is a powerful book . . . Ord's love for huma...

See on Amazon

Browse more on Amazon:Human CompatibleLife 3.0Superforecasting

As an Amazon Associate I earn from qualifying purchases.

eBay marketplace picks

Marketplace Samples

Live-tested eBay searches with available results related to this page.

Shop location

UsingUSA USA USAUKCanadaAustraliaIreland

Selected fromfuture technology poster oneBay.co.uk.

Current eBay listing

The Future of Technology Framed Wall Art Poster Canvas Print Picture

GBP 24.95 | Shipping GBP 4.99NewSeller: ruppalsingh (99.6% positive)Ships from: GB

View listing on eBay

Current eBay listing

Future technology Framed Wall Art Poster Canvas Print Picture

GBP 24.95 | Shipping GBP 4.99NewSeller: mirrashop2018 (99.8% positive)Ships from: GB

View listing on eBay

Current eBay listing

Magazine Cover Motor Technology Future City 30X40 Fine Art Print Poster Bb8003

GBP 12.49 | Free shippingUnspecifiedSeller: bumblebeaver (99.8% positive)Ships from: GB

View listing on eBay

Current eBay listing

Future science and technology Frame Framed Wall Art Poster Canvas Print Picture

GBP 29.95 | Shipping GBP 4.95NewSeller: silkframe (99.7% positive)Ships from: GB

View listing on eBay

Browse more oneBay.co.uk

Example items shown for inspiration; availability and pricing can change. Branchoria may earn a commission if you purchase through outbound eBay links.

Endnotes

1. Source: wiki.aiimpacts.org
Title: 2023 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2023_expert_survey_on_progress_in_ai

Published: August 17, 2023

2. Source: blog.aiimpacts.org
Title: AI Impacts Blog FAQ: Expert Survey on Progress in AI methodology
Link:https://blog.aiimpacts.org/p/faq-expert-survey-on-progress-in

3. Source: blog.aiimpacts.org
Title: reanalyzing the 2023 expert survey
Link:https://blog.aiimpacts.org/p/reanalyzing-the-2023-expert-survey

Published: December 16, 2024

4. Source: lesswrong.com
Title: I’ve found this
Link:https://www.lesswrong.com/posts/RkegCmCgjGhskiFvm/ai-impacts-2023-expert-survey-on-progress-in-ai

Published: January 5, 2024

5. Source: theverysoon.com
Title: ai impacts survey december 2023 edition
Link:https://theverysoon.com/ai-impacts-survey-december-2023-edition/
Published: december 2023

6. Source: blog.aiimpacts.org
Title: 2023 [ai survey]({{ ‘survey-bias/’ | relative_url }}) of 2778 six things
Link:https://blog.aiimpacts.org/p/2023-ai-survey-of-2778-six-things?action=share

7. Source: aiimpacts.org
Title: How bad a future do ML researchers expect? – AI Impacts
Link:https://aiimpacts.org/how-bad-a-future-do-ml-researchers-expect/

8. Source: blog.aiimpacts.org
Title: scoring forecasts from the 2016 expert survey on progress in ai
Link:https://blog.aiimpacts.org/p/scoring-forecasts-from-the-2016-expert-survey-on-progress-in-ai

9. Source: wiki.aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://wiki.aiimpacts.org/ai_timelines/predictions_of_human-level_ai_timelines/ai_timeline_surveys/2022_expert_survey_on_progress_in_ai

10. Source: aiimpacts.org
Title: 2022 expert survey on progress in ai
Link:https://aiimpacts.org/2022-expert-survey-on-progress-in-ai/

11. Source: aiimpacts.org
Title: Some survey results! – AI Impacts
Link:https://aiimpacts.org/some-survey-results/

12. Source: aiimpacts.org
Link:https://aiimpacts.org/blog/

13. Source: aiimpacts.org
Link:https://aiimpacts.org/

14. Source: youtube.com
Title: AI Impacts Survey
Link:https://www.youtube.com/watch?v=xwJx_xqZI3Q

Additional References

15. Source: lesswrong.com
Title: FA Q: Expert Survey on Progress in AI methodology — Less Wrong
Link:https://www.lesswrong.com/posts/BEg5dct7nr2Yq3dJY/faq-expert-survey-on-progress-in-ai-methodology

16. Source: spectrum.ieee.org
Title: ai existential risk survey 2667013207
Link:https://spectrum.ieee.org/amp/ai-existential-risk-survey-2667013207

Published: January 25, 2024

17. Source: aiwiki.ai
Title: Existential risk from AI | AI Wiki
Link:https://aiwiki.ai/wiki/ai_existential_risk

18. Source: forum.effectivealtruism.org
Title: faq expert survey on progress in ai methodology
Link:https://forum.effectivealtruism.org/posts/c977usNg2AmF2m7ud/faq-expert-survey-on-progress-in-ai-methodology

19. Source: youtube.com
Title: Will AI end everything? A guide to guessing | Katja Grace | EAG Bay Area 23
Link:https://www.youtube.com/watch?v=j5Lu01pEDWA

20. Source: youtube.com
Title: Katja Grace on the Largest Survey of AI Researchers
Link:https://www.youtube.com/watch?v=qnqdTAO5OXw

21. Source: weforum.org
Title: How do experts view the development of AI for the future? | World Economic Forum
Link:https://www.weforum.org/agenda/2023/02/experts-ai-developing-over-the-coming-years/

22. Source: weforum.org
Title: How do experts view the development of AI for the future? | World Economic Forum
Link:https://www.weforum.org/stories/2023/02/experts-ai-developing-over-the-coming-years/

23. Source: arxiv.org
Title: arXiv Thousands of AI Authors on the Future of AI
Link:https://arxiv.org/abs/2401.02843

24. Source: youtube.com
Link:https://www.youtube.com/watch?v=fI_1qjjdx5k