Could People Use AI to Cause Extinction?

Introduction

Could people use advanced AI to cause human extinction? In principle, yes—but every credible route from an AI model to extinction contains major practical obstacles. The strongest concern is not that a chatbot supplies a single forbidden recipe. It is that future systems could combine expert scientific advice, automated experimentation, cyber intrusion, operational planning and large-scale coordination, allowing a malicious state or group to overcome barriers that currently keep biological, military or environmental threats difficult to execute.

Biological misuse is usually treated as the most plausible route to exceptionally large casualties. Cyber-enabled attacks could support it, disable defences or destabilise nuclear crises. AI might also intensify military escalation or help an actor plan reckless geoengineering. Yet causing extinction is much harder than causing a disaster: an attacker would need an extraordinarily destructive mechanism, reliable worldwide delivery, resistance to countermeasures and some way of preventing human populations from surviving or recovering. A detailed RAND analysis judged AI-assisted extinction through pathogens, nuclear weapons or geoengineering to be immensely challenging, although not impossible to rule out.[rand.org]rand.orgD. Vermeer, Emily LaOn the Extinction Risk from Artificial Intelligence | RANDMay 6, 2025 — On the Extinction Risk from Artificial Intelligence | RAND RAND R…Published: May 6, 2025

This distinction matters for discussions of AI doom and p(doom)—a person’s subjective estimate of the chance that advanced AI causes an existential catastrophe. Misuse should neither be dismissed because extinction is difficult nor treated as a straightforward extension of present-day cybercrime or bioterrorism. The real question is how much AI changes the capabilities, costs and number of actors able to attempt extreme harm.

Biological and cyber misuse pathways

AI could lower some barriers to biological weapons

Creating a globally devastating biological weapon is not a single information problem. It requires selecting or designing an agent, obtaining materials, carrying out laboratory work, solving failures, producing it reliably, avoiding self-infection and surveillance, and delivering it effectively. Much relevant knowledge is already available in scientific papers and specialist databases, but it is fragmented, technically demanding and difficult for non-experts to apply.

Advanced AI could make that knowledge easier to use. A capable system might search literature, translate technical terminology, compare candidate approaches, propose experimental procedures, diagnose unsuccessful results and guide a user through a long project. It could also connect biology with other skills—coding laboratory equipment, analysing genetic data, ordering materials or managing multiple experiments—which ordinary search engines do not integrate as readily.

Evidence that models can assist with parts of this process is becoming stronger, though it does not show that present systems can enable a successful bioweapon programme. The UK AI Security Institute reports that frontier models progressed from below expert performance on difficult biology questions in 2022 to above its PhD-level baseline by 2025. Its tests cover scientific knowledge, experimental design, laboratory protocols and troubleshooting, but benchmark success is not equivalent to handling organisms safely and reliably in a real laboratory.[aisi.gov.uk]aisi.gov.ukends Report Our first public, evidence‐based assessment of how the world’s most advanced AI systems are evolving, bringin…

Human “uplift” studies ask a more useful question: do people actually perform better with AI than with ordinary internet access? One 2026 preprint found that novices using language models were substantially more accurate on eight computer-based, dual-use biology task sets than internet-only controls. However, these were largely analytical and digital tasks rather than end-to-end wet-laboratory challenges, so the result demonstrates reduced knowledge barriers, not the ability to produce or release a catastrophic pathogen.[arXiv]arxiv.orgarXiv LLM Novice Uplift on Dual-Use, In Silico Biology TasksWe conducted a multi-model, multi-benchmark human uplift study comparing novices with LLM access versus internet-only access across eight…

Other work has found much weaker assistance on complex physical laboratory procedures. A study summarised in OpenAI’s safety documentation estimated only modest and highly uncertain improvement for novices attempting such tasks. This gap is important: models may already be good scientific advisers while remaining unable to compensate for poor equipment, tacit laboratory knowledge, contaminated samples or repeated experimental failure.[OpenAI Deployment Safety Hub]deploymentsafety.openai.comOpen AI Deployment Safety Hub GPT-5.4 Thinking System CardUnder our Preparedness Framework, High cybersecurity capability is defined as a model that removes existing bottlenecks to scaling cyber…

Even so, several developers have treated recent biological capabilities as serious enough to require enhanced safeguards. OpenAI has classified multiple systems as having “High” biological and chemical capability under its preparedness framework, while stressing that this classification measures potentially dangerous assistance rather than proof that a model can independently create a weapon. Anthropic’s scaling policy similarly identifies a threshold at which models could meaningfully help technically capable actors develop or deploy chemical or biological weapons. These are company-defined risk categories, not independent findings that catastrophe is imminent, but they show that the issue has moved from abstract speculation into deployment decisions.[openai.com]OpenAIupdating our preparedness frameworkWe’re releasing an update to our Preparedness Framework, our process for tracking and preparing for advanced AI capabilities that could i…

The most concerning future change would be AI-enabled research and development, not merely better answers to questions. Systems connected to automated laboratories might generate designs, direct robotic experiments, analyse results and iterate around failures. This could shorten development cycles or allow a small team to conduct work that currently requires a larger expert organisation. That remains a prospective capability, and reliable autonomous experimentation is much harder than producing plausible-looking instructions.

Cyber capability could connect advice to action

Cyber misuse matters to AI doom partly because modern civilisation depends on networked infrastructure. AI-assisted attackers could target hospitals, power systems, communications, financial networks, biotechnology facilities or military organisations. A severe cyberattack might cause many deaths or contribute to social collapse, but cyber operations alone are unlikely to kill every human. Their existential relevance is strongest when they enable another threat—for example, stealing pathogen research, sabotaging emergency responses or manipulating systems during a nuclear crisis.

Frontier models are improving at vulnerability discovery, exploitation and multi-step computer use. The UK AI Security Institute found rapid gains across its cyber evaluations, with recent systems completing some apprentice-level tasks at much higher rates than earlier models. A 2026 academic cyber-range benchmark also found that leading agents could complete meaningful portions of realistic exploitation and post-exploitation tasks, although they still failed most tasks without assistance.[aisi.gov.uk]aisi.gov.ukends Report Our first public, evidence‐based assessment of how the world’s most advanced AI systems are evolving, bringin…

AI could increase cyber risk in several different ways:

The key measure is not whether an AI can solve a staged hacking puzzle, but its marginal uplift over tools attackers already possess. Models can produce false positives, become stuck, misunderstand unfamiliar environments and expose their operators. Defenders can also use the same technology to find vulnerabilities, analyse logs and patch software. NIST therefore recommends evaluating misuse in relation to the attacker’s existing resources, the scale or efficiency the model adds and the defensive controls surrounding the target—not treating model capability as a risk in isolation.[NIST Publications]nvlpubs.nist.govOpen source on nist.gov.

This creates an unresolved attacker–defender race. Offence may gain an early advantage because software contains many weak points and an attacker needs to succeed only once, whereas defenders must protect numerous systems. On the other hand, AI-assisted patching, network monitoring and secure software development could reduce the available attack surface. Current evidence supports growing cyber capability, but not a confident prediction that advanced AI will make critical infrastructure globally indefensible.[arXiv]arxiv.orgarXiv Toward Quantitative Modeling of Cybersecurity Risks Due to AI MisusearXiv Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse

Nuclear escalation and geoengineering

AI is more likely to worsen a nuclear crisis than invent a bomb

Nuclear weapons already exist, so an AI-enabled catastrophe would not require a model to discover nuclear physics from scratch. The more realistic concerns involve hacking, intelligence analysis, targeting, autonomous military systems and decision support during a confrontation between nuclear-armed states.

Military AI could compress the time available for leaders to interpret warnings. It might combine sensor data rapidly, recommend responses or direct conventional systems whose actions threaten an adversary’s nuclear forces. If officials overtrust an opaque recommendation—an effect known as automation bias—they may treat uncertain information as confirmation of an attack. Rival states may also assume the worst about one another’s systems, creating pressure to launch or escalate before their forces are disabled.

The Stockholm International Peace Research Institute identifies several pathways by which military AI could raise nuclear escalation risk even when it is not directly connected to launch authority. These include shorter decision times, mistaken threat assessments and autonomous conventional weapons that appear capable of undermining an opponent’s retaliatory forces. The danger is therefore not simply a fictional “AI presses the nuclear button” scenario. Human leaders could deliberately rely on AI in ways that make crises faster, less transparent and less forgiving.[SIPRI]sipri.orgOpen source on sipri.org.

AI could also be used beneficially—to process early-warning data, identify false alarms or improve communications. Whether it increases or reduces danger depends on the system’s role, reliability, cybersecurity and the time humans retain for judgement. Analyses by the Federation of American Scientists emphasise that different forms of integration carry very different risks and that apparently desirable speed can remove opportunities for verification and diplomacy.[Federation of American Scientists]fas.orgMarch2026 AIxNuclear FASMarch2026 AIxNuclear FAS

Even a large nuclear war would not automatically mean human extinction. It could cause immediate mass death, infrastructure collapse and severe climatic effects, but eliminating all surviving populations would be exceptionally difficult. RAND concluded that an intentional actor would face major constraints in turning existing nuclear arsenals into a literal extinction mechanism. Nuclear escalation is therefore a grave global catastrophic risk, while its classification as a credible direct route to extinction remains disputed.[rand.org]rand.orgD. Vermeer, Emily LaOn the Extinction Risk from Artificial Intelligence | RANDMay 6, 2025 — On the Extinction Risk from Artificial Intelligence | RAND RAND R…Published: May 6, 2025

Geoengineering is powerful, but extinction scenarios are remote

Solar geoengineering aims to cool the planet by reflecting a fraction of incoming sunlight, for example by releasing particles into the upper atmosphere. AI could improve climate modelling, material discovery, deployment planning and automated control. Those same capabilities might help a reckless government or private actor pursue deployment without broad international agreement.

The major risks are not that a single prompt instantly changes the climate. Large-scale geoengineering would require aircraft, manufacturing, sustained logistical operations and continuing political control. Its effects could cross borders, alter rainfall patterns, damage the ozone layer or create diplomatic conflict over which regions benefit and which suffer. The US Government Accountability Office notes that outcomes remain highly uncertain and that international governance is limited.[GAO]gao.govgao 26 108837gao 26 108837

A particularly serious concern is termination shock. If solar geoengineering masked warming while greenhouse gases continued to accumulate, suddenly stopping the intervention could produce rapid temperature increases. AI might increase the danger by encouraging optimisation around incomplete models, lowering planning costs or placing complex deployment systems under automated management. Yet an extinction scenario would still require a long chain of assumptions: unusually harmful intervention, inadequate monitoring, failure to stop or modify it, widespread secondary effects and an inability of human populations to adapt.

Geoengineering therefore illustrates a broader point about AI misuse. AI need not supply the destructive energy itself. It can act as an accelerator for a hazardous technology controlled by people. But the physical scale, visibility and infrastructure required for planetary intervention make secret deployment by an isolated individual implausible. State action, institutional recklessness or geopolitical competition would be more credible sources of danger than a lone user with a powerful model.[rand.org]rand.orgD. Vermeer, Emily LaOn the Extinction Risk from Artificial Intelligence | RANDMay 6, 2025 — On the Extinction Risk from Artificial Intelligence | RAND RAND R…Published: May 6, 2025

Practical barriers to existential scale

The phrase “AI could help create a bioweapon” can hide the distance between assistance and extinction. To cause an existential catastrophe, an attacker would need to cross several separate barriers, and failure at any one could reduce the outcome from extinction to a contained incident—or prevent it altogether.

The capability must be genuinely new. A model that restates public knowledge may make work more convenient without changing what an attacker can ultimately accomplish. Risk rises more sharply when AI enables previously infeasible design, experimentation, vulnerability discovery or coordination.

Digital competence must survive contact with the physical world. Biology, nuclear systems and geoengineering depend on materials, machinery, skilled handling and quality control. Language models can sound confident while giving advice that fails under real conditions.

The actor needs access and resources. Dangerous pathogens require laboratories and biological materials. Nuclear use generally requires state authority or extraordinary penetration of protected systems. Planetary geoengineering requires visible industrial infrastructure. AI may lower some barriers without removing them.

The mechanism must scale globally. A local outbreak, regional blackout or military strike can be catastrophic without threatening humanity’s entire future. Extinction requires unusually broad reach, repeated delivery or cascading consequences across continents.

Defences must fail. Public-health surveillance, vaccines, network isolation, military safeguards, attribution, international pressure and human adaptation all obstruct catastrophic plans. Attackers must either evade these systems or move faster than they can respond.

Survivors and recovery matter. Humans live in varied climates and social systems, and some communities are geographically isolated. A disaster capable of killing billions may still leave populations able to rebuild. This makes literal extinction a substantially higher threshold than unprecedented mass death.

These barriers are reasons for scepticism, not grounds for complacency. AI may attack several obstacles simultaneously: improving technical plans, automating cyber access, reducing the expertise required, concealing suspicious activity and accelerating iteration. Risk could increase gradually and then sharply if systems become reliable across whole workflows rather than isolated benchmark tasks.

The greatest uncertainty is therefore not whether current models can answer dangerous questions. It is whether future systems will become dependable operational partners for actors who have access to laboratories, infrastructure, weapons or state power. The 2026 International AI Safety Report describes malicious-use risk as an interaction between model capability, access, actor intent and real-world safeguards. No benchmark alone establishes the probability of catastrophe.[internationalaisafetyreport.org]internationalaisafetyreport.orginternational ai safety report 2026International AI Safety ReportInternational AI Safety Report 2026…

What would change the risk assessment?

Several warning signs would make catastrophic misuse more plausible.

One would be repeated evidence that AI substantially improves performance in real laboratories, not just written biology tests. Another would be agents reliably conducting long cyber campaigns against hardened targets without close human direction. Analysts would also worry about systems that integrate scientific design, procurement, physical automation and operational security into one dependable workflow.

Access patterns matter as much as raw intelligence. A highly dangerous capability confined to secured environments presents a different risk from comparable models whose weights can be copied, modified and run without monitoring. Model theft, weak internal security or widespread release of systems that retain advanced biological or cyber capabilities would reduce the value of provider-level safeguards.

Institutional behaviour is another signal. Competitive pressure may encourage developers or states to deploy systems before evaluations are mature. Conversely, serious pre-deployment testing, secure model storage, external scrutiny and clear thresholds for withholding capabilities would indicate that misuse risks are being treated as operational rather than rhetorical concerns.

The most informative evidence would involve demonstrated uplift across bottlenecks: not merely generating an idea, but helping a user move from conception through experimentation or intrusion to a robust harmful outcome. Present evidence shows rapidly improving technical assistance, but remains incomplete on this end-to-end question.

Reducing catastrophic misuse risk

The most credible protections use several layers, because conversational refusals alone are easy to overestimate. A model may be jailbroken, stolen or accessed through an application whose surrounding tools create capabilities that were absent during testing.

Capability and uplift evaluations can test whether systems materially improve biological, chemical or cyber performance compared with ordinary tools. Evaluations should include realistic, multi-step settings and expert review while avoiding the release of dangerous procedural details.

Access controls can restrict the most capable models, apply stronger identity checks or limit risky tools such as unrestricted code execution and autonomous laboratory interfaces. These controls must be proportionate: many scientific and defensive uses are beneficial, and overly broad restrictions could impede medicine or cybersecurity.

Monitoring and incident response can identify suspicious interaction patterns, repeated attempts to bypass safeguards or unusual use of high-risk capabilities. Monitoring raises legitimate privacy and governance questions, so rules are needed for what is collected, who can inspect it and when authorities should be contacted.

Model and infrastructure security is essential because deployment filters become irrelevant if an attacker steals unrestricted model weights. NIST’s misuse-risk guidance calls for evaluating unauthorised access, threat actors’ resources and the effectiveness of safeguards throughout a model’s lifecycle.[NIST Publications]nvlpubs.nist.govOpen source on nist.gov.

Controls outside the AI model may be more reliable than trying to suppress knowledge. Screening DNA synthesis orders, securing laboratories, improving disease surveillance, hardening critical infrastructure and maintaining strict nuclear command procedures target the physical bottlenecks an attacker must cross. They can protect against both AI-assisted and conventional threats.

International coordination matters where unilateral restraint creates competitive disadvantages. Shared testing methods, incident reporting and agreed limits on AI involvement in nuclear decision-making could reduce the risk that states race towards unsafe deployment. In biology, the Nuclear Threat Initiative has argued for coordinated standards spanning model developers, governments and the life-sciences sector rather than treating AI safety and biosecurity as separate fields.[nti.org]nti.orgSafeguarding Against Global CatastropheSafeguarding Against Global Catastrophe

None of these measures makes catastrophic misuse impossible. Nor does the available evidence justify treating human-directed AI extinction as the default outcome of more capable models. The balanced conclusion is that AI is increasingly able to reduce informational and technical barriers, particularly in biology and cybersecurity, while the jump from assistance to existential destruction remains enormous. The risk becomes most serious where advanced models meet determined actors, physical resources, weak safeguards and competitive institutions willing to accept extreme danger.

Amazon book picks

Further Reading

Books and field guides related to Could People Use AI to Cause Extinction?. Use these as the next step if you want deeper reading beyond the article.

Book

Life 3.0

By Max Tegmark

NEW YORK TIMES BESTSELLER • How will Artificial Intelligence affect crime, war, justice, jobs, society and our very sense of being human?...

See on Amazon

Book

Superintelligence

By Nick Bostrom

The human brain has some capabilities that the brains of other animals lack. It is to these distinctive capabilities that our species owe...

See on Amazon

Book

The Coming Wave

By Mustafa Suleyman

NEW YORK TIMES BESTSELLER • An urgent warning of the unprecedented risks that AI and other fast-developing technologies pose to global or...

See on Amazon

Book

The Precipice

By Toby Ord

What existential threats does humanity face? And how can we secure our future?'The Precipice is a powerful book . . . Ord's love for huma...

See on Amazon

Browse more on Amazon:Life 3.0SuperintelligenceThe Coming Wave

As an Amazon Associate I earn from qualifying purchases.

Endnotes

1. Source: rand.org
Title: D. Vermeer, Emily La
Link:https://www.rand.org/pubs/research_reports/RRA3034-1.html

Published: May 6, 2025

2. Source: aisi.gov.uk
Link:https://www.aisi.gov.uk/frontier-ai-trends-report

3. Source: arxiv.org
Title: arXiv LLM Novice Uplift on Dual-Use, In Silico Biology Tasks
Link:https://arxiv.org/abs/2602.23329

4. Source: deploymentsafety.openai.com
Title: Open AI Deployment Safety Hub GPT-5.4 Thinking System Card
Link:https://deploymentsafety.openai.com/gpt-5-4-thinking/cyber-safeguards

5. Source: OpenAI
Title: updating our preparedness framework
Link:https://openai.com/index/updating-our-preparedness-framework/

6. Source: anthropic.com
Link:https://www.anthropic.com/responsible-scaling-policy/roadmap

7. Source: OpenAI
Title: chatgpt agent system card
Link:https://openai.com/index/chatgpt-agent-system-card/

8. Source: arxiv.org
Link:https://arxiv.org/abs/2606.14295

9. Source: nvlpubs.nist.gov
Link:https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.800-1.ipd2.pdf

10. Source: arxiv.org
Title: arXiv Toward Quantitative Modeling of Cybersecurity Risks Due to AI Misuse
Link:https://arxiv.org/abs/2512.08864

11. Source: arxiv.org
Link:https://arxiv.org/abs/2504.05408

12. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-06/2025_6_ai_and_nuclear_risk.pdf

13. Source: sipri.org
Link:https://www.sipri.org/publications/2025/sipri-insights-peace-and-security/impact-military-artificial-intelligence-nuclear-escalation-risk

14. Source: gao.gov
Title: gao 26 108837
Link:https://www.gao.gov/products/gao-26-108837

15. Source: arxiv.org
Link:https://arxiv.org/abs/2602.21012

16. Source: nti.org
Title: Safeguarding Against Global Catastrophe
Link:https://www.nti.org/analysis/articles/safeguarding-against-global-catastrophe/

17. Source: deploymentsafety.openai.com
Title: gpt 5 6
Link:https://deploymentsafety.openai.com/gpt

18. Source: anthropic.com
Title: ’s Responsible Scaling Policy \ Anthropic
Link:https://www.anthropic.com/responsible-scaling-policy

19. Source: deploymentsafety.openai.com
Title: prompt injection
Link:https://deploymentsafety.openai.com/gpt-5-5-instant/prompt-injection

20. Source: OpenAI
Title: introducing gpt 5 5
Link:https://openai.com/index/introducing-gpt-5-5/

21. Source: rand.org
Link:https://www.rand.org/pubs/perspectives/PEA4611-1.html

22. Source: rand.org
Link:https://www.rand.org/pubs/research_reports/RRA4727-1.html

23. Source: rand.org
Title: RRA4266 1
Link:https://www.rand.org/pubs/research_reports/RRA4266-1.html

24. Source: arxiv.org
Link:https://arxiv.org/abs/2602.21012v1

25. Source: anthropic.com
Title: Responsible Scaling Policy Version 3.0 \ Anthropic
Link:https://www.anthropic.com/news/responsible-scaling-policy-v3?e45d281a_page=1&field_format_value=3&uncat=12

26. Source: anthropic.com
Title: ’s Transparency Hub \ Anthropic
Link:https://www.anthropic.com/transparency/model-report

27. Source: deploymentsafety.openai.com
Link:https://deploymentsafety.openai.com/gpt-5-3-codex/cybersecurity

28. Source: aisi.gov.uk
Title: 5 key findings from our first Frontier AI Trends Report | AISI Work
Link:https://www.aisi.gov.uk/blog/5-key-findings-from-our-first-frontier-ai-trends-report

29. Source: aisi.gov.uk
Title: aisi frontier ai trends report 2025
Link:https://www.aisi.gov.uk/research/aisi-frontier-ai-trends-report-2025

30. Source: GOV.UK
Title: www.gov.uk A I Security Institute – Frontier AI Trends report factsheet
Link:https://www.gov.uk/government/publications/ai-security-institute-frontier-ai-trends-report-factsheet/ai-security-institute-frontier-ai-trends-report-factsheet

31. Source: GOV.UK
Link:https://www.gov.uk/government/news/inaugural-report-pioneered-by-ai-security-institute-gives-clearest-picture-yet-of-capabilities-of-most-advanced-ai

32. Source: deploymentsafety.openai.com
Title: preparedness framework
Link:https://deploymentsafety.openai.com/gpt-5-2/preparedness-framework

33. Source: arxiv.org
Link:https://arxiv.org/abs/2511.19863v1

34. Source: arxiv.org
Link:https://arxiv.org/abs/2511.19863

35. Source: alignment.anthropic.com
Title: sabotage risk report
Link:https://alignment.anthropic.com/2025/sabotage-risk-report/

36. Source: nist.gov
Title: caisi evaluation deepseek ai models finds shortcomings and risks
Link:https://www.nist.gov/news-events/news/2025/09/caisi-evaluation-deepseek-ai-models-finds-shortcomings-and-risks

37. Source: rand.org
Title: Global Risk Index for AI-enabled Biological Tools
Link:https://www.rand.org/pubs/external_publications/EP71093.html

38. Source: OpenAI
Title: gpt oss model card
Link:https://openai.com/index/gpt-oss-model-card/

39. Source: OpenAI
Title: estimating worst case frontier risks of open weight llms
Link:https://openai.com/index/estimating-worst-case-frontier-risks-of-open-weight-llms/?asuniq=68955115

40. Source: OpenAI
Title: preparing for future ai capabilities in biology
Link:https://openai.com/index/preparing-for-future-ai-capabilities-in-biology/

41. Source: sipri.org
Title: nuclear risks grow new arms race looms new sipri yearbook out now
Link:https://www.sipri.org/media/press-release/2025/nuclear-risks-grow-new-arms-race-looms-new-sipri-yearbook-out-now

42. Source: anthropic.com
Title: Cyber toolkits for LLMs \ Anthropic
Link:https://www.anthropic.com/research/cyber-toolkits

43. Source: anthropic.com
Title: Responsible Scaling Policy Updates \ Anthropic
Link:https://www.anthropic.com/rsp-updates?guides=image-generation-social-good

44. Source: GOV.UK
Title: www.gov.uk Frontier AI: capabilities and risks – discussion paper
Link:https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper/frontier-ai-capabilities-and-risks-discussion-paper

45. Source: OpenAI
Title: o3 o4 mini system card
Link:https://openai.com/index/o3-o4-mini-system-card/

46. Source: anthropic.com
Title: Progress from our Frontier Red Team \ Anthropic
Link:https://www.anthropic.com/news/strategic-warning-for-ai-risk-progress-and-insights-from-our-frontier-red-team

47. Source: OpenAI
Title: at the paris ai action summit
Link:https://openai.com/global-affairs/openai-at-the-paris-ai-action-summit/

48. Source: rand.org
Link:https://www.rand.org/content/dam/rand/pubs/research_reports/RRA2900/RRA2990-1/RAND_RRA2990-1.pdf?target=_blank

49. Source: rand.org
Title: RRA2990 1
Link:https://www.rand.org/pubs/research_reports/RRA2990-1.html

50. Source: nist.gov
Title: updated guidelines managing misuse risk dual use foundation models
Link:https://www.nist.gov/news-events/news/2025/01/updated-guidelines-managing-misuse-risk-dual-use-foundation-models

51. Source: anthropic.com
Title: Sabotage evaluations for frontier models \ Anthropic
Link:https://www.anthropic.com/research/sabotage-evaluations

52. Source: anthropic.com
Title: Announcing our updated Responsible Scaling Policy \ Anthropic
Link:https://www.anthropic.com/news/announcing-our-updated-responsible-scaling-policy

53. Source: anthropic.com
Title: A new initiative for developing third-party model evaluations \ Anthropic
Link:https://www.anthropic.com/news/a-new-initiative-for-developing-third-party-model-evaluations

54. Source: anthropic.com
Title: Reflections on our Responsible Scaling Policy \ Anthropic
Link:https://www.anthropic.com/news/reflections-on-our-responsible-scaling-policy

55. Source: anthropic.com
Title: Announcing Anthropic’s Responsible Scaling Policy \ Anthropic
Link:https://www.anthropic.com/news/anthropics-responsible-scaling-policy

56. Source: anthropic.com
Link:https://www.anthropic.com/policy

57. Source: rand.org
Link:https://www.rand.org/content/dam/rand/pubs/research_reports/RRA4200/RRA4219-1/RAND_RRA4219-1.pdf

58. Source: rand.org
Link:https://www.rand.org/content/dam/rand/pubs/research_reports/RRA2900/RRA2990-1/RAND_RRA2990-1.pdf

59. Source: rand.org
Link:https://www.rand.org/content/dam/rand/pubs/research_reports/RRA3000/RRA3034-1/RAND_RRA3034-1.pdf

60. Source: rand.org
Link:https://www.rand.org/content/dam/rand/pubs/research_reports/RRA4200/RRA4266-1/RAND_RRA4266-1.pdf

61. Source: aisi.gov.uk
Link:https://www.aisi.gov.uk/frontier-ai-trends-report?_bhlid=91134dd3ba5a9bde8455c7658ece4a62b69caa87

62. Source: aisi.gov.uk
Link:https://www.aisi.gov.uk/frontier-ai-trends-report?_bhlid=4d8c895f555ab0c1c5ab8ed3c8d5296650233e0f

63. Source: aisi.gov.uk
Link:https://www.aisi.gov.uk/research/measuring-ai-agents-progress-on-multi-step-cyber-attack-scenarios

64. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-10/pragmatic_approaches_to_governance_at_the_ai-nuclear_nexus.pdf

65. Source: sipri.org
Link:https://www.sipri.org/publications/2025/other-publications/pragmatic-approaches-governance-artificial-intelligence-nuclear-nexus

66. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-03/2025_3_advancing_governance_at_the_nexus.pdf

67. Source: sipri.org
Link:https://www.sipri.org/publications/2025/eu-non-proliferation-and-disarmament-papers/artificial-intelligence-non-proliferation-and-disarmament-compendium-state-art

68. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-09/2509_towards_multilateral_policy_on_aws_1.pdf

69. Source: sipri.org
Link:https://www.sipri.org/publications/2025/sipri-insights-peace-and-security/advancing-governance-nexus-artificial-intelligence-and-nuclear-weapons

70. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-01/eunpdc_no_92_0.pdf

71. Source: sipri.org
Link:https://www.sipri.org/sites/default/files/2025-11/1125_civilian_ai.pdf

72. Source: arxiv.org
Link:https://arxiv.org/pdf/2602.21012v1

73. Source: arxiv.org
Link:https://arxiv.org/pdf/2602.21012

74. Source: gao.gov
Title: gao 26 108837
Link:https://www.gao.gov/assets/gao-26-108837.pdf

75. Source: gao.gov
Link:https://www.gao.gov/assets/890/884854.pdf

76. Source: nti.org
Link:https://www.nti.org/wp-content/uploads/2025/11/25-027_NTI_NuclearSecurityImplicationofAIandEmergingTechSmall.pdf

77. Source: nti.org
Link:https://www.nti.org/wp-content/uploads/2025/11/NextGen-Redefining-Biological-Weapons-Final.pdf

78. Source: internationalaisafetyreport.org
Title: international ai safety report 2026
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/international-ai-safety-report-2026.pdf?pubDate=20260710

79. Source: fas.org
Title: March2026 AIxNuclear FAS
Link:https://fas.org/wp-content/uploads/2026/03/March2026_AIxNuclear_FAS.pdf

80. Source: fas.org
Title: June2025 AIxNC3 FAS
Link:https://fas.org/wp-content/uploads/2025/07/June2025_AIxNC3_FAS.pdf

81. Source: internationalaisafetyreport.org
Link:https://internationalaisafetyreport.org/publication/2026-report-extended-summary-policymakers

82. Source: policycommons.net
Title: international ai safety report 2026
Link:https://policycommons.net/artifacts/42998280/international-ai-safety-report-2026/43897332/

83. Source: internationalaisafetyreport.org
Link:https://internationalaisafetyreport.org/publication/second-key-update-technical-safeguards-and-risk-management

84. Source: atiro.turing.ac.uk
Title: International AI Safety Report 2025 Second
Link:https://atiro.turing.ac.uk/esploro/outputs/preprint/International-AI-Safety-Report-2025-Second/9922395209548

85. Source: internationalaisafetyreport.org
Title: international ai safety report 2026 web eng
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/international-ai-safety-report-2026-web-eng.pdf

86. Source: internationalaisafetyreport.org
Title: international ai safety report 2026 1
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/international-ai-safety-report-2026_1.pdf

87. Source: internationalaisafetyreport.org
Title: International AI Safety Report
Link:https://internationalaisafetyreport.org/

88. Source: internationalaisafetyreport.org
Title: rapport international sur la securite de l ia 2026
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/rapport-international-sur-la-securite-de-l-ia-2026.pdf

89. Source: internationalaisafetyreport.org
Title: INTERNATIONA L AI SAFETY REPORT
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/ai-safety-report-2026-extended-summary-for-policymakers.pdf

90. Source: internationalaisafetyreport.org
Link:https://internationalaisafetyreport.org/sites/default/files/2026-02/international-ai-safety-report-2026-executive-summary_1.pdf

Additional References

91. Source: youtube.com
Title: How The AI Fight Will Play Out | Dean Ball, author of America’s AI plan
Link:https://www.youtube.com/watch?v=ZBFG3WvweEM

92. Source: youtube.com
Title: Nick Bostrom: Worries About AI Existential Risk Just Became More Concrete
Link:https://www.youtube.com/watch?v=U_0aPqSAlgo

93. Source: nature.com
Link:https://www.nature.com/articles/s41746-024-01282-7

94. Source: nature.com
Link:https://www.nature.com/articles/s41591-024-03148-7

95. Source: nature.com
Link:https://www.nature.com/articles/s41746-024-01306-2

96. Source: nature.com
Link:https://www.nature.com/articles/s41591-024-03445-1

97. Source: youtube.com
Title: Superintelligence and human security, with Dan Hendrycks
Link:https://www.youtube.com/watch?v=ajjWrYn6-bM

98. Source: youtube.com
Title: AIx Bio: A Framework for Managed Access
Link:https://www.youtube.com/watch?v=AjlmIWnS9sM

99. Source: policycommons.net
Link:https://policycommons.net/artifacts/21432662/advancing-governance-at-the-nexus-of-artificial-intelligence-and-nuclear-weapons/22332920/view/?page=1

100. Source: securityandtechnology.org
Link:https://securityandtechnology.org/wp-content/uploads/2025/09/Artificial-Intelligence-in-Nuclear-Command-Control-Communications.pdf