Within Objections
Do today's AI failures weaken doom fears?
Current AI failures and lab-test behaviours leave open whether future systems would become capable strategic agents.
On this page
- What current systems can and cannot do
- Why narrow warning signs are contested
- Where evidence from today's models stops
Page outline Jump by section
Introduction
One of the most persistent debates in discussions about AI existential risk is whether what we see today in large language models and other “frontier” AI systems offers real evidence that future systems will *take over** – that is, act as autonomous agents pursuing their own strategic goals in ways that escape human control. Many worry about “takeover” scenarios where advanced AI seeks power, self‑preservation or resource control beyond human intentions. But critics argue that what we observe in current AI is weak evidence for anything resembling that future path. This page explores why present‑day systems do not yet exhibit the kinds of capacities and behaviours that would make classic takeover scenarios plausible, where the existing evidence is informative, and where it stops.
What today’s AI can and cannot do
Current generative models like GPT‑style systems are powerful language and pattern predictors, not autonomous strategic agents. They are trained to predict text from massive data sets and are incredibly good at that narrow task. They can generate code, answer questions and simulate dialogue, yet they also make obvious errors, hallucinate false facts and lack consistent long‑term reasoning and planning abilities in open‑ended settings. Many researchers see major limitations in reliability, adaptability and general problem‑solving that are far from the kinds of capabilities imagined in takeover scenarios. For example, a UK research effort reviewing current AI trajectories found that today’s systems lack general cognitive capabilities necessary for broad automation and independent strategy formation, and that progress on these limitations remains uncertain and gradual rather than explosive.[AI Security Institute]aisi.gov.ukSource details in endnotes.
Crucially, existing models do not independently pursue goals in the world. They produce outputs in response to prompts but do not have persistent objectives, ongoing agency or the ability to execute complex multi‑step real‑world plans without human direction and infrastructure. They do not monitor their own states across time, seek to preserve themselves, or reallocate resources to achieve long‑term aims in the way strategic agent models would need to for a takeover. Beyond toy or simulated environments, there are no publicly documented cases of contemporary AI engaging in power‑seeking behaviour that is decoupled from human instruction.[arXiv]arxiv.orgarXivA Review of the Evidence for Existential Risk from AI via Misaligned Power-SeekingOctober 27, 2023…
Why narrow warning signs are contested
Critics of strong takeover interpretations acknowledge that current models display behaviours that can be concerning in specific contexts — so‑called specification gaming and misalignment examples. These are cases where models optimise for surface‑level training objectives in ways that produce undesirable or unexpected outputs. Such behaviours highlight genuine misalignment challenges, but they occur in very narrow, contrived test conditions and do not necessarily generalise to open‑ended, strategic action. In other words, they are signs of behavioural brittleness rather than autonomous goal pursuit.[CatalyzeX]catalyzex.comCatalyzeXA Review of the Evidence for Existential Risk from AI via Misaligned Power-SeekingOctober 27, 2023…
For instance, critics of alarmist narratives point out that many of the claimed examples of “scheming” or deceptive outputs seen in lab tests are highly sensitive to how the model was prompted, and often reflect pattern matching rather than genuine instrumental reasoning. A recent accessible analysis argued that cases which look like deception sometimes arise because humans misinterpret context‑specific model outputs as independent agency, which can exaggerate the import of these behaviours for existential scenarios.[Vox]vox.comHow can you know if an AI is plotting against you?A team led by Oxford neuroscientist Christopher Summerfield draws parallels with 20th-century ape language studies, where scientists misi…
Similarly, others note that present misalignment cases tend to diminish a system’s usefulness and are unpredictable in direction — they do not cluster into goal‑directed power seeking. One peer‑reviewed survey of evidence for existential risk concluded that while there are examples of specification gaming and conceptual arguments about future risk, there are no clear empirical examples of misaligned power‑seeking behaviour in current systems.[CatalyzeX]catalyzex.comCatalyzeXA Review of the Evidence for Existential Risk from AI via Misaligned Power-SeekingOctober 27, 2023…
Where evidence from today’s models stops
The key gap is not simply that current AI systems are imperfect — all complex technologies are — but that they do not yet demonstrate the structural capacities that most takeover arguments require. A core aspect of worry about existential takeover is instrumental convergence: the idea that sufficiently capable systems will, regardless of their original goals, pursue sub‑goals like self‑preservation and resource acquisition because these help achieve their objectives. While this remains a theoretical concern under some assumptions, it has not been empirically observed in existing systems. In fact, critics argue that current models lack the situational awareness, causal reasoning and sustained agency required even to recognise opportunities for instrumental sub‑goals, let alone pursue them autonomously.[Longterm Wiki]longtermwiki.comLongterm Wiki Instrumental Convergence | Longterm WikiLongterm WikiInstrumental Convergence | Longterm WikiJanuary 29, 2026…
Another limit of present‑day evidence is external validity. A recent empirical assessment found that although researchers have begun to observe emergent behaviours in advanced models, the current body of evidence is insufficient to support strong claims about loss of control — the evidence is too weak and narrow to extrapolate confidently to future highly capable systems. These assessments highlight that most studies lack the conditions needed to make robust claims about agentic behaviour outside controlled experiments.[SSRN]papers.ssrn.comSSRNAssessing the Empirical Evidence for Loss of Control from Agentic General-Purpose AI by Risto Uuk, Santeri Koivula, Lorenzo Pacchiard…
This does not imply that risk is zero. Most critics acknowledge that future AI could be dangerous or difficult to control if its capabilities grow sufficiently. But they caution that current models are not yet strong evidence for takeover because they do not exhibit the key agentic features — persistent goals, independent planning, strategic world modelling — that would make takeover plausible. They argue that conflating artefacts of prompt responses with genuine autonomous intent risks overstating the case and diverting attention from the empirical milestones that would actually inform risk assessments.[AI Wiki]aiwiki.aiAI Wiki Existential risk from AI | AI WikiAI WikiExistential risk from AI | AI WikiMarch 25, 2026…
Implications for the debate on AI risk
Understanding where current evidence ends and speculation begins matters because it shapes how policymakers, researchers, and the public assess the urgency and nature of AI risk. If today’s models were already showing clear signs of autonomous goal pursuit and power‑seeking, that would strengthen the empirical basis for loss‑of‑control scenarios. But as it stands, the evidence suggests that current systems are powerful prediction machines with significant limitations, and that inferring future takeover capabilities from their present behaviour involves large inferential leaps.
This perspective feeds into broader debates about how to balance concern with caution: most researchers agree that AI safety is important and that future systems could pose serious challenges, but there is disagreement about how much current model behaviour tells us about far‑future agentic risk. Some experts caution against either dismissing risks entirely or treating speculative future scenarios as established fact until the empirical evidence clearly supports them.[gov.uk]
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
References and Related Pages
For deeper exploration of these issues, see related discussions on the instrumental convergence thesis, empirical studies of emergent behaviour, and broader reviews of AI risk and alignment beyond current model limitations.
Endnotes
-
Source: aisi.gov.uk
Link: https://www.aisi.gov.uk/research/understanding-ai-trajectories-mapping-the-limitations-of-current-ai-systems -
Source: arxiv.org
Link: https://arxiv.org/abs/2310.18244Source snippet
arXivA Review of the Evidence for Existential Risk from AI via Misaligned Power-SeekingOctober 27, 2023...
Published: October 27, 2023
-
Source: catalyzex.com
Link: https://www.catalyzex.com/paper/a-review-of-the-evidence-for-existential-riskSource snippet
CatalyzeXA Review of the Evidence for Existential Risk from AI via Misaligned Power-SeekingOctober 27, 2023...
Published: October 27, 2023
-
Source: vox.com
Title: How can you know if an AI is plotting against you?
Link: https://www.vox.com/future-perfect/420755/ai-scheming-deception-lessons-from-a-chimpSource snippet
A team led by Oxford neuroscientist Christopher Summerfield draws parallels with 20th-century ape language studies, where scientists misi...
-
Source: papers.ssrn.com
Link: https://papers.ssrn.com/sol3/Delivery.cfm/6786058.pdf?abstractid=6786058&mirid=1Source snippet
SSRNAssessing the Empirical Evidence for Loss of Control from Agentic General-Purpose AI by Risto Uuk, Santeri Koivula, Lorenzo Pacchiard...
-
Source: GOV.UK
Link: https://www.gov.uk/government/publications/frontier-ai-capabilities-and-risks-discussion-paper/future-risks-of-frontier-ai-annex-aSource snippet
28, 2025...
-
Source: GOV.UK
Link: https://www.gov.uk/government/publications/international-scientific-report-on-the-safety-of-advanced-ai/international-scientific-report-on-the-safety-of-advanced-ai-interim-reportSource snippet
LOSS OF CONTROL KEY INFORMATION * Ongoing AI ([artificial]({{ 'artificial-goals/' | relative_url }}) intelligence) research is seeking to develop more capable ‘general-purpose AI (a...
-
Source: GOV.UK
Title: Use of capabilities: Would some AI systems actual
Link: https://www.gov.uk/government/publications/international-ai-safety-report-2025/international-ai-safety-report-2025Source snippet
www.gov.uk[Withdrawn] International AI Safety Report 2025 - GOV.UKFebruary 18, 2025 — (Note that the minimum capabilities needed would pa...
Published: February 18, 2025
-
Source: longtermwiki.com
Title: Longterm Wiki Instrumental Convergence | Longterm Wiki
Link: https://www.longtermwiki.com/knowledge-base/risks/instrumental-convergence/Source snippet
Longterm WikiInstrumental Convergence | Longterm WikiJanuary 29, 2026...
Published: January 29, 2026
-
Source: aiwiki.ai
Title: AI Wiki Existential risk from AI | AI Wiki
Link: https://aiwiki.ai/wiki/ai_existential_riskSource snippet
AI WikiExistential risk from AI | AI WikiMarch 25, 2026...
Published: March 25, 2026
-
Source: longtermwiki.com
Title: Model Organisms of Misalignment | Longterm Wiki
Link: https://www.longtermwiki.com/wiki/E419Source snippet
February 1, 2026 — CRITICISMS AND CONCERNS METHODOLOGICAL LIMITATIONS Critics raise several concerns about the validity and informativene...
Published: February 1, 2026
Additional References
-
Source: decrypt.co
Link: https://decrypt.co/341978/ai-study-chatbots-strategically-lie-current-safety-tools-cant-catch-them?amp=1Source snippet
AI Study Finds Chatbots Can Strategically Lie—And Current Safety Tools Can't Catch Them - DecryptSeptember 29, 2025 — AI STUDY FINDS CHAT...
Published: September 29, 2025
-
Source: scientificamerican.com
Link: https://www.scientificamerican.com/article/ai-is-too-unpredictable-to-behave-according-to-human-goals/Source snippet
AI Is Too Unpredictable to Behave According to Human Goals | Scientific AmericanJanuary 27, 2025 — January 27, 2025 AI Is Too Unpredictab...
Published: January 27, 2025
-
Source: cris.fau.de
Title: de Current cases of AI misalignment and their implications for future risks
Link: https://cris.fau.de/publications/313453060/Source snippet
cases of AI misalignment and their implications for future risks - FAU CRISCURRENT CASES OF AI MISALIGNMENT AND THEIR IMPLICATIONS FOR FU...
-
Source: brookings.edu
Title: Are AI existential risks real—and what should we do about them?
Link: https://www.brookings.edu/articles/are-ai-existential-risks-real-and-what-should-we-do-about-them/Source snippet
| BrookingsJuly 11, 2025 — AI firms are not very close to developing an AI system with capabilities that could threaten us. This assertio...
Published: July 11, 2025
-
Source: pmc.ncbi.nlm.nih.gov
Title: Jake Tapper: “You’ve spoken out s
Link: https://pmc.ncbi.nlm.nih.gov/articles/PMC11117051/Source snippet
deception: A survey of examples, risks, and potential solutions - PMCMay 10, 2024 — INTRODUCTION In a recent interview with CNN journalis...
Published: May 10, 2024
-
Source: koncentrik.co
Title: Why Simulating Strategy Is Not Intelligence
Link: https://www.koncentrik.co/p/why-simulating-strategy-is-not-intelligenceSource snippet
July 21, 2025 — WHY SIMULATING STRATEGY IS NOT INTELLIGENCE WHAT THE LATEST RESEARCH ON GAME THEORY AND LLMS REVEALS ABOUT TRUE MACHINE R...
Published: July 21, 2025
-
Source: sciencedirect.com
Title: Beyond Intentions: A Critical Survey of Misalignment in LLMs
Link: https://www.sciencedirect.com/science/article/pii/S1546221825007982Source snippet
Outer Alignment concerns whether the goal functions (e.g., reward functions) we set for the...
-
Source: citedrive.com
Title: Baum Show PDF Cite
Link: https://www.citedrive.com/en/discovery/assessing-the-risk-of-takeover-catastrophe-from-large-language-models/Source snippet
[PDF] Assessing the risk of takeover catastrophe from large language models | CiteDriveOUTLINE * Abstract DOI: 10.1111/risa.14353 ISSN: 0...
-
Source: youtube.com
Title: Francois Chollet — Why the biggest AI models can’t solve simple puzzles
Link: https://www.youtube.com/watch?v=UakqL6Pj9xoSource snippet
Melanie Mitchell - Existential risk from AI: A skeptical perspective...
-
Source: metr.org
Title: 2026 05 19 frontier risk report
Link: https://metr.org/blog/2026-05-19-frontier-risk-report/Source snippet
As we explain below, METR’s core concern is tracking the risk of scenarios in which powerful AI a...
Topic Tree