Within Fast takeoff
When AI starts improving AI itself
The clearest early alarm may be AI systems making the next generation of AI substantially better before society notices disruption.
On this page
- What counts as AI assisted AI research
- Why successor model improvements matter
- Signals that would be early rather than too late
Page outline Jump by section
Introduction
For people worried about AI doom and rapid “FOOM” scenarios, one warning sign stands above most others: AI systems beginning to make the next generation of AI substantially better. The reason is simple. A fast takeoff does not require machines to redesign themselves overnight. It only requires a feedback loop in which increasingly capable AI systems become increasingly important contributors to AI research and development. If each generation helps create a stronger successor, progress could begin to compound. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente…
This idea remains controversial. No existing system has demonstrated the kind of runaway recursive self-improvement imagined in classic intelligence-explosion arguments. Yet AI is already being used to write code, optimise training systems, search for new algorithms, generate research ideas, and automate parts of scientific work that previously required human researchers. The question is not whether AI can assist AI research. It already does. The question is whether that assistance becomes strong enough that capability gains start feeding directly back into the process that creates future models. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente… [Sakana AI]sakana.aiai scientistThe AI Scientist: Towards Fully Automated Open-Ended…13 Aug 2024 — The AI Scientist is a fully automated pipeline for end-to-end paper…
What counts as AI-assisted AI research?
Many discussions of recursive self-improvement evoke science-fiction images of an AI rewriting its own source code. Most modern versions of the argument are less dramatic.
An AI system does not need complete autonomy to become an important part of the AI research pipeline. It only needs to contribute meaningfully to tasks that improve future systems. Examples include:
- Discovering more efficient neural-network architectures.
- Finding better training procedures.
- Optimising data selection and evaluation methods.
- Writing and debugging research code.
- Designing experiments and interpreting results.
- Generating novel algorithmic ideas for human researchers to test.
This distinction matters because many of these activities are already happening. Frontier AI systems are routinely used as coding assistants and research aides. The important threshold is not AI helping researchers work faster. The threshold is AI producing improvements that noticeably increase the capabilities of successor models. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente… [Sakana AI]sakana.aiai scientist first publicationThe AI Scientist Generates its First Peer-Reviewed…12 Mar 2025 — A paper produced by The AI Scientist-v2 passed the peer-review proces…
From a FOOM perspective, the crucial question is whether AI becomes a major source of AI progress rather than merely a productivity tool.
Why successor-model improvements matter
Recursive self-improvement is often described as a loop:
- An AI system helps improve AI research.
- Those improvements produce a more capable AI system.
- The more capable system contributes even more effectively to AI research.
- The cycle repeats.
If each round produces larger gains than the previous one, capability growth could accelerate.
Critically, doom-focused researchers do not necessarily expect a single model to rewrite itself continuously. A more plausible mechanism is a chain of successor models. One generation helps build the next, which then helps build another. The feedback loop operates across generations rather than entirely within a single system. [Axios]axios.comCo-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself…
This is why AI-driven AI research is often viewed as the earliest meaningful FOOM warning sign. Economic disruption, mass automation, or visible societal upheaval may occur later. The research loop begins much closer to the source of capability growth itself.
If a future model contributes enough to architecture design, optimisation, evaluation, and experimentation that researchers cannot easily separate human advances from machine-generated advances, many advocates of the intelligence-explosion hypothesis would regard that as a far more significant signal than most public-facing AI milestones.
The strongest evidence cited by FOOM advocates
Supporters of recursive-self-improvement concerns point to several developments that would have seemed highly speculative only a few years ago.
AI systems are beginning to automate larger parts of research
Sakana AI’s “AI Scientist” project demonstrated systems capable of generating research ideas, conducting experiments, writing papers, producing figures, and performing automated review. The project was explicitly presented as a step toward automating scientific discovery. Subsequent work reported AI-generated papers passing peer review in workshop settings and explored increasingly end-to-end research automation. [Sakana AI]sakana.aiai scientist natureThe AI Scientist: Towards Fully Automated AI Research…26 Mar 2026 — The ability to automate paper generation raises profound ethical a… [Sakana AI]pub.sakana.aiAI Scientist-v2: Workshop-Level Automated Scientific…8 Apr 2025 — The generation process for the workshop-accepted paper began with th…
These systems are far from replacing top human researchers. However, they illustrate a key trend: the automation frontier is moving from isolated tasks toward larger chunks of the research process itself.
AI is starting to discover algorithms
Google DeepMind’s AlphaEvolve was presented as a system for designing and improving algorithms. DeepMind reported applications ranging from data-centre efficiency improvements to AI-training optimisation and advances in algorithmic problems. Notably, some reported gains fed back into the infrastructure used to train advanced AI systems. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente… [2blog.google]blog.googlealphaevolve updatesAlphaEvolve, 1 year later: Impact on science, technology7 May 2026 — It's also accelerating scientific discovery, helping researchers run…
For FOOM advocates, this is significant because algorithmic improvements have historically been a major source of AI progress. A system that helps discover better algorithms is much closer to improving AI itself than a system that merely assists with ordinary software development.
Frontier labs are openly discussing recursive improvement
The idea is no longer confined to internet forums or speculative philosophy. Anthropic researchers have publicly discussed recursive self-improvement as a topic deserving serious study. Anthropic co-founder Jack Clark has argued that there is a substantial possibility that AI systems could autonomously build better successors within the next few years, and the company has identified AI accelerating AI development as an area requiring monitoring. [Axios]axios.comCo-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself… KuCoin The existence of these discussions does not prove a FOOM is coming. It does show that the possibility is being taken seriously by people work [kucoin.com]kucoin.comKuCoinAnthropic co-founder predicts AI R&D will become fully…5 May 2026 — Anthropic co-founder Jack Clark predicts that AI R&D could b… ing close to the frontier.
Why sceptics think this may not lead to FOOM
The strongest objections focus on the difference between research assistance and runaway acceleration.
Current AI systems still depend heavily on human judgement. They make mistakes, struggle with novelty, and often require extensive supervision. Independent evaluations of automated-research systems have found serious weaknesses, including flawed experiments, coding failures, poor literature review, hallucinated results, and limited adaptability. [arXiv]arxiv.orgAlphaEvolve: A coding agent for scientific and algorithmic…by A Novikov · 2025 · Cited by 526 — In this white paper, we present AlphaE…
Sceptics also argue that AI progress depends on many bottlenecks beyond algorithm design:
- Computing hardware.
- Energy supply.
- Data availability.
- Engineering integration.
- Validation and testing.
- Organisational decision-making.
Even if AI dramatically accelerated research output, these constraints could slow real-world capability growth.
Another objection is that research itself may not scale smoothly. Some discoveries are easy to automate; others may require deep human insight, physical experimentation, or conceptual breakthroughs that do not emerge from simply running more automated searches. The existence of AI-assisted research does not automatically imply exponential improvement. [arXiv]arxiv.orgAlphaEvolve: A coding agent for scientific and algorithmic…by A Novikov · 2025 · Cited by 526 — In this white paper, we present AlphaE…
As a result, many AI researchers expect substantial AI-driven research automation without expecting a classic intelligence explosion.
Signals that would be early rather than too late
The challenge with FOOM warning signs is timing. Many observable consequences may arrive only after the underlying process is already advanced.
People concerned about fast takeoff often focus on a narrower set of indicators:
AI-generated breakthroughs become routine. If frontier labs consistently report that major algorithmic advances originated from AI systems rather than human researchers, that would represent a substantial shift. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente…
Successor models are built largely using AI labour. A particularly important threshold would be evidence that much of the research effort behind a new frontier model came from earlier AI systems. [Axios]axios.comCo-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself…
Research timelines compress dramatically. If the interval between major capability jumps shrinks because AI systems are conducting more of the research cycle, that could indicate the beginning of positive feedback. [Axios]axios.comCo-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself…
Human understanding falls behind. Some doom arguments emphasise situations where AI-generated improvements work but are not fully understood by the researchers deploying them. If capability gains increasingly emerge from opaque machine-generated discoveries, oversight could become more difficult. [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente…
Automated AI research outperforms human-led teams. A world in which leading AI advances come primarily from automated systems rather than human experts would look qualitatively different from today’s research environment. [Sakana AI]sakana.aiai scientistThe AI Scientist: Towards Fully Automated Open-Ended…13 Aug 2024 — The AI Scientist is a fully automated pipeline for end-to-end paper…
Why this warning sign matters in AI doom arguments
Many proposed warning signs for AI doom involve visible changes in the economy, labour markets, politics, or military capabilities. AI-driven AI research is different because it sits directly at the mechanism that could produce rapid capability acceleration.
If recursive self-improvement is possible, the earliest detectable stage may not be social disruption but a research loop in which AI systems increasingly help create stronger successors. By the time broader effects become obvious, the underlying process could already be well underway.
That does not mean such a loop will inevitably produce a FOOM. The evidence today is consistent with multiple futures: gradual acceleration, powerful but controllable automation of research, or something much faster. The key point is narrower. Among the many candidate warning signs discussed in AI doom debates, AI improving AI is the one most closely connected to the mechanism that fast-takeoff theories actually depend upon. [Axios]axios.comCo-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself… [Google DeepMind]deepmind.googlealphaevolve a gemini powered coding agent for designing advanced algorithmsGoogle DeepMindAlphaEvolve: A Gemini-powered coding agent for…14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente…
Amazon book picks
Further Reading
Books and field guides related to When AI starts improving AI itself. Use these as the next step if you want deeper reading beyond the article.
Human Compatible
Examines risks that emerge as AI systems become increasingly capable and autonomous.
The Alignment Problem
Explains why increasingly capable AI systems may create new alignment challenges.
Superintelligence
Directly addresses feedback loops where advanced systems help create more capable successors.
Life 3.0
Includes discussion of superintelligence, rapid capability gains and transformative AI futures.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: deepmind.google
Title: alphaevolve a gemini powered coding agent for designing advanced algorithms
Link: https://deepmind.google/blog/alphaevolve-a-gemini-powered-coding-agent-for-designing-advanced-algorithms/Source snippet
Google DeepMindAlphaEvolve: A Gemini-powered coding agent for...14 May 2025 — AlphaEvolve enhanced the efficiency of Google's data cente...
Published: May 2025
-
Source: axios.com
Link: https://www.axios.com/2026/05/07/anthropic-jack-clark-ai-intelligence-explosionSource snippet
Co-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself...
-
Source: sakana.ai
Title: ai scientist
Link: https://sakana.ai/ai-scientist/Source snippet
The AI Scientist: Towards Fully Automated Open-Ended...13 Aug 2024 — The AI Scientist is a fully automated pipeline for end-to-end paper...
-
Source: sakana.ai
Title: ai scientist first publication
Link: https://sakana.ai/ai-scientist-first-publication/Source snippet
The AI Scientist Generates its First Peer-Reviewed...12 Mar 2025 — A paper produced by The AI Scientist-v2 passed the peer-review proces...
-
Source: blog.google
Title: alphaevolve updates
Link: https://blog.google/innovation-and-ai/infrastructure-and-cloud/google-cloud/alphaevolve-updates/Source snippet
AlphaEvolve, 1 year later: Impact on science, technology7 May 2026 — It's also accelerating scientific discovery, helping researchers run...
Published: May 2026
-
Source: arxiv.org
Link: https://arxiv.org/abs/2506.13131Source snippet
AlphaEvolve: A coding agent for scientific and algorithmic...by A Novikov · 2025 · Cited by 526 — In this white paper, we present AlphaE...
-
Source: kucoin.com
Link: https://www.kucoin.com/news/flash/anthropic-co-founder-predicts-ai-r-d-will-become-fully-automated-by-2028Source snippet
KuCoinAnthropic co-founder predicts AI R&D will become fully...5 May 2026 — Anthropic co-founder Jack Clark predicts that AI R&D could b...
Published: May 2026
-
Source: arxiv.org
Link: https://arxiv.org/abs/2502.14297Source snippet
arXivEvaluating Sakana's AI Scientist for Autonomous Research: Wishful Thinking or an Emerging Reality Towards '[Artificial]({{ 'artificial-goals/' | relative_url }}) Research Intel...
-
Source: sakana.ai
Title: ai scientist nature
Link: https://sakana.ai/ai-scientist-nature/Source snippet
The AI Scientist: Towards Fully Automated AI Research...26 Mar 2026 — The ability to automate paper generation raises profound ethical a...
-
Source: pub.sakana.ai
Link: https://pub.sakana.ai/ai-scientist-v2/paper/paper.pdfSource snippet
AI Scientist-v2: Workshop-Level Automated Scientific...8 Apr 2025 — The generation process for the workshop-accepted paper began with th...
-
Source: deepmind.google
Link: https://deepmind.google/blog/alphaevolve-impact/Source snippet
AlphaEvolve: Gemini-powered coding agent scaling impact...5 days ago — Discover how AlphaEvolve optimizes algorithms for genomics, quant...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2408.06292Source snippet
The AI Scientist: Towards Fully Automated Open-Ended...by C Lu · 2024 · Cited by 726 — This paper presents the first comprehensive frame...
-
Source: arxiv.org
Link: https://arxiv.org/abs/2504.08066Source snippet
The AI Scientist-v2: Workshop-Level Automated Scientific...by Y Yamada · 2025 · Cited by 124 — We introduce The AI Scientist-v2, an end...
-
Source: aimagazine.com
Title: google deepmind the uk the first automated ai science lab
Link: https://aimagazine.com/news/google-deepmind-the-uk-the-first-automated-ai-science-labSource snippet
Google DeepMind & The UK: The First Automated...11 Dec 2025 — Google Deepmind's AI for Science models to UK scientists, include: AlphaEv...
-
Source: medium.com
Link: https://medium.com/%40kram254/sakanaais-ai-scientist-is-revolutionizing-automated-research-scientific-discovery-b66dc0a1759bSource snippet
SakanaAI's AI Scientist is Revolutionizing Automated...A groundbreaking model that could redefine how we approach research across fields...
-
Source: techbrew.com
Title: Google Deep Mind’s new AI system can evolve
Link: https://www.techbrew.com/stories/2025/05/27/google-deepmind-ai-system-new-algorithmsSource snippet
Google DeepMind's new AI system can evolve...May 27, 2025 — Google has also designed a user interface for AlphaEvolve that will be avail...
Published: May 27, 2025
-
Source: linkedin.com
Link: https://www.linkedin.com/pulse/alphaevolve-google-deepminds-ai-redefines-algorithm-dinesh-abeysinghe-onphcSource snippet
Google DeepMind's AI That Redefines Algorithm DiscoveryGoogle DeepMind has unveiled AlphaEvolve, an AI system that autonomously discovers...
Additional References
-
Source: medium.com
Link: https://medium.com/%40Micheal-Lanham/alphaevolve-how-deepminds-ai-is-rewriting-the-rules-of-algorithm-discovery-09b9e69e6ae7Source snippet
AlphaEvolve: How DeepMind's AI is Rewriting the Rules of...Google DeepMind's AlphaEvolve is revolutionizing how we discover algorithms b...
-
Source: isg.beel.org
Link: https://isg.beel.org/pubs/2025-sakana-ai-scientist-reproduced.pdfSource snippet
AI Scientist is supposed to “automate the entire research lifecycle”; i.e. it generates research ideas, designs and conducts experiments...
-
Source: techradar.com
Link: https://www.techradar.com/ai-platforms-assistants/you-would-be-able-to-say-to-it-make-a-better-version-of-yourself-and-it-just-goes-off-and-does-that-completely-autonomously-anthropic-co-founder-on-our-wild-recursive-ai-futureSource snippet
Co-founder Jack Clark predicts that by 2028, such self-improving AI could become a reality. This development raises urgent ethical and sa...
-
Source: github.com
Link: https://github.com/SakanaAI/AI-ScientistSource snippet
The AI Scientist: Towards Fully Automated Open-Ended...We're excited to introduce The AI Scientist, the first comprehensive system for f...
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/ankaj-mohindroo-b9948035_ai-enterpriseai-gaiinsights-activity-7457455314263744513-iehSSource snippet
AI Systems to Build Themselves by 2028: Jack ClarkJack Clark, co-founder of Anthropic, just made a striking claim: AI systems are about t...
-
Source: researchgate.net
Link: https://www.researchgate.net/publication/383060918_The_AI_Scientist_Towards_Fully_Automated_Open-Ended_Scientific_DiscoverySource snippet
(PDF) The AI Scientist: Towards Fully Automated Open-...13 Aug 2024 — This paper presents the first comprehensive framework for fully au...
-
Source: research.google
Title: ai as a research partner advancing theoretical computer science with alphaevolve
Link: https://research.google/blog/ai-as-a-research-partner-advancing-theoretical-computer-science-with-alphaevolve/Source snippet
Advancing theoretical computer science with AlphaEvolveAI as a research partner: Advancing theoretical computer science with AlphaEvolve...
-
Source: medium.com
Link: https://medium.com/%402019be04004/alphaevolve-the-ai-coding-agent-redefining-algorithm-design-0fb4b346e1faSource snippet
AlphaEvolve: The AI Coding Agent Redefining Algorithm...Announced on May 14, 2025, AlphaEvolve is a game-changer, blending the creative...
Published: May 14, 2025
-
Source: mindstudio.ai
Title: jack clark anthropic 60 percent recursive self improvement 2028
Link: https://www.mindstudio.ai/blog/jack-clark-anthropic-60-percent-recursive-self-improvement-2028/Source snippet
60% Chance of Recursive AI Self-Improvement by 202810 May 2026 — Anthropic co-founder Jack Clark publicly put 60% odds on recursive AI se...
Published: May 2026
-
Source: github.com
Link: https://github.com/sakanaai/ai-scientist-v2Source snippet
e, with AI playing a pivotal role in transforming how scientific discoveries are...Read more...
Topic Tree