Within AI Takeoff
Could AI train the next AI itself?
The strongest intelligence explosion claim depends on whether future systems can plan, run, and verify the training of better models without humans.
On this page
- What autonomous successor training would require
- Why some researchers see 2028 as a warning window
- Where human bottlenecks may still break the loop
Page outline Jump by section
Introduction
Could an AI really train the next AI itself? In a limited sense, parts of this are already happening. AI systems are increasingly used to write code, generate training data, evaluate model outputs, tune systems, and assist researchers building newer models. The more important question for AI doom and intelligence-explosion debates is whether future systems could perform most or all of the work needed to create a more capable successor without meaningful human involvement.
This matters because many arguments about recursive AI improvement depend on such a loop existing. If an AI can substantially automate AI research and development, each generation of systems could help produce the next generation faster. If crucial human bottlenecks remain, the feedback loop may be much weaker than intelligence-explosion scenarios assume. The debate is therefore not whether AI can contribute to successor training—it already can—but whether it can eventually replace the researchers, engineers, evaluators, and decision-makers who currently drive frontier AI progress. [Import AI]jack-clark.netImport AIImport AI 455: Automating AI ResearchMay 4, 2026 — 4 May 2026 — As of March 2026, AI systems are able to post-train models to get about half as much of the uplift as ones tra… [Axios]axios.comThis self-learning approach is attracting significant attention from both top AI labs like Google DeepMind and startups, as it promises t…
What autonomous successor training would require
Training a frontier AI model is not one task. It is a long chain of activities involving research, engineering, testing, and deployment. For an AI to train its own successor in the sense relevant to recursive improvement, it would need to automate most of this chain.
A simplified version looks like this:
- Identify weaknesses in the current system.
- Generate new architectural ideas, algorithms, or training methods.
- Write and debug the necessary software.
- Design and run large-scale experiments.
- Interpret experimental results.
- Select promising approaches.
- Prepare and curate training data.
- Evaluate safety and capability changes.
- Launch the next training run.
- Verify that the resulting model is genuinely better.
Many of these tasks are already being partially automated. Frontier models can write substantial amounts of code, conduct limited research tasks, analyse experimental outputs, and assist with model evaluation. Researchers increasingly use AI systems as collaborators in AI development itself. [Import AI]jack-clark.netImport AIImport AI 455: Automating AI ResearchMay 4, 2026 — 4 May 2026 — As of March 2026, AI systems are able to post-train models to get about half as much of the uplift as ones tra… [LinkedIn However]linkedin.comThe 2026 State of AI Agents Report by Anthropic ClaudeCode agents handle multi-step development workflows, not just code completion: plan…, there is a large difference between helping with individual tasks and autonomously managing an entire research programme. Today’s systems often perform well on bounded, verifiable work but struggle with long-horizon planning, ambiguous objectives, novel scientific reasoning, and coordination across many moving parts. [Business Insider]businessinsider.comBusiness Insider Why AI hasn't replaced every 'automatable' jobAccording to Benjamin Todd, president of 80,000 Hours, the reason lies in AI's current limitations—it often automates only parts of a job…
Why some researchers see 2028 as a warning window
One reason successor training has become a serious topic is that several forecasting efforts now focus specifically on AI’s ability to automate AI research.
In 2026, Anthropic co-founder Jack Clark argued that there may be better than even odds that AI R&D becomes largely automated by the end of 2028. His argument was not that machines suddenly become magical scientists. Rather, he pointed to steady improvements in coding, experiment execution, model tuning, and other research tasks that collectively make up much of frontier AI development. [KuCoin]kucoin.comKuCoinAnthropic co-founder predicts AI R&D will become fully…5 days ago — Anthropic co-founder Jack Clark predicts that AI R&D could b… [Scott Loftesness]sjl.usSource details in endnotes.
Related forecasting work from the research organisation METR has focused on when AI systems might automate AI development itself, treating this as a key milestone because it could create a positive feedback loop in capability growth. [Metr]metr.orgIt is based on the AI Futures model,MetrA simpler AI timelines model predicts 99% AI R&D…February 10, 2026 — 10 Feb 2026 — In this post, I describe a simple model for for…
The underlying reasoning is straightforward. If AI systems become capable enough to perform substantial fractions of AI research:
- More research can be conducted simultaneously.
- Experiments can run continuously.
- Engineering bottlenecks shrink.
- New models can be developed faster.
- Progress may increasingly depend on machine labour rather than scarce human expertise.
In that world, the pace of capability gains could accelerate even if no dramatic scientific breakthrough occurs. Several researchers interviewed in a 2025–26 study identified automation of AI research as one of the most important pathways through which recursive improvement could emerge. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions
Importantly, these forecasts remain predictions rather than observations. No current frontier model can independently run a leading AI laboratory from start to finish.
The strongest evidence that the loop could exist
Supporters of recursive-improvement concerns point to several developments.
First, AI performance on software engineering tasks has improved rapidly. Benchmarks that were once difficult for frontier systems have seen dramatic gains in only a few years. Since AI development is itself heavily dependent on software engineering, progress in coding directly affects the possibility of automating AI research. [importai.substack.com]importai.substack.comA I systems are about to start building themselvesImport AI 455May 4, 2026 — Solving real-world software engineering problems: SWE-Bench is a widely used coding test which evaluates how w…
Second, AI-generated data is already used to improve future models. Variants of what is sometimes called the “Karpathy loop” involve models generating outputs, stronger models filtering or evaluating those outputs, and the resulting material being used to train later systems. This is not full autonomous successor training, but it is a primitive form of AI contributing to the creation of future AI. [mindstudio.ai]mindstudio.aiWhat the Karpathy Loop Means for AI BuildersMay 26, 2026 — 6 days ago — The Karpathy Loop is a recursive model improvement cycle in which AI models generate training outputs, a stro…
Third, major labs increasingly use AI tools internally to accelerate development work. Some observers argue that the industry is gradually moving from “AI-assisted researchers” toward “researchers supervising AI researchers”. If that trend continues, the amount of human labour required per capability advance could decline substantially. [Scott Loftesness]sjl.usSource details in endnotes. [Anthropic]anthropic.cominstitute agendaAnthropicFocus areas for The Anthropic Institute7 May 2026 — Our agenda focuses on four areas for research: Economic diffusion; Threats a…
For doom-focused thinkers, the concern is not that one model instantly creates a vastly smarter successor. It is that many small automation gains accumulate until most of the research pipeline can be executed by machines.
Where human bottlenecks may still break the loop
The strongest objections focus on the difficulty of replacing humans completely.
New ideas are not the same as optimisation
Many improvements in machine learning come from experimentation and engineering. But some advances depend on genuinely new concepts, unexpected insights, or shifts in research direction.
Sceptics argue that AI systems may become extremely useful research assistants while still failing to generate the kinds of breakthroughs that produce major jumps in capability. If so, they may accelerate research without creating a runaway self-improvement cycle. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions
Compute remains a physical constraint
Even if an AI could design a better successor, training that successor requires enormous computing infrastructure.
Data centres, specialised chips, electricity supplies, networking equipment, and capital investment remain physical resources that cannot be conjured by software alone. Several researchers argue that compute constraints could significantly slow any recursive-improvement process. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions
This does not eliminate risk, but it suggests that capability growth may be limited by economics and infrastructure rather than pure intelligence.
Verification may be harder than generation
Building a candidate successor model is one challenge. Verifying that it is genuinely better, safer, and more reliable is another.
As models become more capable, evaluation itself becomes difficult. Researchers already report challenges in measuring advanced systems because existing benchmarks are increasingly saturated or fail to capture real-world performance. A future AI might propose improvements faster than humans can confidently assess them. [LinkedIn]linkedin.comThe 2026 State of AI Agents Report by Anthropic ClaudeCode agents handle multi-step development workflows, not just code completion: plan…
Paradoxically, this could either slow recursive improvement or make it harder to control.
Alignment research may not automate easily
Some AI safety proposals assume that advanced AI systems will help solve alignment—the challenge of ensuring powerful systems pursue intended goals.
Critics note that automating alignment research may be significantly harder than automating ordinary engineering work. If the hardest remaining problems are precisely the ones involving safety, oversight, and control, then successor training may advance faster than humanity’s ability to verify that the resulting systems remain aligned. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions
What this means for AI doom arguments
Successor training occupies a special place in AI doom discussions because it is one of the clearest mechanisms by which capabilities could accelerate unexpectedly.
The strongest doom arguments do not require an AI to rewrite its own source code overnight or instantly become superintelligent. They require something more modest but potentially more plausible: AI systems gradually automating larger fractions of AI development until progress becomes increasingly machine-driven rather than human-driven. At that point, each generation of systems could help create the next generation faster than before. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions Axios Yet the evidence remains incomplete. No current system can independently run frontier AI research. Researchers disagree sharply about timelin [axios.com]axios.comThis self-learning approach is attracting significant attention from both top AI labs like Google DeepMind and startups, as it promises t… es, about whether automation of AI R&D would produce explosive growth, and about how much physical and organisational bottlenecks would slow the process. Interviews with leading researchers reveal substantial disagreement on exactly these questions. [arXiv]arxiv.orgarXiv AI Researchers' Views on Automating AI R&D and Intelligence ExplosionsarXiv AI Researchers' Views on Automating AI R&D and Intelligence Explosions
The key warning sign, therefore, is not whether an AI can assist with coding or training. Those capabilities already exist. The more significant indicator would be AI systems reliably planning, executing, evaluating, and improving large-scale AI research projects with minimal human intervention. If that threshold is crossed, concerns about recursive improvement and intelligence explosion move from largely theoretical arguments to practical questions about how quickly the feedback loop can run and whether humans remain able to steer it. [Import AI]jack-clark.netImport AIImport AI 455: Automating AI ResearchMay 4, 2026 — 4 May 2026 — As of March 2026, AI systems are able to post-train models to get about half as much of the uplift as ones tra… [Metr]metr.orgIt is based on the AI Futures model,MetrA simpler AI timelines model predicts 99% AI R&D…February 10, 2026 — 10 Feb 2026 — In this post, I describe a simple model for for…
Amazon book picks
Further Reading
Books and field guides related to Could AI train the next AI itself?. Use these as the next step if you want deeper reading beyond the article.
Human Compatible
Examines why advanced autonomous systems may be difficult to control.
The Singularity is Near
Popularizes recursive improvement and intelligence-explosion themes.
eBay marketplace picks
Marketplace Samples
Example marketplace items related to this page. Use the search link to explore similar finds on eBay.
Endnotes
-
Source: jack-clark.net
Title: Import AIImport AI 455: Automating AI Research
Link: https://jack-clark.net/2026/05/04/import-ai-455-automating-ai-research/Source snippet
May 4, 2026 — 4 May 2026 — As of March 2026, AI systems are able to post-train models to get about half as much of the uplift as ones tra...
Published: May 4, 2026
-
Source: axios.com
Link: https://www.axios.com/2026/01/27/models-improve-aiSource snippet
This self-learning approach is attracting significant attention from both top AI labs like Google DeepMind and startups, as it promises t...
-
Source: arxiv.org
Title: arXiv AI Researchers’ Views on Automating AI R&D and Intelligence Explosions
Link: https://arxiv.org/abs/2603.03338 -
Source: linkedin.com
Link: https://www.linkedin.com/posts/fciucci_the-2026-state-of-ai-agents-report-by-anthropic-activity-7414680232613527552-K3CNSource snippet
The 2026 State of AI Agents Report by Anthropic ClaudeCode agents handle multi-step development workflows, not just code completion: plan...
-
Source: kucoin.com
Link: https://www.kucoin.com/news/flash/anthropic-co-founder-predicts-ai-r-d-will-become-fully-automated-by-2028Source snippet
KuCoinAnthropic co-founder predicts AI R&D will become fully...5 days ago — Anthropic co-founder Jack Clark predicts that AI R&D could b...
-
Source: axios.com
Link: https://www.axios.com/2026/05/07/anthropic-jack-clark-ai-intelligence-explosionSource snippet
Co-founder Jack Clark predicts a greater than 60% chance that by 2028, an AI system could autonomously build a better version of itself...
-
Source: metr.org
Title: It is based on the AI Futures model,
Link: https://metr.org/notes/2026-02-10-simpler-ai-timelines-model/Source snippet
MetrA simpler [AI timelines]({{ 'timeline-effects/' | relative_url }}) model predicts 99% AI R&D...February 10, 2026 — 10 Feb 2026 — In this post, I describe a simple model for for...
Published: February 10, 2026
-
Source: importai.substack.com
Title: A I systems are about to start building themselves
Link: https://importai.substack.com/p/import-ai-455-automating-ai-researchSource snippet
Import AI 455May 4, 2026 — Solving real-world software engineering problems: SWE-Bench is a widely used coding test which evaluates how w...
Published: May 4, 2026
-
Source: mindstudio.ai
Title: What the Karpathy Loop Means for AI Builders
Link: https://www.mindstudio.ai/blog/andrej-karpathy-joins-anthropic-karpathy-loop-explained/Source snippet
May 26, 2026 — 6 days ago — The Karpathy Loop is a recursive model improvement cycle in which AI models generate training outputs, a stro...
Published: May 26, 2026
-
Source: anthropic.com
Title: institute agenda
Link: https://www.anthropic.com/research/anthropic-institute-agenda?via=joinSource snippet
AnthropicFocus areas for The Anthropic Institute7 May 2026 — Our agenda focuses on four areas for research: Economic diffusion; Threats a...
Published: May 2026
-
Source: arxiv.org
Title: arXiv Will Compute Bottlenecks Prevent an Intelligence Explosion?
Link: https://arxiv.org/abs/2507.23181Source snippet
arXivWill Compute Bottlenecks Prevent an Intelligence Explosion?July 31, 2025...
Published: July 31, 2025
-
Source: arxiv.org
Link: https://arxiv.org/abs/2511.10668 -
Source: linkedin.com
Link: https://www.linkedin.com/posts/andrew-hall-40a88444_ai-models-have-gotten-so-good-that-we-no-activity-7423413030778118144-WK2mSource snippet
LinkedInAI Models' Recursive Improvement: Challenges and...January 31, 2026 — AI models have gotten so good that we no longer know how t...
Published: January 31, 2026
-
Source: arxiv.org
Link: https://arxiv.org/html/2605.06390v3Source snippet
arXivAutomated Alignment is Harder Than You Think14 May 2026 — A leading proposal for aligning [artificial]({{ 'artificial-goals/' | relative_url }}) superintelligence (ASI) is to u...
Published: May 2026
-
Source: anthropic.com
Link: https://www.anthropic.com/research -
Source: linkedin.com
Link: https://www.linkedin.com/posts/sekoul_theres-a-pretty-clean-way-to-track-how-capable-activity-7457836671393439745-gC_OSource snippet
AI Autonomy Grows 10x Per Year, Successor Training...Which leads to his core claim: there's a 60%+ chance that by end of 2028, an AI sys...
-
Source: linkedin.com
Link: https://www.linkedin.com/posts/matthewsavarick_the-ceo-of-anthropic-just-said-the-quiet-activity-7429530623226052609-4n2WSource snippet
Dario Amodei predicts AI surpassing humans by 2026-2027believes AI systems smarter than the best humans in almost everything could arrive...
-
Source: anthropic.com
Link: https://www.anthropic.com/ -
Source: anthropic.com
Link: https://www.anthropic.com/economic-indexSource snippet
Anthropic Economic Index: Understanding AI's effects...Mar 24, 2026 — The Anthropic Economic Index reveals the shape of AI adoption acro...
-
Source: mindstudio.ai
Title: What Is Recursive Self-Improvement in AI?
Link: https://www.mindstudio.ai/blog/what-is-recursive-self-improvement-ai-intelligence-explosion/Source snippet
The Intelligence...13 May 2026 — Recursive self-improvement is when AI builds its own successor without human input. Learn what it means...
Published: May 2026
-
Source: intelligence.org
Title: thoughts on ai 2027
Link: https://intelligence.org/2025/04/09/thoughts-on-ai-2027/Source snippet
Apr 9, 2025 — One of the core things I think AI 2027 does right is put their emphasis on recursive self-improvement (RSI). I see a lot of...
-
Source: businessinsider.com
Title: Business Insider Why AI hasn’t replaced every ‘automatable’ job
Link: https://www.businessinsider.com/why-ai-hasnt-replaced-every-automatable-job-yet-2026-5Source snippet
According to Benjamin Todd, president of 80,000 Hours, the reason lies in AI's current limitations—it often automates only parts of a job...
-
Source: sjl.us
Link: https://sjl.us/2026/05/08/breakout/ -
Source: Wikipedia
Link: https://en.wikipedia.org/wiki/AnthropicSource snippet
AnthropicAnthropic is an American artificial intelligence (AI) company headquartered in San Francisco. It has developed a range of lar...
-
Source: businessinsider.com
Title: openai safety team ai self improvement challenge job 2026 5
Link: https://www.businessinsider.com/openai-safety-team-ai-self-improvement-challenge-job-2026-5Source snippet
OpenAI Hires in Preparation for AI That Could Train Itself23 May 2026 — OpenAI and Sam Altman aim to automate AI research. They are now h...
Published: May 2026
-
Source: instagram.com
Title: Anthropic just put a number on the intelligence explosion
Link: https://www.instagram.com/p/DYyuuKWRKAm/Source snippet
He warned of a 60% chance an AI will fully train its successor by 2028, leading to a possible "intelligence explosion." Clark also mentio...
-
Source: anthropic.skilljar.com
Link: https://anthropic.skilljar.com/Source snippet
CoursesThis course empowers students to develop AI Fluency skills that enhance learning, career planning, and academic success through re...
-
Source: forbes.com
Link: https://www.forbes.com/sites/anishasircar/2026/01/28/anthropic-ceo-warns-superhuman-ai-could-arrive-by-2027-with-civilization-level-risks/Source snippet
Anthropic CEO Warns Superhuman AI Could Arrive By...Jan 28, 2026 — Dario Amodei warns that superhuman AI arriving within two years could...
-
Source: facebook.com
Link: https://www.facebook.com/pivotwealth/posts/anthropic-just-accidentally-leaked-their-most-advanced-ai-model-and-it-makes-the/1555832719883872/Source snippet
the current one look outdated. The same [current model]({{ 'current-models/' | relative_url }}) that already...
-
Source: the-decoder.com
Link: https://the-decoder.com/anthropic-co-founder-maps-out-how-recursive-ai-improvement-could-outpace-the-humans-meant-to-supervise-it/Source snippet
Anthropic co-founder maps out how recursive AI improvement...5 May 2026 — Jack Clark argues in a long essay that the building blocks for...
Published: May 2026
-
Source: x.com
Link: https://x.com/AnthropicAISource snippet
Anthropic (@AnthropicAI) / Posts / XAnthropic✓... We're an AI safety and research company that builds reliable, interpretable, and steer...
-
Source: forklog.com
Title: anthropic co founder forecasts self developing ai by 2028
Link: https://forklog.com/en/anthropic-co-founder-forecasts-self-developing-ai-by-2028/Source snippet
Anthropic co-founder forecasts 'self-developing AI' by 20285 May 2026 — By 2028, AI systems capable of designing and training their own s...
Published: May 2026
Additional References
-
Source: situational-awareness.ai
Link: https://situational-awareness.ai/superalignment/Source snippet
IIIc. SuperalignmentOur current alignment techniques (methods to ensure we can reliably control, steer, and trust AI systems) won't scale...
-
Source: theguardian.com
Link: https://www.theguardian.com/technology/2026/jan/06/leading-ai-expert-delays-timeline-possible-destruction-humanitySource snippet
Previously, in his widely discussed "AI 2027" scenario, Kokotajlo predicted that AI would achieve fully autonomous coding by 2027 and the...
-
Source: forethought.org
Link: https://www.forethought.org/research/will-ai-r-and-d-automation-cause-a-software-intelligence-explosionSource snippet
earch, feedback loops could overcome diminishing returns, significantly accelerating AI progress.Read more...
-
Source: medium.com
Title: flourishing an optimistic alternative to ai 2027 0fed085a6863
Link: https://medium.com/%40papricaalison/flourishing-an-optimistic-alternative-to-ai-2027-0fed085a6863Source snippet
Flourishing — An Optimistic Alternative to AI 2027Occurrence 2: Automated AI R&D leads to self-improving ASIs. · AI 2027 forecast impact...
-
Source: instagram.com
Title: The timeline for AI autonomy just accelerated drastically
Link: https://www.instagram.com/reel/DY44llOxyML/Source snippet
co-founder Jack Clark believes there's a serious possibility that, by 2028, AI systems could start developing better su...
-
Source: lesswrong.com
Title: ais will greatly change engineering in ai companies well
Link: https://www.lesswrong.com/posts/uRdJio8pnTqHpWa4t/ais-will-greatly-change-engineering-in-ai-companies-wellSource snippet
complete tasks that take top human research... (This could even happen before full AI R&D automation but after automation of research...
-
Source: instagram.com
Title: G V on Instagram: “What if AI could improve itself?
Link: https://www.instagram.com/reel/DYVC5FZGALK/Source snippet
That's the...The core idea is called open-endedness—AI systems that co-evolve by constantly challenging each other, like biological evol...
-
Source: blog.redwoodresearch.org
Title: whats up with anthropic predicting
Link: https://blog.redwoodresearch.org/p/whats-up-with-anthropic-predictingSource snippet
redwoodresearch.orgWhat's up with Anthropic predicting AGI by early 2027?Nov 3, 2025 — As far as I'm aware, Anthropic is the only AI comp...
-
Source: youtube.com
Link: https://www.youtube.com/watch?v=ckbGs4F86LUSource snippet
The AI Intelligence Explosion: Why Recursive Self-Improvement Changes Everything...
-
Source: youtube.com
Title: The AI Intelligence Explosion: Why Recursive Self-Improvement Changes Everything
Link: https://www.youtube.com/watch?v=xRQQFCWhobcSource snippet
They FINALLY Made an AI That Doesn't Need Us Anymore... SELF-TRAINED and No Limits...
Topic Tree