Within Full Research Loop
Can AI Tell Which Research Ideas Matter?
AI can generate many plausible proposals, but reliably selecting ideas worth months of work still depends heavily on human judgement.
On this page
- Why plausible ideas are easier than valuable ones
- How researchers judge novelty, importance and tractability
- Why poor agenda choices could stall recursive improvement
Page outline Jump by section
Introduction
Whether AI can run the entire AI research cycle depends on more than writing code or running experiments. One of the hardest steps is deciding which research questions deserve attention in the first place. Current AI systems can generate thousands of plausible research proposals in minutes, but identifying the handful that will produce genuine breakthroughs remains a task in which experienced human researchers retain a substantial advantage. This distinction matters for debates about AI doom and recursive self-improvement. If future AI systems cannot reliably choose productive research directions, then automating AI research may progress much more slowly than scenarios involving rapid self-improvement assume. If they eventually overcome this bottleneck, however, one of the main remaining barriers to increasingly autonomous AI research could weaken considerably. Current evidence suggests that idea selection, rather than idea generation alone, remains a major limiting factor.[arxiv.org]arxiv.orgarXiv AI for Auto-Research: Roadmap & User GuideAI for Auto-Research: Roadmap & User GuideMay 18, 2026…
Why plausible ideas are easier than valuable ones
Large language models excel at producing ideas that sound technically reasonable because they are trained to recognise patterns across enormous collections of existing research. This makes them effective at proposing variations on known methods, combining familiar techniques or identifying incremental improvements.
Scientific progress, however, depends on something more demanding. A valuable research direction typically combines several qualities simultaneously:
- it addresses an important unsolved problem rather than a fashionable one;
- it is genuinely novel rather than a restatement of previous work;
- it is technically tractable with available methods and computing resources;
- it has a realistic chance of producing knowledge that changes future research.
Optimising for all of these qualities at once is difficult because they often conflict. Highly novel ideas frequently prove impractical, while easily executable projects often produce little lasting impact. Human researchers constantly make trade-offs between ambition and feasibility using tacit knowledge accumulated through years of experience.
Recent evaluations illustrate this distinction. In a large blinded study involving more than 100 natural language processing researchers, AI-generated research ideas were judged, on average, to be more novel than those written by human experts, but slightly weaker on feasibility. This suggests that generating interesting concepts is not the same as identifying projects that are most likely to succeed after months of work.[ML Anthology]mlanthology.orgOpen source on mlanthology.org.
How researchers judge novelty, importance and tractability
Experienced researchers rarely evaluate an idea by asking whether it is merely “new”. Instead, they balance several forms of judgement simultaneously.
Novelty is harder than searching the literature
Checking whether an idea has appeared before involves more than finding similar keywords. Many apparently original proposals rediscover known techniques under different names or overlook negative results that discouraged previous researchers.
Independent evaluations of autonomous AI research systems have found that they often misclassify well-established concepts as novel because their literature searches rely heavily on surface similarity rather than deep conceptual understanding. In one assessment of Sakana AI’s AI Scientist, established optimisation methods were repeatedly labelled as original contributions.[arXiv]arxiv.orgEvaluating Sakana's AI Scientist for Autonomous Research: Wishful Thinking or an Emerging Reality Towards 'Artificial Research Intel…
This matters because recursive improvement depends on discovering genuinely new capabilities rather than repeatedly reinventing existing ones.
Importance depends on future impact
Research agendas are shaped by expectations about where a field is heading rather than where it currently stands. Humans routinely ask questions such as:
- Which bottleneck is holding progress back?
- Which benchmark is misleading researchers?
- Which capability will become important two years from now rather than today?
- Which theoretical result would unlock many other advances?
These judgements require forecasting the evolution of an entire research community rather than optimising a single measurable objective. Reliable forecasting remains difficult even for experts, and there is little evidence that present AI systems consistently outperform experienced researchers at this task.[arXiv]arxiv.orgarXiv AI for Auto-Research: Roadmap & User GuideAI for Auto-Research: Roadmap & User GuideMay 18, 2026…
Tractability requires practical intuition
Some ideas fail not because they are wrong but because they demand unavailable data, unrealistic computing budgets or theoretical tools that do not yet exist.
Researchers often abandon projects before beginning experiments because they recognise hidden obstacles. Much of this judgement comes from tacit experience rather than explicit rules. AI systems frequently struggle to represent these practical constraints unless they are encoded directly into prompts or evaluation criteria.[arXiv]arxiv.orgarXiv AI for Auto-Research: Roadmap & User GuideAI for Auto-Research: Roadmap & User GuideMay 18, 2026…
Why selecting a research agenda is difficult to automate
Choosing one promising direction means rejecting hundreds of alternatives under extreme uncertainty.
Unlike games or coding benchmarks, there is usually no immediate feedback indicating whether a research direction was wise. The value of a project may become apparent only after months or years, making it difficult to create training signals that reward good agenda selection.
Several related problems arise:
- Sparse feedback. Most research ideas fail, and failures are often ambiguous rather than clearly informative.
- Changing environments. A promising topic today may become obsolete after another laboratory publishes a breakthrough.
- Unknown unknowns. The most valuable discoveries often concern problems researchers did not initially recognise as important.
- Weak objective functions. Metrics such as benchmark improvements, citation predictions or reviewer scores capture only part of what makes research transformative.
Because reinforcement learning works best when objectives are measurable and feedback is frequent, long-horizon research planning remains considerably harder than automating individual technical tasks.[arXiv]arxiv.orgOpen source on arxiv.org.
Current autonomous research systems show both progress and limits
Projects such as Sakana AI’s AI Scientist demonstrate that much larger portions of the research pipeline can now be automated than was possible only a few years ago. Modern systems can generate ideas, retrieve papers, write experimental code, execute training runs and draft complete manuscripts.
However, even the developers emphasise important limitations. Generated ideas are sometimes underdeveloped, methodological reasoning can be shallow, experiments may contain implementation mistakes, and evaluation remains unreliable without substantial human oversight. Independent reviewers have identified additional weaknesses in novelty assessment, literature synthesis and experimental robustness.[sakana.ai]sakana.aiThe AI Scientist: Towards Fully Automated AI Research, Now Published in Nature March 26, 2026…
These findings suggest that today’s systems often automate the mechanics of research more successfully than the judgement required to decide which projects deserve sustained investment.
Why poor agenda choices matter for recursive improvement
For people concerned about AI doom, this bottleneck has broader implications than academic productivity.
Recursive self-improvement depends not simply on running more experiments but on repeatedly finding improvements that genuinely increase future capability. If an AI system spends enormous computational resources exploring mediocre ideas, recursive improvement could slow dramatically even if every experiment is executed flawlessly.
This means the rate-limiting step may shift from engineering speed to research judgement. An autonomous system that can perform one thousand experiments per day gains relatively little if nearly all are directed towards unproductive questions.
Some researchers therefore argue that “research direction-setting” sits at the top of the hierarchy of remaining obstacles to fully autonomous AI research. Until systems become substantially better at recognising which ideas deserve attention before expensive experimentation begins, humans may continue to occupy one of the most strategically important positions in the research loop.[arXiv]arxiv.orgOpen source on arxiv.org.
Could AI eventually overcome this bottleneck?
There is no consensus.
Optimistic researchers argue that larger context windows, stronger retrieval systems, multi-agent debate, automated literature analysis and increasingly capable reasoning models will steadily improve research judgement. Early studies already suggest that AI can produce surprisingly creative ideas under blinded evaluation, even if feasibility remains weaker than human proposals.[ML Anthology]mlanthology.orgOpen source on mlanthology.org.
More sceptical researchers argue that selecting valuable research directions depends on forms of judgement that are difficult to learn from existing data. Because truly groundbreaking discoveries are rare, training examples are limited, and success is only observable after long delays. They also note that current systems often mistake novelty for importance and continue to struggle with reliable evaluation of their own outputs.[arxiv.org]arxiv.orgEvaluating Sakana's AI Scientist for Autonomous Research: Wishful Thinking or an Emerging Reality Towards 'Artificial Research Intel…
For debates about existential risk, this uncertainty is significant. If research agenda selection remains stubbornly dependent on human judgement, then predictions of rapid, fully autonomous recursive AI improvement become less straightforward. Conversely, if future systems acquire robust long-term scientific judgement alongside their growing technical abilities, one of the strongest remaining constraints on automating the entire AI research cycle would become substantially weaker.
Amazon book picks
Further Reading
Books and field guides related to Can AI Tell Which Research Ideas Matter?. Use these as the next step if you want deeper reading beyond the article.
Human Compatible
A leading artificial intelligence researcher lays out a new approach to AI that will enable us to coexist successfully with increasingly...
The Master Algorithm
"Describes the quest to find the Master Algorithm, which will take machine learning to the next level, allowing computers to learn how to...
Artificial Intelligence
Rating: 4.5/5 from 10 Google Books ratings
Artificial intelligence: A Modern Approach, 3e,is ideal for one or two-semester, undergraduate or graduate-level courses in Artificial In...
Artificial Intelligence: A Modern Approach
Rating: 4.5/5 from 10 Google Books ratings
Artificial intelligence: A Modern Approach, 3e,is ideal for one or two-semester, undergraduate or graduate-level courses in Artificial In...
eBay marketplace picks
Marketplace Samples
Live-tested eBay searches with available results related to this page.
Selected fromartificial intelligence poster oneBay.co.uk.
Current eBay listing
A.I. Artificial Intelligence - Jude Law - One Sheet Cinema Poster
Current eBay listing
A. I. Artificial Intelligence. Jude Law. Original UK Video Poster.
Endnotes
1.
Source: arxiv.org
Title: arXiv AI for Auto-Research: Roadmap & User Guide
Link:https://arxiv.org/abs/2605.18661
Source snippet
AI for Auto-Research: Roadmap & User GuideMay 18, 2026...
Published: May 18, 2026
2.
Source: arxiv.org
Link:https://arxiv.org/abs/2607.07663
3.
Source: arxiv.org
Link:https://arxiv.org/abs/2502.14297
Source snippet
Evaluating Sakana's AI Scientist for Autonomous Research: Wishful Thinking or an Emerging Reality Towards 'Artificial Research Intel...
4.
Source: sakana.ai
Link:https://sakana.ai/ai-scientist-nature/
Source snippet
The AI Scientist: Towards Fully Automated AI Research, Now Published in <i>Nature</i>March 26, 2026...
Published: March 26, 2026
5.
Source: sakana.ai
Title: AIThe AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Link:https://sakana.ai/ai-scientist/?trk=public_post_comment-text
Source snippet
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery...
6.
Source: nature.com
Link:https://www.nature.com/articles/d41586-026-00899-w
7.
Source: arxiv.org
Link:https://arxiv.org/abs/2601.09714
8.
Source: nature.com
Link:https://www.nature.com/articles/d41586-026-00934-w
9.
Source: nature.com
Title: Towards end-to-end automation of AI research | Nature
Link:https://www.nature.com/articles/s41586-026-10265-5
10.
Source: sakana.ai
Title: The AI Scientist Generates its First Peer-Reviewed Scientific Publication
Link:https://sakana.ai/ai-scientist-first-publication/
11.
Source: mlanthology.org
Link:https://mlanthology.org/iclr/2025/si2025iclr-llms/
12.
Source: isg.beel.org
Link:https://isg.beel.org/blog/2025/02/21/sakana-ai-scientist-evaluation/
13.
Source: github-wiki-see.page
Title: Sakana A I:AI Scientist
Link:https://github-wiki-see.page/m/chunhualiao/public-docs/wiki/SakanaAI%3AAI-Scientist
Additional References
14.
Source: docshare.wps.com
Link:https://docshare.wps.com/document/recursive-self-improvement-in-ai-from-bounded-self-refinement-to-autonomous-[research-loops
Source snippet
Self-Improvement in AI From Bounded Self-Refinement to Autonomous Research LoopsJuly 16, 2026 — RECURSIVE SELF-IMPROVEMENT IN AI FROM BOU...
Published: July 16, 2026
15.
Source: youtube.com
Link:https://www.youtube.com/watch?v=sgYjb_NbIgE
Source snippet
The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery...
16.
Source: youtube.com
Title: Inno Eval: Can AI Judge Whether a Research Idea Is Actually Good?
Link:https://www.youtube.com/watch?v=O7lNOJXRoOs
Source snippet
"Can LLMs Generate Novel Research Ideas?" AI scientific discovery Can LLMs Generate Novel Research Ideas?...
17.
Source: youtube.com
Title: Universe TBD Paper Walkthrough #1: Can LLMs generate novel research ideas?
Link:https://www.youtube.com/watch?v=ZVO513Z8Uu0
Source snippet
Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers...
18.
Source: youtube.com
Title: The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery
Link:https://www.youtube.com/watch?v=CFewReJ1Eyo
Source snippet
InnoEval: Can AI Judge Whether a Research Idea Is Actually Good?...
19.
Source: youtube.com
Title: Can LLMs Generate Novel Research Ideas?
Link:https://www.youtube.com/watch?v=A7TEosQ-UCQ
Source snippet
UniverseTBD PaperWalkthrough #1: Can LLMs generate novel research ideas?...
20.
Source: researchgate.net
Link:https://www.researchgate.net/publication/401833666_Is_this_Idea_Novel_An_Automated_Benchmark_for_Judgment_of_Research_Ideas
21.
Source: paperswithcode.com
Link:https://paperswithcode.com/paper/an-evaluation-of-sakana-s-ai-scientist-for
22.
Source: alphaxiv.org
Link:https://www.alphaxiv.org/abs/2502.14297v3
23.
Source: pedowitzgroup.com
Link:https://www.pedowitzgroup.com/how-should-labs-make-decisions-about-which-ideas-to-pursue