Abstract
Frontier LLMs are increasingly used to automate scientific research through iterative search. We distinguish idea-driven search from solution-driven search and identify three core challenges: organizing evolving research ideas, selecting promising directions, and maintaining alignment between ideas and their implementations. To address these challenges, we introduce the Agentic Idea Manager (AIM), a fully autonomous framework for managing and exploring research directions in idea-driven automated research. Inspired by Bayesian optimization, AIM uses an Agentic Surrogate and an Agentic Acquisition mechanism to organize discovered ideas and guide their selection. A Solution Auditor maintains idea-solution integrity, while a Resource Planner adaptively allocates the remaining experimental budget across parallel search branches. Experiments on 10 AutoLab benchmark tasks show that AIM surpasses the strongest baseline by 1.6 percentage points on System Optimization tasks and 4.9 percentage points on long-horizon Model Development & CUDA tasks. Notably, AIM reaches the best baseline performance up to 3.1x faster in wall-clock time. We further provide a theoretical analysis of when searching over ideas becomes beneficial. Our analysis shows that explicit idea-level allocation makes semantic coverage directly controllable, and that broader coverage becomes increasingly valuable when competitive research directions are sparse among many plausible alternatives. Project Page: https://imhgchoi.github.io/agentic-idea-manager/
Community
Implementing and evaluating research ideas is expensive. As agents generate more candidates, choosing what to explore and learning from previous experiments becomes increasingly important. Numerous approaches have been proposed for automated research. We distinguish two ways to organize those approaches: solution-driven search, which searches directly over executable implementations, and idea-driven search, which explicitly selects research directions before delegating their implementation.
In this work, we introduce the Agentic Idea Manager (AIM), a framework that makes managing these ideas a central part of the research process. This is what AIM does:
š” Organizes evolving ideas into semantic clusters and estimates their promise using experimental evidence.
š Balances exploration of new directions with refinement of promising ones.
š§ Audits whether implementations faithfully realize their intended ideas.
āļø Adaptively allocates the experimental budget across parallel search branches.
Across 10 AutoLab tasks, AIM improves average scores over the strongest baseline by:
ā¢ +1.6 percentage points on System Optimization.
ā¢ +4.9 percentage points on long-horizon Model Development & CUDA.
It also reaches the best baseline performance up to 3.1x faster in wall-clock time.
On top of empirical results, our theoretical analysis examines when idea-level search is useful, highlighting the value of broader coverage when plausible directions to study into are sparse.
Get this paper in your agent:
hf papers read 2609.38445 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper
Collections including this paper 0
No Collection including this paper