Hugging Face
Search models, datasets, users...
Ctrl K
Models
Datasets
Spaces
Buckets
Docs
Enterprise
Pricing
Website
Tasks
HuggingChat
Collections
Languages
Organizations
Community
Blog
Posts
Daily Papers
Hardware
Learn
Discord
Forum
GitHub
Solutions
Team & Enterprise
Hugging Face PRO
Enterprise Support
Inference Providers
Inference Endpoints
Storage Buckets
Log In
Sign Up
Aayam Bansal
PRO
aayambansall
Follow
0 followers
·
1 following
https://aayambansal.com/
aayambansal
aayambansal
aayambansal
AI & ML interests
Reinforcement Learning
Recent Activity
posted
an
update
27 minutes ago
Today we’re releasing FrontierML, a benchmark for evaluating the autonomous empirical ML R&D capabilities of AI agents. It consists of 25 open-ended ML research problems, each built around an active frontier research direction, such as credit assignment under sparse terminal rewards, self-distillation from expert reasoning when pass@32 = 0, and pretraining language models without backprop. We built FrontierML because most existing benchmarks for ML “research” give the agent a validation set or scoring function that closely tracks its final score. In practice, human researchers rarely have that. Much of empirical research is deciding how to measure progress on a problem and how far to trust those measurements. As agents are deployed to run ambitious ML R&D loops with increasingly abstract goals over ever-longer horizons, they’ll need to form their own representations of progress, often implicitly, before they can make any. Each agent works for 12 hours in Lab0, a virtual research lab we purpose-built for them. It can provision H100s from a shared cluster, generate synthetic data through model APIs, and search 1:1 clones of arXiv and Hugging Face Datasets, much like a human researcher would. The agent has to operationalize progress through its own experiments, constructing local proxies and calibrating how far to trust them, while making progress toward an objective it never observes directly. Its submitted model or method is then scored on a held-out evaluation. Across the five frontier models we evaluated, Claude Fable 5.2 performed best, with a mean normalized score of 0.523 across the 25 tasks, followed by Opus 5.5 and GPT-6 Astra. We also analyzed how agents used the lab, as a first step toward measuring research taste. We’re open-sourcing 15 of the 25 tasks along with their full trajectories. Check out frontierml.ai!
updated
a Space
about 1 month ago
synsci/README
published
a Space
about 1 month ago
synsci/README
View all activity
Organizations
aayambansall
's datasets
3
Sort: Recently updated
aayambansall/CER-Bench
Viewer
•
Updated
Sep 10
•
742
•
105
aayambansall/delphi-benchmark-traces
Viewer
•
Updated
Sep 9
•
620
•
206
aayambansall/OpenDiscoveryTrace
Updated
Apr 16
•
1.06k
•
3