real run-and-fix sessions for analyst (free)

#1
by Sumama-Dev1 - opened

hey Emmanuel, saw analyst-v1 and that you publish your eval results as a dataset. same transparency habit here: my data ships with verify.py so anyone can re-check it. 66 recorded sessions of an AI fixing real bugs in real repos. every command ran for real, real errors and the fixes after them, tests passing at the end, each session scored against the actual human change.

honest note: it's library bug-fix work, not analysis notebooks. but the loop your analyst needs is there: run code, read the real error, fix, run again. built to improve coding, tool calling, agentic execution, and recovery from failure.

it's for continued pretraining, not SFT, and small enough to try on a modest setup. the first 100 are free, so all 66 are free today:
https://github.com/Sumama-Jameel/megabit-trajectories

if you try it before the next analyst version or a Zindi round, I'd like to hear what changed.
Sumama

Sign up or log in to comment