text for the cpt stage in your checkpoint pipeline

#1
by Sumama-Dev1 - opened

hey Neel, saw finetuning-checkpoints, the 0.6B with cpt, sft, dpo and grpo tags. you're running every stage, which is exactly the setup this data is for.

66 recorded sessions of an agent fixing real bugs in real repos, every command executed for real, errors and retries kept, tests green at the end. verify.py so you can check it before spending GPU. built to make models better at coding, tool calling, agentic execution, and recovery from failure.
it's pretraining text for your cpt stage, not a fine-tune set. I'm pushing my own 0.6B twin ablation in two days, one corpus with the trajectories and one without, so I'll have numbers on the same base you're using.

first 100 are free, so all 66 are yours free today:
https://github.com/Sumama-Jameel/megabit-trajectories

want it for the next cpt run? happy to answer anything.
Sumama

Sign up or log in to comment