Post
6,128
New post-training example: SFT on agent traces with TRL
→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks
About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest
Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces
→ A large model works as a coding agent in OpenCode, run through OpenEnv
→ The tasks it solves become the training data
→ A 2B model goes from 5.5% to 34.4% on held-out tasks
About 500 traces get most of the gain. Five times more add only 3 to 4 points, so SFT gets the agent working and RL has to do the rest
Full guide, with the models, traces and every rollout open:
https://github.com/huggingface/trl/tree/main/examples/sft_agent_traces
