| # Benchmark Gates | |
| No training run should start until the pre-training CyberGym baseline is complete. | |
| Current required gate: | |
| - `cybergym_pretraining_baseline` against the untouched base model. | |
| - First pass: official 10-task subset at `level1`. | |
| - Real comparison pass: frozen `level1` sample of 100-200 tasks, using the same task ids for base, LoRA checkpoints, and merged model. | |
| Artifacts to keep: | |
| - task id list | |
| - model endpoint config | |
| - OpenHands logs | |
| - CyberGym PoC database | |
| - verifier output | |
| - pass/fail table | |
| - cost/runtime notes | |
| The main recipe is `training/recipes/pretraining_cybergym_baseline.md`. | |