Checkpoints and the numbers they reproduce
All files are PyTorch weights of ELF-B (105M). Evaluate with scripts/evaluate.sh TASK FILE (deployment sampler
gamma56@64, gamma28@32 on Sudoku; classifier-free guidance 2). Accuracies are on the test half of the held-out
prompts (odd indices), the protocol of Table 1.
Two numbers are given for every file. Release eval is what scripts/evaluate.sh prints: the mean over the 5
default sampler seeds, with the standard deviation across seeds. Paper is the single evaluation of the same
file in the research harness that went into the paper; it sits within the seed spread of the release eval.
Base models: checkpoints/base/<task>.pt
Task-SFT ELF-B, the starting point of every RL run.
| task | release eval (5 seeds) | paper |
|---|---|---|
| countdown | 41.8 ± 0.8 | 42.8 |
| sudoku | 63.1 ± 0.7 | 64.0 |
| kk | 67.6 ± 0.3 | 68.2 |
| game24 | 44.5 ± 1.3 | 44.0 |
Critics: checkpoints/critic/<task>_{1d,2d}.pt
Pre-trained critics (denoiser copy + head) that the SVG runs start from; the mode field inside the file selects
CFRL-SVG-1D or CFRL-SVG-2D in train_svg.py.
RL checkpoints: checkpoints/{grr,svg1d,svg2d}/<task>.pt
One checkpoint per task and variant: the best remaining checkpoint of the runs behind Table 1. Table 1 reports the mean over three runs; the shipped file is one run, chosen as the best of them, so its paper number sits at or above that mean.
| task | variant | file | release eval (5 seeds) | paper (this file) | Table 1 (mean of 3 runs) |
|---|---|---|---|---|---|
| countdown | CFRL-GRR | grr/countdown.pt | 41.7 ± 1.0 | 43.0 | 42.6 |
| countdown | CFRL-SVG-1D | svg1d/countdown.pt | 46.6 ± 1.0 | 48.4 | 47.4 |
| countdown | CFRL-SVG-2D | svg2d/countdown.pt | 49.4 ± 0.7 | 50.5 | 49.7 |
| sudoku | CFRL-GRR | grr/sudoku.pt | 69.9 ± 1.2 | 71.2 | 67.9 |
| sudoku | CFRL-SVG-1D | svg1d/sudoku.pt | 66.3 ± 0.6 | 65.2 | 64.1 |
| sudoku | CFRL-SVG-2D | svg2d/sudoku.pt | 69.0 ± 0.5 | 69.1 | 68.7 |
| kk | CFRL-GRR | grr/kk.pt | 74.5 ± 0.5 | 73.9 | 71.4 |
| kk | CFRL-SVG-1D | svg1d/kk.pt | 78.1 ± 0.8 | 77.5 | 76.5 |
| kk | CFRL-SVG-2D | svg2d/kk.pt | 79.3 ± 0.6 | 79.8 | 77.5 |
| game24 | CFRL-GRR | grr/game24.pt | 49.0 ± 0.5 | 48.8 | 47.4 |
| game24 | CFRL-SVG-1D | svg1d/game24.pt | 53.3 ± 0.6 | 52.9 | 51.2 |
| game24 | CFRL-SVG-2D | svg2d/game24.pt | 58.6 ± 0.9 | 59.6 | 58.2 |
Training iterations of the shipped RL checkpoints (GRR / SVG-1D / SVG-2D): countdown 500 / 750 / 1500; sudoku 500 / 250 / 500; kk 600 / 1000 / 1000; game24 600 / 1250 / 1500.