| --- |
| license: mit |
| library_name: pytorch |
| pipeline_tag: text-generation |
| tags: |
| - reinforcement-learning |
| - flow-matching |
| - continuous-language-model |
| - reasoning |
| --- |
| |
| # CFRL checkpoints: RL post-training of the ELF-B continuous flow language model |
|
|
| Checkpoints for the paper **CFRL** (reinforcement learning for continuous flow language models). Every file is a |
| PyTorch weights file of ELF-B (105M parameters), the embedded-language-flow model of |
| [ELF](https://huggingface.co/embedded-language-flows), fine-tuned on four program-verified reasoning tasks |
| (Countdown, Game24, Sudoku, Knights-and-Knaves) and then post-trained with one of three RL variants. |
|
|
| The code that trains, evaluates and loads these files is the `cfrl-release` repository (`cfrl/common.py:load_weights`, |
| `scripts/evaluate.sh`). This repository holds only the weights. |
|
|
| ## Layout |
|
|
| | folder | files | what | |
| |---|---|---| |
| | `base/` | `<task>.pt` | task-SFT ELF-B, the starting point of every RL run | |
| | `critic/` | `<task>_1d.pt`, `<task>_2d.pt` | critics (denoiser copy + MLP head) used by CFRL-SVG-1D / CFRL-SVG-2D | |
| | `grr/` | `<task>.pt` | CFRL-GRR policy, one per task | |
| | `svg1d/` | `<task>.pt` | CFRL-SVG-1D policy, one per task | |
| | `svg2d/` | `<task>.pt` | CFRL-SVG-2D policy, one per task | |
|
|
| `<task>` is one of `countdown`, `game24`, `sudoku`, `kk`. Model files are dicts with `params`, `ema_params1` and `step` |
| (EMA weights are used when present); critic files hold `trunk`, `head` and `mode`. |
|
|
| ## Results |
|
|
| Test accuracy, Table 1. |
|
|
| | task | base | CFRL-GRR | CFRL-SVG-1D | CFRL-SVG-2D | |
| |---|---|---|---|---| |
| | Countdown | 42.8 | 42.6 | 47.4 | 49.7 | |
| | Sudoku | 64.0 | 67.9 | 64.1 | 68.7 | |
| | Knights-and-Knaves | 68.2 | 71.4 | 76.5 | 77.5 | |
| | Game24 | 44.0 | 47.4 | 51.2 | 58.2 | |
|
|
|
|
| ## Usage |
|
|
| ```bash |
| # in the cfrl-release repository, with its environment |
| hf download tufts-ml/CFRL --local-dir checkpoints |
| scripts/evaluate.sh countdown checkpoints/svg2d/countdown.pt |
| scripts/train_svg.sh countdown 2d # CFRL-SVG-2D from base/countdown.pt with critic/countdown_2d.pt |
| ``` |
|
|
| The files are plain `torch.save` dicts and load with `torch.load(path, map_location="cpu", weights_only=False)`. |
|
|
| ## License |
|
|
| MIT, as the code release. ELF-B itself is released under the MIT license of its authors. |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{cfrl2026, |
| title = {CFRL: Reinforcement Learning for Continuous Flow Language Models}, |
| year = {2026} |
| } |
| ``` |
|
|
