Blame for README.md

---
license: mit
library_name: frm
tags:
- flow-reasoning-models
- reasoning
- pytorch
---

# Flow Reasoning Models — lightweight release draft

One selected checkpoint per task, with portable benchmark/calibration assets and
measured results. This repository stays private during testing.

## Setup

GitHub installation only; nothing is published to PyPI.

```bash
python -m venv .venv
source .venv/bin/activate
pip install "git+https://github.com/helblazer811/Flow-Reasoning-Models.git"
```

## Download and evaluate

Use standard Hugging Face downloads:

```python
from huggingface_hub import snapshot_download
snapshot_download("helblazer811/FlowReasoningModel", allow_patterns=["sudoku/*"], local_dir="models")
```

```bash
python -m frm.evaluate --model-dir models/sudoku
```

Replace `sudoku` with `maze_unique` or `zebra` for the other models. Evaluation
checks the downloaded files and uses calibrated sequential restarts. Results go
under `outputs/`. Add `--max-samples 8` for a functional check, not a benchmark score.
The evaluator computes fixed-depth comparisons too; reported policy NFE excludes
that additional comparison work. For manual download, obtain all files in the
corresponding model folder from this repository.

## Selected models and measured results

| Bundle | Training seed | Benchmark | Sequential restart accuracy | Mean NFE |
|---|---:|---|---:|---:|
| sudoku | 0 | EqR Sudoku-Lite, 2,048 puzzles | 99.7070% (2042/2048) | 899.75 |
| maze_unique | 0 | Official EqR Maze-Unique, 1,000 puzzles | 99.9% (999/1000) | 15.708 |
| zebra | 0 | 5x5 Zebra, 1,024-puzzle seed0 subset | 99.6094% (1020/1024) | 6.2402 |

Sudoku costs 0.8062 mean TFLOPs/puzzle; Maze-Unique costs 0.1781. Zebra FLOPs
are unprofiled for this preprocessing, so only NFE is reported. Zebra seed0 scores99.8047% at257NFE with the fixed-depth held-t trajectory;
its stopping/restart policy scores99.6094% at6.2402meanNFE. Its earlier SDE
result is99.6094%. Every test puzzle accepts on the first attempt. FLOPs for this
5x5 setup are unprofiled; NFE is reported instead.

All three hosted checkpoints now use training seed0. Maze-Unique restart seeds0/1/2:
99.9%,98.5%,99.8%. Zebra SDE seeds0/1/2:99.6094%,99.8047%,99.5117%.
Only one new Sudoku training seed was run. Zebra seed0 was preferred for a
consistent release after its measured default score99.6094% exceeded seed1's
99.4141%. Source checkpoint/step/seed details remain in each manifest.

## Scope and reproducibility

- Sudoku is the independently tested fixed-time FPF training variant, evaluated on
  Sudoku-Lite; this does not establish the paper's full-test headline. Its independent
  development calibration excludes known training and test puzzles.
- Maze-Unique uses the official pinned EqR1k/1k dataset and standard two-stage FPF.
  Its256-puzzle calibration is sampled from the training split as in the published policy.
- Zebra currently trains/tests5x5 with repacked answers. It does **not** reproduce
  the paper's mixed3x3–6x6 pinned-answer dataset. Calibration uses training puzzles
  explicitly disjoint from test; no test labels select stopping/restart candidates.

Each folder contains `model.pt`, `training.yaml`, `threshold.json`, `test.npz`,
`calibration.npz`, `manifest.json`, `provenance.json`, and `expected_results.json`.
The manifest records the original checkpoint hash and the compact inference-checkpoint
hash. Optimizer state is omitted; raw weights and EMA are retained. These are compact PyTorch checkpoints from the recorded training runs.
For training from scratch, download `training.yaml` and run
`python -m frm.train --config training.yaml` on the matching data.

Training and evaluation code is available in
[Flow-Reasoning-Models](https://github.com/helblazer811/Flow-Reasoning-Models).
This lightweight release supports
the models and evaluations above, not every paper baseline, ablation, table or figure.