MARAM release: maram-350m-v2 trunk, judge-v2 head, 125M ablation models, tokenizer, inference package, held-out files, model cards, licence
7b41231 verified | license: other | |
| license_name: timegravity-research-only-1.0 | |
| license_link: LICENSE | |
| language: | |
| - en | |
| pipeline_tag: text-generation | |
| tags: | |
| - reasoning | |
| - verifier | |
| - parallel-decoding | |
| - test-time-compute | |
| - small-model | |
| # maram-350m | |
| A 339,267,586-parameter decoder-only language model trained from scratch on one 48 GB GPU to run K = 3 reasoning streams in parallel inside one sequence (block-interleaved every N = 64 tokens) under a tree attention mask, with a 2,100,226-parameter judge head that reads a periodic `<chk>` token and predicts which stream is right. This repository holds both the trunk (`maram-350m-v2`, the second and valid 350M training run) and the same trunk with the judge head fine-tuned on the model's own streams (`maram-350m-v2-judge-v2`), which is the checkpoint to use for native decoding. | |
| Paper: *MARAM: a free in-weights verifier for parallel reasoning streams — selection beats voting, matches beam at equal tokens, and pruning doesn't help* (Angurajan, 2026), DOI [10.5281/zenodo.23857054](https://doi.org/10.5281/zenodo.23857054). Every number below is copied from the paper and its `RESULTS.md`; see `MODEL_CARD_maram-350m-v2.md` for the full card and `MODEL_CARD.md` for the 125M ablation models. | |
| ## What it does | |
| On held-out 5–7-step arithmetic (synth-hard, n = 2,000, 95 % Wilson intervals), with judge-v2: | |
| | method | accuracy | tokens per problem | | |
| |---|---|---| | |
| | greedy decompose | 0.7095 [0.689, 0.729] | 145 | | |
| | best-of-3 anchored, value vote | 0.7325 [0.713, 0.751] | 421 | | |
| | beam 2 on the same trunk | 0.7625 [0.743, 0.781] | 366 | | |
| | judge selector over 3 streams, no pruning | 0.7730 [0.754, 0.791] | 425 | | |
| | gated native, K3 N64, τ 0.35 δ 0.1 | 0.7755 [0.757, 0.793] | 425 | | |
| | beam 3 on the same trunk | 0.8070 [0.789, 0.824] | 617 | | |
| Judge selection beats the value vote on identical samples and matches beam-2 at equal tokens; beam-3 is higher at 1.45× the tokens. Easy 2–5-step arithmetic is near saturation (greedy 0.9310); Countdown and GSM8K are at chance for every method. The margin exists only at the trained K = 3 and N = 64. Details and limitations: the paper, §4–5. | |
| ## Use | |
| ``` | |
| pip install -r maram_inference/requirements.txt | |
| cd maram_inference | |
| python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode greedy1 --strat decompose \ | |
| --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6." | |
| python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode native --K 3 --N 64 \ | |
| --strats decompose,decompose,decompose --temps 0,0.3,0.3 --strat_pool decompose --fork_temp 0.5 \ | |
| --prune gated --tau 0.35 --delta 0.1 --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6." | |
| ``` | |
| `--prune none` gives the selector-only rule. The model answers prompts of the form "Compute the value of <expression>." and is not a general assistant. | |
| ## Files | |
| - `maram-350m-v2/`, `maram-350m-v2-judge-v2/`: `model.safetensors` (fp32), `config.json`, `train_config.json`, `export_info.json` (source checkpoint and its SHA-256), `CHECKSUMS.txt`. | |
| - `tokenizer.json`: byte-level BPE, 16,384 entries, every digit a single token. | |
| - `maram_inference/`: mask, model, judge head, decoder (`--prune {always,gated,none}`), tokenizer wrapper, loader, CLI. No training code is released. | |
| - `heldout/{synth,synth_hard,countdown}.jsonl`: the 2,000-problem held-out sets of the paper, with `CHECKSUMS.txt`. GSM8K test is public. | |
| - `maram-125m/`, `maram-125m-judge-v2/`, `maram-flat-125m/`: the 125M ablation models. | |
| - `MODEL_CARD_maram-350m-v2.md`, `MODEL_CARD.md`. | |
| ## Licence | |
| Weights, tokenizer, inference package, model cards and held-out files: Timegravity Research-Only Licence v1.0, see `LICENSE` (SPDX `LicenseRef-Timegravity-Research-Only-1.0`; commercial licences on request). Paper: CC BY 4.0. | |
| ## Citation | |
| ``` | |
| @misc{angurajan2026maram, | |
| author = {Angurajan, Vignesh}, | |
| title = {MARAM: a free in-weights verifier for parallel reasoning streams -- selection beats voting, matches beam at equal tokens, and pruning doesn't help}, | |
| year = {2026}, | |
| publisher = {Zenodo}, | |
| doi = {10.5281/zenodo.23857054}, | |
| url = {https://doi.org/10.5281/zenodo.23857054} | |
| } | |
| ``` | |