---
license: other
license_name: timegravity-research-only-1.0
license_link: LICENSE
language:
- en
pipeline_tag: text-generation
tags:
- reasoning
- verifier
- parallel-decoding
- test-time-compute
- small-model
---

# maram-350m

A 339,267,586-parameter decoder-only language model trained from scratch on one 48 GB GPU to run K = 3 reasoning streams in parallel inside one sequence (block-interleaved every N = 64 tokens) under a tree attention mask, with a 2,100,226-parameter judge head that reads a periodic `<chk>` token and predicts which stream is right. This repository holds both the trunk (`maram-350m-v2`, the second and valid 350M training run) and the same trunk with the judge head fine-tuned on the model's own streams (`maram-350m-v2-judge-v2`), which is the checkpoint to use for native decoding.

Paper: *MARAM: a free in-weights verifier for parallel reasoning streams — selection beats voting, matches beam at equal tokens, and pruning doesn't help* (Angurajan, 2026), DOI [10.5281/zenodo.23857054](https://doi.org/10.5281/zenodo.23857054). Every number below is copied from the paper and its `RESULTS.md`; see `MODEL_CARD_maram-350m-v2.md` for the full card and `MODEL_CARD.md` for the 125M ablation models.

## What it does
On held-out 5–7-step arithmetic (synth-hard, n = 2,000, 95 % Wilson intervals), with judge-v2:

| method | accuracy | tokens per problem |
|---|---|---|
| greedy decompose | 0.7095 [0.689, 0.729] | 145 |
| best-of-3 anchored, value vote | 0.7325 [0.713, 0.751] | 421 |
| beam 2 on the same trunk | 0.7625 [0.743, 0.781] | 366 |
| judge selector over 3 streams, no pruning | 0.7730 [0.754, 0.791] | 425 |
| gated native, K3 N64, τ 0.35 δ 0.1 | 0.7755 [0.757, 0.793] | 425 |
| beam 3 on the same trunk | 0.8070 [0.789, 0.824] | 617 |

Judge selection beats the value vote on identical samples and matches beam-2 at equal tokens; beam-3 is higher at 1.45× the tokens. Easy 2–5-step arithmetic is near saturation (greedy 0.9310); Countdown and GSM8K are at chance for every method. The margin exists only at the trained K = 3 and N = 64. Details and limitations: the paper, §4–5.

## Use
```
pip install -r maram_inference/requirements.txt
cd maram_inference
python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode greedy1 --strat decompose \
    --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6."
python generate.py --model ../maram-350m-v2-judge-v2 --tokenizer ../tokenizer.json --mode native --K 3 --N 64 \
    --strats decompose,decompose,decompose --temps 0,0.3,0.3 --strat_pool decompose --fork_temp 0.5 \
    --prune gated --tau 0.35 --delta 0.1 --prompt "Compute the value of ((39 - 11) - 20) + 865) * 6."
```
`--prune none` gives the selector-only rule. The model answers prompts of the form "Compute the value of <expression>." and is not a general assistant.

## Files
- `maram-350m-v2/`, `maram-350m-v2-judge-v2/`: `model.safetensors` (fp32), `config.json`, `train_config.json`, `export_info.json` (source checkpoint and its SHA-256), `CHECKSUMS.txt`.
- `tokenizer.json`: byte-level BPE, 16,384 entries, every digit a single token.
- `maram_inference/`: mask, model, judge head, decoder (`--prune {always,gated,none}`), tokenizer wrapper, loader, CLI. No training code is released.
- `heldout/{synth,synth_hard,countdown}.jsonl`: the 2,000-problem held-out sets of the paper, with `CHECKSUMS.txt`. GSM8K test is public.
- `maram-125m/`, `maram-125m-judge-v2/`, `maram-flat-125m/`: the 125M ablation models.
- `MODEL_CARD_maram-350m-v2.md`, `MODEL_CARD.md`.

## Licence
Weights, tokenizer, inference package, model cards and held-out files: Timegravity Research-Only Licence v1.0, see `LICENSE` (SPDX `LicenseRef-Timegravity-Research-Only-1.0`; commercial licences on request). Paper: CC BY 4.0.

## Citation
```
@misc{angurajan2026maram,
  author    = {Angurajan, Vignesh},
  title     = {MARAM: a free in-weights verifier for parallel reasoning streams -- selection beats voting, matches beam at equal tokens, and pruning doesn't help},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.23857054},
  url       = {https://doi.org/10.5281/zenodo.23857054}
}
```