sean-diab/reviser-checkpoints

Reviser checkpoints

Checkpoints for Reviser: Writing and Editing Text with Autoregressive Cursor Actions (arXiv:2609.20830).

Reviser is a decoder-only Transformer that writes and edits text with cursor actions: insert a token, move the cursor, delete the token before the cursor (backspace), or stop. A deterministic executor applies each action to a canvas. The model is autoregressive over its own actions, so a step costs the same as a step of an autoregressive (AR) model of the same size.

• Code: github.com/Sean-Diab/Reviser
• Demo: huggingface.co/spaces/sean-diab/reviser-demo
• Step-by-step replays of 70 example trajectories: sean-diab.github.io/reviser_trajectories.html

Files

The models of the current version of the paper (v2) are under v2/. The two files at the top level are the models of the first version; they are unchanged and are the "released" rows of Table 1.

File	Model	Paper	Size
reviser_100m.pt	Reviser 100M, first version ("released")	Table 1	406 MB
reviser_300m.pt	Reviser 300M, first version ("released")	Table 1	1.19 GB
v2/continuation/reviser-100m-runjump.pt	100M, continued on the run-and-jump order	Table 1	407 MB
v2/continuation/reviser-100m-runjump-delete.pt	100M, run-and-jump + Delete (self-correction)	Table 1	407 MB
v2/continuation/reviser-100m-runjump-delete-rl-seed1.pt	100M + RL, seed 1	Table 1	433 MB
v2/continuation/reviser-100m-runjump-delete-rl-seed2.pt	100M + RL, seed 2	Table 1	433 MB
v2/continuation/reviser-100m-runjump-delete-rl-seed3.pt	100M + RL, seed 3	Table 1	433 MB
v2/continuation/reviser-300m-runjump-delete.pt	300M, run-and-jump + Delete	Table 1	1.19 GB
v2/continuation/reviser-300m-runjump-delete-rl-seed3.pt	300M + RL, seed 3	Table 1	1.21 GB
v2/continuation/ar-100m.pt	AR baseline 100M (same architecture, inserts only)	Table 1	407 MB
v2/continuation/ar-300m.pt	AR baseline 300M	Table 1	1.19 GB
v2/editing/reviser-edit-100m.pt	Reviser-Edit 100M (training seed 1)	Table 5	406 MB
v2/editing/reviser-edit-300m.pt	Reviser-Edit 300M (training seed 1)	Table 5	1.18 GB
v2/editing/ar-rewriter-100m.pt	Matched AR rewriter 100M (training seed 1)	Table 5	406 MB
v2/editing/ar-rewriter-300m.pt	Matched AR rewriter 300M (training seed 1)	Table 5	1.18 GB

v2/SHA256SUMS lists the hashes of the files. v2/manifest.json gives, for each file, its paper row, its number of values and the SHA-256 of the training checkpoint it was made from. That hash is the checkpoint_sha256 recorded with the model outputs in the code repository (samples/continuation/models.json), so every output there can be traced to a file here.

Each v2 file is a PyTorch dictionary that loads with torch.load(..., weights_only=True). It holds the weights (model), a release entry (name, paper, paper row, source hash) and, depending on the model family:

• continuation models: state, the trainer's step and token counters (not in ar-300m.pt);
• editing models: model_config and action_space, which the code needs to build the model, and the training step and counters.

No optimizer state is included.

Use

Install the code repository (pip install -r requirements.txt), then download a file:

from huggingface_hub import hf_hub_download
path = hf_hub_download("sean-diab/reviser-checkpoints",
                       "v2/continuation/reviser-100m-runjump-delete-rl-seed3.pt", local_dir="checkpoints")

Continuation (Table 1). Decode the 3,000 C4 prompts of the paper (35-token prompt, temperature 0.9, top-k 50, at most 220 new actions, prompt locked):

python scripts/papereval/decode_batched.py --ckpt checkpoints/v2/continuation/reviser-100m-runjump-delete-rl-seed3.pt \
  --name rl3 --lock_prompt --out_dir DEC
# 300M models load through the 300M shape shim
python scripts/humanorder/big608_300m/arch_shim.py scripts/papereval/decode_batched.py \
  --ckpt checkpoints/v2/continuation/reviser-300m-runjump-delete-rl-seed3.pt --name rl300 --lock_prompt --out_dir DEC

The AR baselines use the same command without --lock_prompt. --n 10 --device cpu runs a quick test without a GPU.

Editing (Table 5). Type-greedy decoding: take the action type (insert, move, delete or stop) with the largest total probability, then its most likely action. Reviser-Edit starts with the input on the canvas and edits it in place; the AR rewriter writes the output from scratch (--layout ar). The CoEdIT and IteraTeR files are built by python -m reviser_v2.realdata.build (see the code repository).

PYTHONPATH=src python -m reviser_v2.evaluate_edits --data coedit --split validation --type-greedy \
  --ckpt checkpoints/v2/editing/reviser-edit-100m.pt
PYTHONPATH=src python -m reviser_v2.evaluate_edits --data coedit --split validation --type-greedy --layout ar \
  --ckpt checkpoints/v2/editing/ar-rewriter-100m.pt

The paper's outputs were decoded on GPUs with float16 autocast. In full precision on a CPU, a few outputs change where two actions are nearly tied (93 of the first 96 CoEdIT outputs of the four editing models were identical in our test).

Training

All models have 100M (24 layers, width 512) or 300M (26 layers, width 896) parameters and use the GPT-2 tokenizer with extra action tokens.

• First-version Reviser models (top-level files): trained on restoration trajectories of FineWeb text.
• Continuation models (v2/continuation/reviser-*): the first-version models, trained further for 608.7M tokens on the run-and-jump writing order, with or without self-correction (Delete): for each token, with probability 0.05, the writer types a wrong token from nearby text, deletes it right away and types the right one. The training texts are 2,640,539 FineWeb texts; the prompt (35 tokens) is never edited. +RL: 200 steps of group-relative policy-gradient fine-tuning of the self-correction model, with a reward made of the continuation's likelihood under Qwen2.5-3B-Instruct and penalties for repetition, short outputs and extra moves (Appendix D of the paper).
• AR baselines: the same architecture, trained left to right on FineWeb text (first version; no extra training).
• Editing models: Reviser-Edit starts from the first-version Reviser of the same size and the AR rewriter from the AR baseline of the same size. Both are trained on the same examples in the same order (187,797 steps of 64 episodes). The mix contains the CoEdIT and IteraTeR train splits, C4_200M grammar errors, restoration, no-change and synthetic repair examples from C4 and DCLM, and edit drafts that Qwen2.5-32B-Instruct and Claude Haiku 5.5 wrote from clean paragraphs (Appendix C.2). Evaluation items were removed from all training pools.

Results

See Tables 1 and 5 of the paper; the code repository holds every scored output (samples/). For example, at 100M, continued training lowers the GPT-2 evalPPL of the continuations from 32.3 (first version) to 23.2 (run-and-jump), 22.2 (+ Delete) and 18.4 (+ RL); the AR baseline has 13.4. Part of the RL gain in perplexity is the reward itself, which is a likelihood under a Qwen2.5 model.

Limitations

• Small research models (100M and 300M) trained on English web text and public edit corpora. They are not instruction-tuned or tuned for safety, and their outputs can repeat inaccuracies and biases of web text.
• All training orders are synthetic; the edit trajectories are derived from diffs, not recorded keystrokes.
• The model reads its action history, not the current text. Delete is a single-token backspace; the continuation models use it mainly on the token just typed.
• An editing model can make small changes that alter the meaning or attribution of a text.

License

The v2 checkpoints are released under CC BY-NC 4.0 (non-commercial). The RL models were trained with a reward computed by Qwen2.5-3B-Instruct (Qwen Research License), and part of the editing data was written by Qwen2.5-32B-Instruct and Claude Haiku 5.5. The code is Apache-2.0 (see the code repository).

Citation

@misc{diab2026reviser,
  title         = {Reviser: Writing and Editing Text with Autoregressive Cursor Actions},
  author        = {Sean Diab},
  year          = {2026},
  eprint        = {2609.20830},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2609.20830}
}