---
license: other
license_name: openmdw-1.1
base_model: nvidia/Nemotron-3.5-Parse
library_name: transformers
pipeline_tag: image-text-to-text
tags:
- document-parsing
- ocr
- tables
- charts
- ground-truth-sft
- grpo
- verifiable-rewards
- routed-refinement
- thrisul
---
# Thrisul-Parser (v2)

A derivative of [`nvidia/Nemotron-3.5-Parse`](https://huggingface.co/nvidia/Nemotron-3.5-Parse) (631M parameters).
v2 replaces v1's self-distillation with a **three-stage quality pipeline**: (1) **ground-truth SFT** whose targets are *exact annotations serialized in
the base model's own output format* (never the model's own outputs - this removes the imitation ceiling), (2) **GRPO with verifiable rewards**
(reward computed from ground truth: table TEDS, chart numeric-F1, format validity, bbox IoU), and (3) **verifier-routed refinement** re-labelled on the
final policy. Every P0 defect found in the v1 audit is addressed; the audit trail lives in the training notebook.

## What is new vs the base model
1. **GT-SFT (weights).** LoRA r=32 (decoder linears) trained on exact ground truth: procedurally rendered tables/charts/full pages with pixel-exact
   supervision, the **Thrisul-Doc D1-D4 custom corpora** (complex-table archetypes, hard charts, degradation curriculum, multi-column/financial/
   adversarial pages - all exact-GT by construction), 300 real ChartNet charts (CSV truth, CDLA-2.0), 0 FinTabNet.c financial tables (HTML
   truth, CDLA-2.0), and GT-serialized text-page replay. Targets were discovered to round-trip through the base model's own `postprocess_document`
   (mode: nemotron-parse) before training.
   Numeric tokens up-weighted (1+0.5x); inverse-sqrt length weighting; checkpoint selected on held-out structure scores, not "last".
2. **GRPO with verifiable rewards (weights).** 303 GRPO groups mined from hard buckets (lambda per blueprint 7.4). Group-normalized advantages (G rollouts/prompt), k3 KL to the SFT reference, prompts
   mined from the SFT policy's hard failures. Reward = (0.4, 0.25, 0.15, 0.15, 0.05) over (table, chart, structure validity, bbox IoU, EOS).
3. **Verifier-routed refinement (inference).** The parser runs once, a gradient-boosted router scores each table/chart region
   (decoder-token uncertainty + structure sanity + render-vs-crop similarity), and only regions predicted to gain are re-decoded at native
   resolution. Router labels were rebuilt on the FINAL policy (v1 fitted them on the base model - distribution shift fixed). Code ships with the
   model: `thrisul_pipeline.py` (`parse_page(..., mode="routed")`), `thrisul_router.joblib`.

## Training data (exact-GT or permissively licensed; NO benchmark pages were used for training)
- Procedurally rendered tables (merged cells, multi-level headers, numeric/currency/percent columns), charts (bar/hbar/line/pie/stacked), and
  full synthetic pages (title/sections/paragraphs/table/chart) with pixel-exact block annotations - generated by `thrisul_pipeline.py` itself.
- **Thrisul-Doc D1-D4 custom corpora (this repo, `thrisul_pipeline.py` v2.1, exact GT by construction, seeds disclosed in `thrisul_config.json`):**
  **D1** complex-table archetypes - 3-level header trees, colspan+rowspan merged cells, borderless layouts, unit rows, multi-index row labels,
  decimal look-alikes, zero/dash columns; **D2** hard charts - dual-axis bar+line, log-scale, 4-series crowded legends, negative bars, 100%-stacked
  (segments sum to exactly 100.0), irregular long categories; **D3** degradation curriculum - JPEG/blur/downscale/rotate/shadow/tone/occlusion/noise
  in tiers 1-3 applied to D1/D2/synthetic parents whose ground truth is inherited unchanged; **D4** multi-column + financial + adversarial pages -
  2-column reading-order ground truth, financial statements with parenthesized negatives and notes, decimal-shift look-alike traps, footer-lookalike
  text. Held-out eval slices for every D domain use seed ranges disjoint from training (61M/62M/63M/64M vs 6.1M-6.4M).
- Real charts from ChartNet (`ibm-granite/ChartNet`, CSV ground truth, CDLA-Permissive-2.0).
- Real financial tables from FinTabNet.c (`bsmock/FinTabNet.c`, HTML ground truth, CDLA-Permissive-2.0).
- Compute: Kaggle free tier (T4), LoRA, 1583 SFT optimizer steps + 303 GRPO groups. Targets: 6330 (~533 tokens mean).

<!-- THRISUL-BENCH:BEGIN -->
## Official benchmark results (thrisul-parser)

Measured 2026-10-31 on Kaggle 2×T4 (Transformers, fp16, greedy) by this benchmark notebook. Run tag: `v1`. Coverage: ParseBench 60 pages/dimension (official total 2,078) · olmOCR-Bench 400 pages (official total = all unique bench pages) · OmniDocBench: packaged · Extra suites: IndicVisionBench=53.05217098342906, MOSCAR=pending.

> Sampling disclosure: these runs cover a **seeded stratified subset** unless the config covered every page. Partial coverage is labeled — never quoted as the full-suite official score. Base-model deltas on identical pages are in `results.json`.

### ParseBench (official evaluator, sampled)

| group | score |
|---|---|
| chart | 0.206 |
| table | 0.168 |
| text_formatting | 0.037 |
| text_content | 0.491 |
| text | 0.091 |
| overall_rule_avg | 0.145 |
| **mean of table/chart/text** | **0.155** |

### olmOCR-Bench (official scorer, covered subset)

| category | pass rate |
|---|---|
| Overall | 29.5 |
| CI95_half_width | 1.5 |
| absent | 61.2 |
| baseline | 98.5 |
| math | 4.9 |
| order | 14.0 |
| present | 15.1 |
| table | 18.0 |
| arxiv_math | 7.6 |
| headers_footers | 56.7 |
| long_tiny_text | 14.0 |
| multi_column | 13.5 |
| old_scans | 24.8 |
| old_scans_math | 2.8 |
| table_tests | 18.3 |

### IndicVisionBench OCR + other suites (our protocol, official data)

| suite | metric | n | score |
|---|---|---|---|
| IndicVisionBench | ANLS-C | 862 | 53.05 |

> Metrics computed by this notebook on the official data with the IndicVisionBench paper's OCR protocol (arXiv 2511.04727, following OCRBenchv2: ANLS-C/ANLS-W/CER/WER) — not the authors' official harness; per-language breakdown lives in `results.json` (`indicvisionbench.agg`).


### Context A — olmOCR-Bench (pass rates, higher is better)

| System | Protocol | arxiv_math | old_scans_math | tables | old_scans | headers_footers | multi_column | long_tiny_text | baseline | Overall |
|---|---|---|---|---|---|---|---|---|---|---|
| **thrisul-parser v2.1** | **this run, 400-page covered subset** | 7.6 | 2.8 | 18.3 | 24.8 | **56.7** | 13.5 | 14.0 | 98.5 | **29.5 ± 1.5** |
| Nemotron-3.5-Parse (base) | **this run, identical pages** | 42.6 | 10.9 | 85.8 | 36.5 | 18.3 | 79.6 | 69.4 | 99.8 | **55.4 ± 1.6** |
| Chandra OCR 0.1.0* | published, full corpus | 82.2 | 80.3 | 88.0 | 50.4 | 90.8 | 81.2 | 92.3 | 99.9 | 83.1 ± 0.9 |
| olmOCR v0.4.0 | published, full corpus | 83.0 | 82.3 | 84.9 | 47.7 | 96.1 | 83.7 | 81.9 | 99.7 | 82.4 ± 1.1 |
| Infinity-Parser 7B* | published, full corpus | 84.4 | 83.8 | 85.0 | 47.9 | 88.7 | 84.2 | 86.4 | 99.8 | 82.5 |
| PaddleOCR-VL* | published, full corpus | 85.7 | 71.0 | 84.1 | 37.8 | 97.0 | 79.9 | 85.7 | 98.5 | 80.0 ± 1.0 |
| Marker 1.10.1 | published, full corpus | 83.8 | 66.8 | 72.9 | 33.5 | 86.6 | 80.0 | 85.7 | 99.3 | 76.1 ± 1.1 |
| DeepSeek-OCR | published, full corpus | 77.2 | 73.6 | 80.2 | 33.3 | 96.1 | 66.4 | 79.4 | 99.8 | 75.7 ± 1.0 |
| MinerU 2.5.4* | published, full corpus | 76.6 | 54.6 | 84.9 | 33.7 | 96.6 | 78.2 | 83.5 | 93.7 | 75.2 ± 1.1 |
| Mistral OCR API | published, full corpus | 77.2 | 67.5 | 60.6 | 29.3 | 93.6 | 71.3 | 77.1 | 99.4 | 72.0 ± 1.1 |
| Nanonets-OCR2-3B | published, full corpus | 75.4 | 46.1 | 86.8 | 40.9 | 32.1 | 81.9 | 93.0 | 99.6 | 69.5 ± 1.1 |

Published rows: [allenai/olmocr README leaderboard](https://github.com/allenai/olmocr), table dated 2025-10-21 (olmOCR v0.4.0). `*` = not fully open source per that README. The two "this run" rows share one protocol and are comparable **only to each other**; the gap to the published rows mostly reflects the covered subset, T4-class hardware, and our candidate generation — it is **not** a claim that those systems are 45+ points ahead of the base model.

### Context B — ParseBench (published 0–100 aggregate, self-reported)

| System | ParseBench (published) |
|---|---|
| LlamaParse Agentic Plus (API) | 88.8 |
| LlamaParse Agentic (API) | 83.07 |
| Anthropic Opus 5.5 (Effort High) | 78.9 |
| oi-parser | 78.3 |
| LightOnOCR-3-4B | 75.1 |
| LightOnOCR-3-0.8B | 74.6 |
| Infinity-Parser2-Pro (35B) | 74.3 |
| Infinity-Parser2-Flash | 73.25 |
| **Nemotron-3.5-Parse (base of this model; single/double/triple pass)** | **72.74 / 74.92 / 75.20** |
| MinerU2.5-Pro-2605-1.2B | 72.78 |
| KDL-Frontier-Parser-nano | 72.38 |
| Chandra-ocr-2 | 70.1 |
| PaddleOCR-VL-1.6 | 67.43 |
| Nemotron-Parse-2.0 | 63.91 |
| Google Gemini 3 Flash (Thinking Minimal) | 71.04 |
| OpenAI GPT-5 Mini (Reasoning Medium) | 51.52 |
| **thrisul-parser v2.1** | sampled official-protocol run: 0.145 overall_rule_avg / 0.155 mean-of-3 (different aggregation — **not** the 0–100 leaderboard scale); official full-corpus run pending |

Sources: parsebench.ai leaderboard (self-reported), as captured in this repo's `results.json` (`published_parsebench`). The base model ranks #1 among sub-1B parsers on that leaderboard.

### Context C — the wider parser landscape (qualitative)

| Parser | Access | Scale | License | Notes |
|---|---|---|---|---|
| **thrisul-parser v2.1** | open weights | 631M (+2.8M LoRA) | OpenMDW 1.1 (via base) | GT-SFT + GRPO + verifier-routed refinement; this repo |
| Nemotron-3.5-Parse (base) | open weights | 631M | OpenMDW 1.1 | bbox + classes + markdown output; #1 sub-1B on ParseBench |
| Nougat | open weights | ~250M–350M | MIT | academic PDFs; LaTeX output; no layout coordinates |
| GOT-OCR 2.0 | open weights | ~580M | MIT | "general OCR theory"; unified formulas/tables/text |
| olmOCR | open weights | ~7B | Apache-2.0 | PDF → plain markdown; strong on olmOCR-Bench by construction |
| Marker | open weights | pipeline+DL | GPL-3.0 | fast PDF→markdown; commercial use restricted |
| MinerU 2.5 | open weights | 1.2B (Pro) | AGPL-3.0 | layout+recognition pipeline; strong OmniDocBench scores |
| Docling | open toolkit | — | MIT | IBM; layout models + table structure; heavy CPU-friendly pipeline |
| Surya | open weights | — | GPL-3.0 | OCR/layout/reading-order toolkit (Datalab) |
| DeepSeek-OCR | open weights | ~3B | MIT | "contexts optical compression"; long-doc focus |
| PaddleOCR-VL | open weights | 0.9B | Apache-2.0 | strong olmOCR-Bench full-corpus score |
| Chandra | open weights | — | see repo¹ | Datalab; top published olmOCR-Bench row above |
| Mistral OCR | API only | — | commercial | strong tables; per-page pricing |
| LlamaParse | API only | — | commercial | multiple modes; ParseBench head rows above |
- Google Gemini 3 Flash (Thinking Minimal): 71.04
- 
<!-- THRISUL-BENCH:END -->

## Results (read the caveats)
All numbers below were measured by us on **held-out synthetic data and a seeded ParseBench-proxy sample on a T4 with Transformers** (no vLLM).
They are **not** the official ParseBench score and are **not comparable** to H100/vLLM numbers anywhere below. Headline deltas carry bootstrap 95% CIs.

Held-out synthetic regions (page-scale input; `routed` re-decodes only router-selected regions):

| subset | system | score | refine_frac | n |
|---|---|---|---|---|
| all | thr_routed | 0.952 | 1.000 | 226 |
| all | base_page | 0.766 | 0.000 | 226 |
| all | base_native | 0.939 | 1.000 | 226 |
| all | thr_page | 0.722 | 0.000 | 226 |
| all | thr_native | 0.952 | 1.000 | 226 |
| table | thr_routed | 0.964 | 1.000 | 128 |
| table | base_page | 0.813 | 0.000 | 128 |
| table | base_native | 0.975 | 1.000 | 128 |
| table | thr_page | 0.722 | 0.000 | 128 |
| table | thr_native | 0.964 | 1.000 | 128 |
| chart | thr_routed | 0.936 | 1.000 | 98 |
| chart | base_page | 0.704 | 0.000 | 98 |
| chart | base_native | 0.892 | 1.000 | 98 |
| chart | thr_page | 0.721 | 0.000 | 98 |
| chart | thr_native | 0.936 | 1.000 | 98 |
| d1_complex_tables | thr_routed | 0.912 | 1.000 | 30 |
| d1_complex_tables | base_page | 0.743 | 0.000 | 30 |
| d1_complex_tables | base_native | 0.937 | 1.000 | 30 |
| d1_complex_tables | thr_page | 0.678 | 0.000 | 30 |
| d1_complex_tables | thr_native | 0.912 | 1.000 | 30 |
| d2_hard_charts | thr_routed | 0.951 | 1.000 | 30 |
| d2_hard_charts | base_page | 0.594 | 0.000 | 30 |
| d2_hard_charts | base_native | 0.909 | 1.000 | 30 |
| d2_hard_charts | thr_page | 0.600 | 0.000 | 30 |
| d2_hard_charts | thr_native | 0.951 | 1.000 | 30 |
| d3_degraded | thr_routed | 0.863 | 1.000 | 16 |
| d3_degraded | base_page | 0.720 | 0.000 | 16 |
| d3_degraded | base_native | 0.911 | 1.000 | 16 |
| d3_degraded | thr_page | 0.653 | 0.000 | 16 |
| d3_degraded | thr_native | 0.863 | 1.000 | 16 |

Full synthetic pages through the real pipeline (table score, mean-over-tables blended with best-table):

| system | table_score | sec_per_page | pages_per_s | n |
|---|---|---|---|---|
| base_single | 0.995 | 5.878 | 0.170 | 32 |
| base_double | 0.991 | 8.488 | 0.118 | 32 |
| thr_single | 0.914 | 8.985 | 0.111 | 32 |
| thr_routed | 0.740 | 13.248 | 0.075 | 32 |

ParseBench-proxy on real enterprise PDFs (seeded stratified sample; crude proxy, small n):

not run (dataset load failed in this session)

Verifier ablation (out-of-fold, final policy):

| features | spearman | auc | refine_frac_for_90pct_gain | gain_captured_at_30pct |
|---|---|---|---|---|
| entropy | 0.591 | 0.813 | 0.876 | 0.656 |
| all (entropy+structure+render) | 0.595 | 0.814 | 0.823 | 0.634 |
| entropy+structure | 0.588 | 0.821 | 0.783 | 0.629 |
| render only | 0.390 | 0.678 | 0.916 | 0.531 |
| structure | 0.108 | 0.569 | 0.960 | 0.355 |

Pre-registered verdicts measured this run:

FAIL  E1' quality: Thrisul single-pass > base single-pass (held-out regions)
PASS  E2' efficiency: routed ≈ base double-pass quality at partial refine cost
FAIL  E1'' quality: Thrisul single-pass > base single-pass (full pages)

Text regression check (Thrisul-single vs base-single page markdown similarity): 77.2 %.

### Published reference numbers (self-reported by their authors; different protocol/hardware - context only)
- LlamaParse Agentic Plus (API): 88.8
- Anthropic Opus 5.5 (Effort High): 78.9
- oi-parser: 78.3
- Nemotron-3.5-Parse (single / double / triple pass) = Thrisul base: 72.74 / 74.92 / 75.20
- LightOnOCR-3-4B: 75.1
- LightOnOCR-3-0.8B: 74.6
- Infinity-Parser2-Pro (35B): 74.3
- Infinity-Parser2-Flash: 73.25
- MinerU2.5-Pro-2605-1.2B: 72.78
- KDL-Frontier-Parser-nano: 72.38
- Chandra-ocr-2: 70.1
- PaddleOCR-VL-1.6: 67.43
- PaddleOCR-VL-1.5: 65.95
- Nemotron-Parse-2.0: 63.91

### Leaderboard template row (fill `Official score` after the locked external ParseBench run)
| Field | Value |
|---|---|
| Model version | sandeeprdy1729/thrisul-parser (merged sha in `thrisul_config.json`) |
| Base checkpoint | nvidia/Nemotron-3.5-Parse @ 4035ffb1d8 |
| Params / trainable | 631M / 2.8M LoRA |
| Inference settings | single vs routed vs double; max_side=1664; greedy; rep-pen 1.1; max_new_tokens 3072 |
| Official score | _to fill after external locked run_ |
| Measured here | region/page/proxy tables above (T4, Transformers) |
| Date | 2026-10-30 |

## Usage
```python
import torch, sys
from huggingface_hub import snapshot_download
from transformers import AutoModel, AutoProcessor, GenerationConfig
path = snapshot_download("sandeeprdy1729/thrisul-parser")
sys.path.insert(0, path)
from postprocessing import postprocess_document          # shipped from the base model
import thrisul_pipeline as tp
model = AutoModel.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).cuda().eval()   # use float16 on T4
processor = AutoProcessor.from_pretrained(path, trust_remote_code=True)
gen_cfg = GenerationConfig.from_pretrained(path)
router = tp.Router.load(path + "/thrisul_router.joblib")
post = lambda text, image_size: postprocess_document(text, image_size=image_size)
out = tp.parse_page(model, processor, gen_cfg, image, post, mode="routed", router=router)   # or mode="single" / "all"
print(out["md"])
```

## Limitations (honest)
- Real-document gains beyond the proxy above are unproven; the official ParseBench/OmniDocBench/olmOCR-Bench runs are external and pending.
- The router is fitted on synthetic crops including the D1-D3 domains plus ChartNet/FinTabNet; distribution shift on unseen real layouts remains
  possible (pages where region mapping fails silently fall back to single pass - counted in `parse_page` info).
- GRPO was time-boxed on free T4s; longer reward optimization is the most promising next lever (see the notebook's backlog).
- Coordinates in the base model are content-normalized; the GT serializer's round-trip is verified in-repo (`thrisul_config.json` records the mode).
- Evaluated on a T4 with Transformers; throughput numbers do not transfer to H100/vLLM.

## License and attribution
Derivative work of NVIDIA Nemotron 3.5 Parse; use is governed by the base model's license (OpenMDW 1.1) and the third-party notices in this
repository. Training data: own synthetic renderers and the Thrisul-Doc D1-D4 corpora (this repo), ChartNet and FinTabNet.c under
CDLA-Permissive-2.0. See `LICENSE` and the notice files copied from the base repo.