Thrisul-Parser (v2)

A derivative of nvidia/Nemotron-3.5-Parse (631M parameters). v2 replaces v1's self-distillation with a three-stage quality pipeline: (1) ground-truth SFT whose targets are exact annotations serialized in the base model's own output format (never the model's own outputs - this removes the imitation ceiling), (2) GRPO with verifiable rewards (reward computed from ground truth: table TEDS, chart numeric-F1, format validity, bbox IoU), and (3) verifier-routed refinement re-labelled on the final policy. Every P0 defect found in the v1 audit is addressed; the audit trail lives in the training notebook.

What is new vs the base model

1. GT-SFT (weights). LoRA r=32 (decoder linears) trained on exact ground truth: procedurally rendered tables/charts/full pages with pixel-exact supervision, the Thrisul-Doc D1-D4 custom corpora (complex-table archetypes, hard charts, degradation curriculum, multi-column/financial/ adversarial pages - all exact-GT by construction), 300 real ChartNet charts (CSV truth, CDLA-2.0), 0 FinTabNet.c financial tables (HTML truth, CDLA-2.0), and GT-serialized text-page replay. Targets were discovered to round-trip through the base model's own postprocess_document (mode: nemotron-parse) before training. Numeric tokens up-weighted (1+0.5x); inverse-sqrt length weighting; checkpoint selected on held-out structure scores, not "last".
2. GRPO with verifiable rewards (weights). 303 GRPO groups mined from hard buckets (lambda per blueprint 7.4). Group-normalized advantages (G rollouts/prompt), k3 KL to the SFT reference, prompts mined from the SFT policy's hard failures. Reward = (0.4, 0.25, 0.15, 0.15, 0.05) over (table, chart, structure validity, bbox IoU, EOS).
3. Verifier-routed refinement (inference). The parser runs once, a gradient-boosted router scores each table/chart region (decoder-token uncertainty + structure sanity + render-vs-crop similarity), and only regions predicted to gain are re-decoded at native resolution. Router labels were rebuilt on the FINAL policy (v1 fitted them on the base model - distribution shift fixed). Code ships with the model: thrisul_pipeline.py (parse_page(..., mode="routed")), thrisul_router.joblib.

Training data (exact-GT or permissively licensed; NO benchmark pages were used for training)

• Procedurally rendered tables (merged cells, multi-level headers, numeric/currency/percent columns), charts (bar/hbar/line/pie/stacked), and full synthetic pages (title/sections/paragraphs/table/chart) with pixel-exact block annotations - generated by thrisul_pipeline.py itself.
• Thrisul-Doc D1-D4 custom corpora (this repo, thrisul_pipeline.py v2.1, exact GT by construction, seeds disclosed in thrisul_config.json): D1 complex-table archetypes - 3-level header trees, colspan+rowspan merged cells, borderless layouts, unit rows, multi-index row labels, decimal look-alikes, zero/dash columns; D2 hard charts - dual-axis bar+line, log-scale, 4-series crowded legends, negative bars, 100%-stacked (segments sum to exactly 100.0), irregular long categories; D3 degradation curriculum - JPEG/blur/downscale/rotate/shadow/tone/occlusion/noise in tiers 1-3 applied to D1/D2/synthetic parents whose ground truth is inherited unchanged; D4 multi-column + financial + adversarial pages - 2-column reading-order ground truth, financial statements with parenthesized negatives and notes, decimal-shift look-alike traps, footer-lookalike text. Held-out eval slices for every D domain use seed ranges disjoint from training (61M/62M/63M/64M vs 6.1M-6.4M).
• Real charts from ChartNet (ibm-granite/ChartNet, CSV ground truth, CDLA-Permissive-2.0).
• Real financial tables from FinTabNet.c (bsmock/FinTabNet.c, HTML ground truth, CDLA-Permissive-2.0).
• Compute: Kaggle free tier (T4), LoRA, 1583 SFT optimizer steps + 303 GRPO groups. Targets: 6330 (~533 tokens mean).

Official benchmark results (thrisul-parser)

Measured 2026-10-31 on Kaggle 2×T4 (Transformers, fp16, greedy) by this benchmark notebook. Run tag: v1. Coverage: ParseBench 60 pages/dimension (official total 2,078) · olmOCR-Bench 400 pages (official total = all unique bench pages) · OmniDocBench: packaged · Extra suites: IndicVisionBench=53.05217098342906, MOSCAR=pending.

Sampling disclosure: these runs cover a seeded stratified subset unless the config covered every page. Partial coverage is labeled — never quoted as the full-suite official score. Base-model deltas on identical pages are in results.json.

ParseBench (official evaluator, sampled)

group	score
chart	0.206
table	0.168
text_formatting	0.037
text_content	0.491
text	0.091
overall_rule_avg	0.145
mean of table/chart/text	0.155

olmOCR-Bench (official scorer, covered subset)

category	pass rate
Overall	29.5
CI95_half_width	1.5
absent	61.2
baseline	98.5
math	4.9
order	14.0
present	15.1
table	18.0
arxiv_math	7.6
headers_footers	56.7
long_tiny_text	14.0
multi_column	13.5
old_scans	24.8
old_scans_math	2.8
table_tests	18.3

IndicVisionBench OCR + other suites (our protocol, official data)

suite	metric	n	score
IndicVisionBench	ANLS-C	862	53.05

Metrics computed by this notebook on the official data with the IndicVisionBench paper's OCR protocol (arXiv 2511.04727, following OCRBenchv2: ANLS-C/ANLS-W/CER/WER) — not the authors' official harness; per-language breakdown lives in results.json (indicvisionbench.agg).

Context A — olmOCR-Bench (pass rates, higher is better)

System	Protocol	arxiv_math	old_scans_math	tables	old_scans	headers_footers	multi_column	long_tiny_text	baseline	Overall
thrisul-parser v2.1	this run, 400-page covered subset	7.6	2.8	18.3	24.8	56.7	13.5	14.0	98.5	29.5 ± 1.5
Nemotron-3.5-Parse (base)	this run, identical pages	42.6	10.9	85.8	36.5	18.3	79.6	69.4	99.8	55.4 ± 1.6
Chandra OCR 0.1.0*	published, full corpus	82.2	80.3	88.0	50.4	90.8	81.2	92.3	99.9	83.1 ± 0.9
olmOCR v0.4.0	published, full corpus	83.0	82.3	84.9	47.7	96.1	83.7	81.9	99.7	82.4 ± 1.1
Infinity-Parser 7B*	published, full corpus	84.4	83.8	85.0	47.9	88.7	84.2	86.4	99.8	82.5
PaddleOCR-VL*	published, full corpus	85.7	71.0	84.1	37.8	97.0	79.9	85.7	98.5	80.0 ± 1.0
Marker 1.10.1	published, full corpus	83.8	66.8	72.9	33.5	86.6	80.0	85.7	99.3	76.1 ± 1.1
DeepSeek-OCR	published, full corpus	77.2	73.6	80.2	33.3	96.1	66.4	79.4	99.8	75.7 ± 1.0
MinerU 2.5.4*	published, full corpus	76.6	54.6	84.9	33.7	96.6	78.2	83.5	93.7	75.2 ± 1.1
Mistral OCR API	published, full corpus	77.2	67.5	60.6	29.3	93.6	71.3	77.1	99.4	72.0 ± 1.1
Nanonets-OCR2-3B	published, full corpus	75.4	46.1	86.8	40.9	32.1	81.9	93.0	99.6	69.5 ± 1.1

Published rows: allenai/olmocr README leaderboard, table dated 2025-10-21 (olmOCR v0.4.0). * = not fully open source per that README. The two "this run" rows share one protocol and are comparable only to each other; the gap to the published rows mostly reflects the covered subset, T4-class hardware, and our candidate generation — it is not a claim that those systems are 45+ points ahead of the base model.

Context B — ParseBench (published 0–100 aggregate, self-reported)

System	ParseBench (published)
LlamaParse Agentic Plus (API)	88.8
LlamaParse Agentic (API)	83.07
Anthropic Opus 5.5 (Effort High)	78.9
oi-parser	78.3
LightOnOCR-3-4B	75.1
LightOnOCR-3-0.8B	74.6
Infinity-Parser2-Pro (35B)	74.3
Infinity-Parser2-Flash	73.25
Nemotron-3.5-Parse (base of this model; single/double/triple pass)	72.74 / 74.92 / 75.20
MinerU2.5-Pro-2605-1.2B	72.78
KDL-Frontier-Parser-nano	72.38
Chandra-ocr-2	70.1
PaddleOCR-VL-1.6	67.43
Nemotron-Parse-2.0	63.91
Google Gemini 3 Flash (Thinking Minimal)	71.04
OpenAI GPT-5 Mini (Reasoning Medium)	51.52
thrisul-parser v2.1	sampled official-protocol run: 0.145 overall_rule_avg / 0.155 mean-of-3 (different aggregation — not the 0–100 leaderboard scale); official full-corpus run pending

Sources: parsebench.ai leaderboard (self-reported), as captured in this repo's results.json (published_parsebench). The base model ranks #1 among sub-1B parsers on that leaderboard.

Context C — the wider parser landscape (qualitative)

Parser	Access	Scale	License	Notes
thrisul-parser v2.1	open weights	631M (+2.8M LoRA)	OpenMDW 1.1 (via base)	GT-SFT + GRPO + verifier-routed refinement; this repo
Nemotron-3.5-Parse (base)	open weights	631M	OpenMDW 1.1	bbox + classes + markdown output; #1 sub-1B on ParseBench
Nougat	open weights	~250M–350M	MIT	academic PDFs; LaTeX output; no layout coordinates
GOT-OCR 2.0	open weights	~580M	MIT	"general OCR theory"; unified formulas/tables/text
olmOCR	open weights	~7B	Apache-2.0	PDF → plain markdown; strong on olmOCR-Bench by construction
Marker	open weights	pipeline+DL	GPL-3.0	fast PDF→markdown; commercial use restricted
MinerU 2.5	open weights	1.2B (Pro)	AGPL-3.0	layout+recognition pipeline; strong OmniDocBench scores
Docling	open toolkit	—	MIT	IBM; layout models + table structure; heavy CPU-friendly pipeline
Surya	open weights	—	GPL-3.0	OCR/layout/reading-order toolkit (Datalab)
DeepSeek-OCR	open weights	~3B	MIT	"contexts optical compression"; long-doc focus
PaddleOCR-VL	open weights	0.9B	Apache-2.0	strong olmOCR-Bench full-corpus score
Chandra	open weights	—	see repo¹	Datalab; top published olmOCR-Bench row above
Mistral OCR	API only	—	commercial	strong tables; per-page pricing
LlamaParse	API only	—	commercial	multiple modes; ParseBench head rows above

• Google Gemini 3 Flash (Thinking Minimal): 71.04

Results (read the caveats)

All numbers below were measured by us on held-out synthetic data and a seeded ParseBench-proxy sample on a T4 with Transformers (no vLLM). They are not the official ParseBench score and are not comparable to H100/vLLM numbers anywhere below. Headline deltas carry bootstrap 95% CIs.

Held-out synthetic regions (page-scale input; routed re-decodes only router-selected regions):

subset	system	score	refine_frac	n
all	thr_routed	0.952	1.000	226
all	base_page	0.766	0.000	226
all	base_native	0.939	1.000	226
all	thr_page	0.722	0.000	226
all	thr_native	0.952	1.000	226
table	thr_routed	0.964	1.000	128
table	base_page	0.813	0.000	128
table	base_native	0.975	1.000	128
table	thr_page	0.722	0.000	128
table	thr_native	0.964	1.000	128
chart	thr_routed	0.936	1.000	98
chart	base_page	0.704	0.000	98
chart	base_native	0.892	1.000	98
chart	thr_page	0.721	0.000	98
chart	thr_native	0.936	1.000	98
d1_complex_tables	thr_routed	0.912	1.000	30
d1_complex_tables	base_page	0.743	0.000	30
d1_complex_tables	base_native	0.937	1.000	30
d1_complex_tables	thr_page	0.678	0.000	30
d1_complex_tables	thr_native	0.912	1.000	30
d2_hard_charts	thr_routed	0.951	1.000	30
d2_hard_charts	base_page	0.594	0.000	30
d2_hard_charts	base_native	0.909	1.000	30
d2_hard_charts	thr_page	0.600	0.000	30
d2_hard_charts	thr_native	0.951	1.000	30
d3_degraded	thr_routed	0.863	1.000	16
d3_degraded	base_page	0.720	0.000	16
d3_degraded	base_native	0.911	1.000	16
d3_degraded	thr_page	0.653	0.000	16
d3_degraded	thr_native	0.863	1.000	16

Full synthetic pages through the real pipeline (table score, mean-over-tables blended with best-table):

system	table_score	sec_per_page	pages_per_s	n
base_single	0.995	5.878	0.170	32
base_double	0.991	8.488	0.118	32
thr_single	0.914	8.985	0.111	32
thr_routed	0.740	13.248	0.075	32

ParseBench-proxy on real enterprise PDFs (seeded stratified sample; crude proxy, small n):

not run (dataset load failed in this session)

Verifier ablation (out-of-fold, final policy):

features	spearman	auc	refine_frac_for_90pct_gain	gain_captured_at_30pct
entropy	0.591	0.813	0.876	0.656
all (entropy+structure+render)	0.595	0.814	0.823	0.634
entropy+structure	0.588	0.821	0.783	0.629
render only	0.390	0.678	0.916	0.531
structure	0.108	0.569	0.960	0.355

Pre-registered verdicts measured this run:

FAIL E1' quality: Thrisul single-pass > base single-pass (held-out regions) PASS E2' efficiency: routed ≈ base double-pass quality at partial refine cost FAIL E1'' quality: Thrisul single-pass > base single-pass (full pages)

Text regression check (Thrisul-single vs base-single page markdown similarity): 77.2 %.

Published reference numbers (self-reported by their authors; different protocol/hardware - context only)

• LlamaParse Agentic Plus (API): 88.8
• Anthropic Opus 5.5 (Effort High): 78.9
• oi-parser: 78.3
• Nemotron-3.5-Parse (single / double / triple pass) = Thrisul base: 72.74 / 74.92 / 75.20
• LightOnOCR-3-4B: 75.1
• LightOnOCR-3-0.8B: 74.6
• Infinity-Parser2-Pro (35B): 74.3
• Infinity-Parser2-Flash: 73.25
• MinerU2.5-Pro-2605-1.2B: 72.78
• KDL-Frontier-Parser-nano: 72.38
• Chandra-ocr-2: 70.1
• PaddleOCR-VL-1.6: 67.43
• PaddleOCR-VL-1.5: 65.95
• Nemotron-Parse-2.0: 63.91

Leaderboard template row (fill Official score after the locked external ParseBench run)

Field	Value
Model version	sandeeprdy1729/thrisul-parser (merged sha in thrisul_config.json)
Base checkpoint	nvidia/Nemotron-3.5-Parse @ 4035ffb1d8
Params / trainable	631M / 2.8M LoRA
Inference settings	single vs routed vs double; max_side=1664; greedy; rep-pen 1.1; max_new_tokens 3072
Official score	to fill after external locked run
Measured here	region/page/proxy tables above (T4, Transformers)
Date	2026-10-30

Usage

import torch, sys
from huggingface_hub import snapshot_download
from transformers import AutoModel, AutoProcessor, GenerationConfig
path = snapshot_download("sandeeprdy1729/thrisul-parser")
sys.path.insert(0, path)
from postprocessing import postprocess_document          # shipped from the base model
import thrisul_pipeline as tp
model = AutoModel.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).cuda().eval()   # use float16 on T4
processor = AutoProcessor.from_pretrained(path, trust_remote_code=True)
gen_cfg = GenerationConfig.from_pretrained(path)
router = tp.Router.load(path + "/thrisul_router.joblib")
post = lambda text, image_size: postprocess_document(text, image_size=image_size)
out = tp.parse_page(model, processor, gen_cfg, image, post, mode="routed", router=router)   # or mode="single" / "all"
print(out["md"])

Limitations (honest)

• Real-document gains beyond the proxy above are unproven; the official ParseBench/OmniDocBench/olmOCR-Bench runs are external and pending.
• The router is fitted on synthetic crops including the D1-D3 domains plus ChartNet/FinTabNet; distribution shift on unseen real layouts remains possible (pages where region mapping fails silently fall back to single pass - counted in parse_page info).
• GRPO was time-boxed on free T4s; longer reward optimization is the most promising next lever (see the notebook's backlog).
• Coordinates in the base model are content-normalized; the GT serializer's round-trip is verified in-repo (thrisul_config.json records the mode).
• Evaluated on a T4 with Transformers; throughput numbers do not transfer to H100/vLLM.

License and attribution

Derivative work of NVIDIA Nemotron 3.5 Parse; use is governed by the base model's license (OpenMDW 1.1) and the third-party notices in this repository. Training data: own synthetic renderers and the Thrisul-Doc D1-D4 corpora (this repo), ChartNet and FinTabNet.c under CDLA-Permissive-2.0. See LICENSE and the notice files copied from the base repo.