Hugging Face's logo

sandeeprdy1729
/
thrisul-parser

Image-Text-to-Text
Transformers
Joblib
Safetensors
nemotron_parse
text-generation
document-parsing
ocr
tables
charts
ground-truth-sft
grpo
verifiable-rewards
routed-refinement
thrisul
conversational
custom_code
Model card Files
xet
 Community
thrisul-parser / README.md
sandeeprdy1729's picture
sandeeprdy1729
updated model card
62bb8ea verified
preview code
|
Raw History Blame Contribute Delete
17.3 kB
metadata 
license: other
license_name: openmdw-1.1
base_model: nvidia/Nemotron-3.5-Parse
library_name: transformers
pipeline_tag: image-text-to-text
tags:
  - document-parsing
  - ocr
  - tables
  - charts
  - ground-truth-sft
  - grpo
  - verifiable-rewards
  - routed-refinement
  - thrisul

Thrisul-Parser (v2)

A derivative of nvidia/Nemotron-3.5-Parse (631M parameters). v2 replaces v1's self-distillation with a three-stage quality pipeline: (1) ground-truth SFT whose targets are exact annotations serialized in the base model's own output format (never the model's own outputs - this removes the imitation ceiling), (2) GRPO with verifiable rewards (reward computed from ground truth: table TEDS, chart numeric-F1, format validity, bbox IoU), and (3) verifier-routed refinement re-labelled on the final policy. Every P0 defect found in the v1 audit is addressed; the audit trail lives in the training notebook.

What is new vs the base model

  1. GT-SFT (weights). LoRA r=32 (decoder linears) trained on exact ground truth: procedurally rendered tables/charts/full pages with pixel-exact supervision, the Thrisul-Doc D1-D4 custom corpora (complex-table archetypes, hard charts, degradation curriculum, multi-column/financial/ adversarial pages - all exact-GT by construction), 300 real ChartNet charts (CSV truth, CDLA-2.0), 0 FinTabNet.c financial tables (HTML truth, CDLA-2.0), and GT-serialized text-page replay. Targets were discovered to round-trip through the base model's own postprocess_document (mode: nemotron-parse) before training. Numeric tokens up-weighted (1+0.5x); inverse-sqrt length weighting; checkpoint selected on held-out structure scores, not "last".
  2. GRPO with verifiable rewards (weights). 303 GRPO groups mined from hard buckets (lambda per blueprint 7.4). Group-normalized advantages (G rollouts/prompt), k3 KL to the SFT reference, prompts mined from the SFT policy's hard failures. Reward = (0.4, 0.25, 0.15, 0.15, 0.05) over (table, chart, structure validity, bbox IoU, EOS).
  3. Verifier-routed refinement (inference). The parser runs once, a gradient-boosted router scores each table/chart region (decoder-token uncertainty + structure sanity + render-vs-crop similarity), and only regions predicted to gain are re-decoded at native resolution. Router labels were rebuilt on the FINAL policy (v1 fitted them on the base model - distribution shift fixed). Code ships with the model: thrisul_pipeline.py (parse_page(..., mode="routed")), thrisul_router.joblib.

Training data (exact-GT or permissively licensed; NO benchmark pages were used for training)

  • Procedurally rendered tables (merged cells, multi-level headers, numeric/currency/percent columns), charts (bar/hbar/line/pie/stacked), and full synthetic pages (title/sections/paragraphs/table/chart) with pixel-exact block annotations - generated by thrisul_pipeline.py itself.
  • Thrisul-Doc D1-D4 custom corpora (this repo, thrisul_pipeline.py v2.1, exact GT by construction, seeds disclosed in thrisul_config.json): D1 complex-table archetypes - 3-level header trees, colspan+rowspan merged cells, borderless layouts, unit rows, multi-index row labels, decimal look-alikes, zero/dash columns; D2 hard charts - dual-axis bar+line, log-scale, 4-series crowded legends, negative bars, 100%-stacked (segments sum to exactly 100.0), irregular long categories; D3 degradation curriculum - JPEG/blur/downscale/rotate/shadow/tone/occlusion/noise in tiers 1-3 applied to D1/D2/synthetic parents whose ground truth is inherited unchanged; D4 multi-column + financial + adversarial pages - 2-column reading-order ground truth, financial statements with parenthesized negatives and notes, decimal-shift look-alike traps, footer-lookalike text. Held-out eval slices for every D domain use seed ranges disjoint from training (61M/62M/63M/64M vs 6.1M-6.4M).
  • Real charts from ChartNet (ibm-granite/ChartNet, CSV ground truth, CDLA-Permissive-2.0).
  • Real financial tables from FinTabNet.c (bsmock/FinTabNet.c, HTML ground truth, CDLA-Permissive-2.0).
  • Compute: Kaggle free tier (T4), LoRA, 1583 SFT optimizer steps + 303 GRPO groups. Targets: 6330 (~533 tokens mean).

Official benchmark results (thrisul-parser)

Measured 2026-10-31 on Kaggle 2×T4 (Transformers, fp16, greedy) by this benchmark notebook. Run tag: v1. Coverage: ParseBench 60 pages/dimension (official total 2,078) · olmOCR-Bench 400 pages (official total = all unique bench pages) · OmniDocBench: packaged · Extra suites: IndicVisionBench=53.05217098342906, MOSCAR=pending.

Sampling disclosure: these runs cover a seeded stratified subset unless the config covered every page. Partial coverage is labeled — never quoted as the full-suite official score. Base-model deltas on identical pages are in results.json.

ParseBench (official evaluator, sampled)

group score
chart 0.206
table 0.168
text_formatting 0.037
text_content 0.491
text 0.091
overall_rule_avg 0.145
mean of table/chart/text 0.155

olmOCR-Bench (official scorer, covered subset)

category pass rate
Overall 29.5
CI95_half_width 1.5
absent 61.2
baseline 98.5
math 4.9
order 14.0
present 15.1
table 18.0
arxiv_math 7.6
headers_footers 56.7
long_tiny_text 14.0
multi_column 13.5
old_scans 24.8
old_scans_math 2.8
table_tests 18.3

IndicVisionBench OCR + other suites (our protocol, official data)

suite metric n score
IndicVisionBench ANLS-C 862 53.05

Metrics computed by this notebook on the official data with the IndicVisionBench paper's OCR protocol (arXiv 2511.04727, following OCRBenchv2: ANLS-C/ANLS-W/CER/WER) — not the authors' official harness; per-language breakdown lives in results.json (indicvisionbench.agg).

Context A — olmOCR-Bench (pass rates, higher is better)

System Protocol arxiv_math old_scans_math tables old_scans headers_footers multi_column long_tiny_text baseline Overall
thrisul-parser v2.1 this run, 400-page covered subset 7.6 2.8 18.3 24.8 56.7 13.5 14.0 98.5 29.5 ± 1.5
Nemotron-3.5-Parse (base) this run, identical pages 42.6 10.9 85.8 36.5 18.3 79.6 69.4 99.8 55.4 ± 1.6
Chandra OCR 0.1.0* published, full corpus 82.2 80.3 88.0 50.4 90.8 81.2 92.3 99.9 83.1 ± 0.9
olmOCR v0.4.0 published, full corpus 83.0 82.3 84.9 47.7 96.1 83.7 81.9 99.7 82.4 ± 1.1
Infinity-Parser 7B* published, full corpus 84.4 83.8 85.0 47.9 88.7 84.2 86.4 99.8 82.5
PaddleOCR-VL* published, full corpus 85.7 71.0 84.1 37.8 97.0 79.9 85.7 98.5 80.0 ± 1.0
Marker 1.10.1 published, full corpus 83.8 66.8 72.9 33.5 86.6 80.0 85.7 99.3 76.1 ± 1.1
DeepSeek-OCR published, full corpus 77.2 73.6 80.2 33.3 96.1 66.4 79.4 99.8 75.7 ± 1.0
MinerU 2.5.4* published, full corpus 76.6 54.6 84.9 33.7 96.6 78.2 83.5 93.7 75.2 ± 1.1
Mistral OCR API published, full corpus 77.2 67.5 60.6 29.3 93.6 71.3 77.1 99.4 72.0 ± 1.1
Nanonets-OCR2-3B published, full corpus 75.4 46.1 86.8 40.9 32.1 81.9 93.0 99.6 69.5 ± 1.1

Published rows: allenai/olmocr README leaderboard, table dated 2025-10-21 (olmOCR v0.4.0). * = not fully open source per that README. The two "this run" rows share one protocol and are comparable only to each other; the gap to the published rows mostly reflects the covered subset, T4-class hardware, and our candidate generation — it is not a claim that those systems are 45+ points ahead of the base model.

Context B — ParseBench (published 0–100 aggregate, self-reported)

System ParseBench (published)
LlamaParse Agentic Plus (API) 88.8
LlamaParse Agentic (API) 83.07
Anthropic Opus 5.5 (Effort High) 78.9
oi-parser 78.3
LightOnOCR-3-4B 75.1
LightOnOCR-3-0.8B 74.6
Infinity-Parser2-Pro (35B) 74.3
Infinity-Parser2-Flash 73.25
Nemotron-3.5-Parse (base of this model; single/double/triple pass) 72.74 / 74.92 / 75.20
MinerU2.5-Pro-2605-1.2B 72.78
KDL-Frontier-Parser-nano 72.38
Chandra-ocr-2 70.1
PaddleOCR-VL-1.6 67.43
Nemotron-Parse-2.0 63.91
Google Gemini 3 Flash (Thinking Minimal) 71.04
OpenAI GPT-5 Mini (Reasoning Medium) 51.52
thrisul-parser v2.1 sampled official-protocol run: 0.145 overall_rule_avg / 0.155 mean-of-3 (different aggregation — not the 0–100 leaderboard scale); official full-corpus run pending

Sources: parsebench.ai leaderboard (self-reported), as captured in this repo's results.json (published_parsebench). The base model ranks #1 among sub-1B parsers on that leaderboard.

Context C — the wider parser landscape (qualitative)

Parser Access Scale License Notes
thrisul-parser v2.1 open weights 631M (+2.8M LoRA) OpenMDW 1.1 (via base) GT-SFT + GRPO + verifier-routed refinement; this repo
Nemotron-3.5-Parse (base) open weights 631M OpenMDW 1.1 bbox + classes + markdown output; #1 sub-1B on ParseBench
Nougat open weights ~250M–350M MIT academic PDFs; LaTeX output; no layout coordinates
GOT-OCR 2.0 open weights ~580M MIT "general OCR theory"; unified formulas/tables/text
olmOCR open weights ~7B Apache-2.0 PDF → plain markdown; strong on olmOCR-Bench by construction
Marker open weights pipeline+DL GPL-3.0 fast PDF→markdown; commercial use restricted
MinerU 2.5 open weights 1.2B (Pro) AGPL-3.0 layout+recognition pipeline; strong OmniDocBench scores
Docling open toolkit — MIT IBM; layout models + table structure; heavy CPU-friendly pipeline
Surya open weights — GPL-3.0 OCR/layout/reading-order toolkit (Datalab)
DeepSeek-OCR open weights ~3B MIT "contexts optical compression"; long-doc focus
PaddleOCR-VL open weights 0.9B Apache-2.0 strong olmOCR-Bench full-corpus score
Chandra open weights — see repo¹ Datalab; top published olmOCR-Bench row above
Mistral OCR API only — commercial strong tables; per-page pricing
LlamaParse API only — commercial multiple modes; ParseBench head rows above
  • Google Gemini 3 Flash (Thinking Minimal): 71.04

Results (read the caveats)

All numbers below were measured by us on held-out synthetic data and a seeded ParseBench-proxy sample on a T4 with Transformers (no vLLM). They are not the official ParseBench score and are not comparable to H100/vLLM numbers anywhere below. Headline deltas carry bootstrap 95% CIs.

Held-out synthetic regions (page-scale input; routed re-decodes only router-selected regions):

subset system score refine_frac n
all thr_routed 0.952 1.000 226
all base_page 0.766 0.000 226
all base_native 0.939 1.000 226
all thr_page 0.722 0.000 226
all thr_native 0.952 1.000 226
table thr_routed 0.964 1.000 128
table base_page 0.813 0.000 128
table base_native 0.975 1.000 128
table thr_page 0.722 0.000 128
table thr_native 0.964 1.000 128
chart thr_routed 0.936 1.000 98
chart base_page 0.704 0.000 98
chart base_native 0.892 1.000 98
chart thr_page 0.721 0.000 98
chart thr_native 0.936 1.000 98
d1_complex_tables thr_routed 0.912 1.000 30
d1_complex_tables base_page 0.743 0.000 30
d1_complex_tables base_native 0.937 1.000 30
d1_complex_tables thr_page 0.678 0.000 30
d1_complex_tables thr_native 0.912 1.000 30
d2_hard_charts thr_routed 0.951 1.000 30
d2_hard_charts base_page 0.594 0.000 30
d2_hard_charts base_native 0.909 1.000 30
d2_hard_charts thr_page 0.600 0.000 30
d2_hard_charts thr_native 0.951 1.000 30
d3_degraded thr_routed 0.863 1.000 16
d3_degraded base_page 0.720 0.000 16
d3_degraded base_native 0.911 1.000 16
d3_degraded thr_page 0.653 0.000 16
d3_degraded thr_native 0.863 1.000 16

Full synthetic pages through the real pipeline (table score, mean-over-tables blended with best-table):

system table_score sec_per_page pages_per_s n
base_single 0.995 5.878 0.170 32
base_double 0.991 8.488 0.118 32
thr_single 0.914 8.985 0.111 32
thr_routed 0.740 13.248 0.075 32

ParseBench-proxy on real enterprise PDFs (seeded stratified sample; crude proxy, small n):

not run (dataset load failed in this session)

Verifier ablation (out-of-fold, final policy):

features spearman auc refine_frac_for_90pct_gain gain_captured_at_30pct
entropy 0.591 0.813 0.876 0.656
all (entropy+structure+render) 0.595 0.814 0.823 0.634
entropy+structure 0.588 0.821 0.783 0.629
render only 0.390 0.678 0.916 0.531
structure 0.108 0.569 0.960 0.355

Pre-registered verdicts measured this run:

FAIL E1' quality: Thrisul single-pass > base single-pass (held-out regions) PASS E2' efficiency: routed ≈ base double-pass quality at partial refine cost FAIL E1'' quality: Thrisul single-pass > base single-pass (full pages)

Text regression check (Thrisul-single vs base-single page markdown similarity): 77.2 %.

Published reference numbers (self-reported by their authors; different protocol/hardware - context only)

  • LlamaParse Agentic Plus (API): 88.8
  • Anthropic Opus 5.5 (Effort High): 78.9
  • oi-parser: 78.3
  • Nemotron-3.5-Parse (single / double / triple pass) = Thrisul base: 72.74 / 74.92 / 75.20
  • LightOnOCR-3-4B: 75.1
  • LightOnOCR-3-0.8B: 74.6
  • Infinity-Parser2-Pro (35B): 74.3
  • Infinity-Parser2-Flash: 73.25
  • MinerU2.5-Pro-2605-1.2B: 72.78
  • KDL-Frontier-Parser-nano: 72.38
  • Chandra-ocr-2: 70.1
  • PaddleOCR-VL-1.6: 67.43
  • PaddleOCR-VL-1.5: 65.95
  • Nemotron-Parse-2.0: 63.91

Leaderboard template row (fill Official score after the locked external ParseBench run)

Field Value
Model version sandeeprdy1729/thrisul-parser (merged sha in thrisul_config.json)
Base checkpoint nvidia/Nemotron-3.5-Parse @ 4035ffb1d8
Params / trainable 631M / 2.8M LoRA
Inference settings single vs routed vs double; max_side=1664; greedy; rep-pen 1.1; max_new_tokens 3072
Official score to fill after external locked run
Measured here region/page/proxy tables above (T4, Transformers)
Date 2026-10-30

Usage 

import torch, sys
from huggingface_hub import snapshot_download
from transformers import AutoModel, AutoProcessor, GenerationConfig
path = snapshot_download("sandeeprdy1729/thrisul-parser")
sys.path.insert(0, path)
from postprocessing import postprocess_document          # shipped from the base model
import thrisul_pipeline as tp
model = AutoModel.from_pretrained(path, trust_remote_code=True, dtype=torch.bfloat16).cuda().eval()   # use float16 on T4
processor = AutoProcessor.from_pretrained(path, trust_remote_code=True)
gen_cfg = GenerationConfig.from_pretrained(path)
router = tp.Router.load(path + "/thrisul_router.joblib")
post = lambda text, image_size: postprocess_document(text, image_size=image_size)
out = tp.parse_page(model, processor, gen_cfg, image, post, mode="routed", router=router)   # or mode="single" / "all"
print(out["md"])

Limitations (honest)

  • Real-document gains beyond the proxy above are unproven; the official ParseBench/OmniDocBench/olmOCR-Bench runs are external and pending.
  • The router is fitted on synthetic crops including the D1-D3 domains plus ChartNet/FinTabNet; distribution shift on unseen real layouts remains possible (pages where region mapping fails silently fall back to single pass - counted in parse_page info).
  • GRPO was time-boxed on free T4s; longer reward optimization is the most promising next lever (see the notebook's backlog).
  • Coordinates in the base model are content-normalized; the GT serializer's round-trip is verified in-repo (thrisul_config.json records the mode).
  • Evaluated on a T4 with Transformers; throughput numbers do not transfer to H100/vLLM.

License and attribution

Derivative work of NVIDIA Nemotron 3.5 Parse; use is governed by the base model's license (OpenMDW 1.1) and the third-party notices in this repository. Training data: own synthetic renderers and the Thrisul-Doc D1-D4 corpora (this repo), ChartNet and FinTabNet.c under CDLA-Permissive-2.0. See LICENSE and the notice files copied from the base repo.