- shrew-ocr-preview
shrew-ocr-preview
One document page image in, one JSON object out: metadata, a summary, self-contained semantic chunks sized for RAG, and figures and tables with boxes and HTML. A text mode takes HTML, markdown or plain text and returns the same schema. Fine-tuned from Granite Vision 4.1 4B; merged bf16 weights, v0.5.
New in v0.5: two further supervised fine-tuning stages on a revised training cut, then a small weight edit against repetition loops. On OmniDocBench v1.6 the overall score (official notebook formula, page-averaged table TEDS) goes from 77.12 (v0.4.1) to 83.45, mostly from table structure (page-averaged TEDS 60.79 โ 75.25). Table localization recall is up; figure recall is down, far false positives are up, and the model splits large tables (and occasionally figures) into stacked strips more often (see Limitations). v0.4.x was scored on different GPUs and a different server build than v0.5 (see Results). The loop edit's effect on held-out pages is not statistically significant.
Preview: weights update in place under this name. Pin the tag
v0.5or a commit (revision=) to reproduce.
Quick start
Needs shrew-server โฅ 0.5.0, the reference server, which implements the whole input contract (v0.5 weights use the new output budgets and box format). Compose quickstart is in its README. shrew-server v0.5.0 defaults to the v0.5 model contract; set SHREW_CONTRACT=v0.4 to serve v0.4 or v0.4.1 weights. It is installed from a clone of its repository (pip install .[server,figures]), not from PyPI. It plans every request inside the model's 30,000-token trained budget (prompt, image and output together), so serve with --max-model-len 32768; its /health reports degraded if the served context is below 30,000. VLM_MODEL must equal vLLM's --served-model-name.
vllm serve btbtyler09/shrew-ocr-preview --revision v0.5 --trust-remote-code \
--served-model-name shrew-ocr-preview \
--max-model-len 32768 --limit-mm-per-prompt '{"image":1}' --no-enable-prefix-caching
VLM_URL=http://localhost:8000 VLM_MODEL=shrew-ocr-preview shrew serve
curl -X POST localhost:8080/v1/convert -F file=@doc.pdf -F pipeline_mode=structured
Output schema
One request = one page. Page-level keys: metadata, summary, semantic_chunks, tables, plus figures. Direct generations from the model name the figures key figures; the /v1/convert response of shrew-server v0.5.0 returns them as images. Boxes in the server response are quadrilaterals: four [x, y] corner points, clockwise from the top left (top left, top right, bottom right, bottom left), on a 0โ1000 grid over the page image; v0.4.x returned [x0, y0, x1, y1].
{
"metadata": {"title", "authors" (list), "organization", "year", "doc_type"} โ null where unknown,
"summary": str | null,
"semantic_chunks": [{"chunk_id", "title", "content", "keywords", "section_type"}],
"images": [{"bbox": [[x,y],[x,y],[x,y],[x,y]] | null, "caption", "description", ...}],
"tables": [{"html": "<table>โฆ", "bbox": [[x,y],[x,y],[x,y],[x,y]] | null, "caption", "description", ...}]
}
section_type takes values from a fixed trained taxonomy that the server documents. Consumers that validate section_type should accept the full set the server documents.
Training
Two LoRA supervised stages (language model r=512, vision r=64), merged to full weights, starting from v0.4.1. Stage 1: 3,000 steps at 64 rows per step. Stage 2: continued from the stage-1 adapter on a revised training cut of 480,513 rows (one page per row, maximum sequence 30,000 tokens, never truncated), stopped at step 1,500 (about 0.2 epoch of the cut). The training mix includes synthetic document renders (see Attribution), real pages, handwriting, formulas, tables and several languages. Loss is on the assistant tokens only; the boxes of model-labelled tables and figures were masked from the loss. The vision tower and projectors changed in this fine-tune, so they are not byte-identical to v0.4.x. The loop edit below is applied on top.
Results: OmniDocBench v1.6
Official evaluator on all 1,651 pages, CDM on, bf16 weights; overall is the official notebook formula. The right-hand column is v0.5 without the loop edit (the same weights with the edit not applied), the control for the edit.
| metric | v0.4 | v0.4.1 | v0.5 (this repo) | v0.5 without loop edit |
|---|---|---|---|---|
| text edit distance (lower is better) | .1457 | .1367 | .1084 | .1070 |
| formula CDM | 84.18 | 84.24 | 85.94 | 85.01 |
| table TEDS, page-averaged (official notebook formula) | 61.41 | 60.79 | 75.25 | 75.52 |
| table TEDS structure only, page-averaged | 65.32 | 64.61 | 80.24 | 80.50 |
| table TEDS, per-table mean (as printed on the v0.4 and v0.4.1 cards) | 56.72 | 57.04 | 70.10 | 69.66 |
| table TEDS structure only, per-table mean | 60.40 | 60.64 | 75.36 | 74.29 |
| reading-order edit distance (lower is better) | .2275 | .2185 | .2058 | .2089 |
| Overall, page-averaged TEDS (official notebook formula) | 77.01 | 77.12 | 83.45 | 83.28 |
| Overall, per-table TEDS | 75.44 | 75.87 | 81.74 | 81.33 |
Two TEDS aggregations are shown because the earlier cards used a different one. The v0.4 and v0.4.1 cards printed the per-table mean (56.72 and 57.04); the official notebook aggregates TEDS per page (61.41 and 60.79 for the same outputs and runs), and that is what the other metrics and the first Overall line use. The second Overall line is ((1 โ text edit distance) ร 100 + TEDS + CDM) / 3 with the per-table TEDS, recomputed here from the same formula for all columns; the v0.4.1 card printed CDM 84.21 โ 85.19 and overall 75.45 โ 76.19, and this re-score of the same runs gives CDM 84.18 โ 84.24, so the CDM gain claimed there does not reproduce and is corrected here. The two aggregations rank v0.5 against v0.5 without the loop edit in opposite directions on TEDS (75.25 vs 75.52 page-averaged; 70.10 vs 69.66 per-table), which is no detectable difference.
Precision and serving differ by column. v0.4 and v0.4.1: bf16 through shrew-server v0.4.0 on one RX 9070, the shipped recipe. v0.5 and the without-loop-edit column: bf16 through shrew-server v0.5.0 (a development build) on 8 MI100 (vLLM v0.28 gfx908, TRITON_ATTN). One pass each. With the final v0.5.0 guard build the same weights score text .1093, CDM 86.22, page-averaged TEDS 75.09, structure-only 80.40, reading order .2059, overall 83.46. v0.4.x was not re-run on the v0.5 setup; the same v0.5 weights moved by 0.01 overall between two server builds, far less than the 6.3-point gap to v0.4.1, but do not read the third digit. Overall 83.45 vs 83.28 and CDM 85.94 vs 85.01 (v0.5 vs without the loop edit) are within noise; the paired CIs below cover text, reading order, table and formula edit distance, and CDM was not paired.
Paired, v0.5 minus v0.5 without the loop edit, per page, 95 % CI: text edit distance +.0014 [โ.005, +.008]; reading order โ.003 [โ.010, +.004]; table edit distance +.006 [โ.009, +.022]; formula edit distance โ.005 [โ.024, +.014]. No detectable difference on any of them.
Gates over the 1,651 pages, v0.5 without the loop edit vs v0.5: first pass OK 1,590 vs 1,597; degenerate 52 vs 41; failed after the retry 24 vs 23 (13 of them the same pages). Gate counts for v0.4 and v0.4.1 were taken on the earlier server and are not in this table.
Results: localization on a private 541-page held-out set
A private 541-page held-out localization set (353 tables, 699 figures; pages overlapping the v0.4.1 training lineage removed; not re-checked against the v0.5 cut). v0.4 and v0.4.1 were run on a different GPU and server build than v0.5; a v0.4 run unparsed on 33 pages counts those as misses, which is why the v0.4 row differs from the v0.4.1 card. Figure and table recall at IoU 0.5; far false positives are boxes far from any reference box.
| arm | figure recall@0.5 | table recall@0.5 | far false positives (fig / table) |
|---|---|---|---|
| v0.4 (508 of 541 pages parsed) | .655 (458/699) | .595 (210/353) | 36 / 13 |
| v0.4.1 | .708 (495/699) | .575 (203/353) | 36 / 16 |
| v0.5 without loop edit | .627 (438/699) | .734 (259/353) | 60 / 29 |
| v0.5 (this repo) | .631 (441/699) | .703 (248/353) | 61 / 31 |
Honest read: table recall is up (+12.8 points over v0.4.1), figure recall is down (โ7.7 points) and far false positives are up (figure 36 โ 61, table 16 โ 31 against v0.4.1). The model more often splits large tables into stacked strips (table strips 32 โ 45 against v0.4.1; figure strips 12 โ 13, flat): pages with three or more strips of either kind were 2 for v0.4, 2 for v0.4.1 and 4 for v0.5. The loop edit versus v0.5 without it: table hits 259 โ 248 (page-level sign test p โ .23, not significant).
OHR-Bench retrieval and reliability
v0.5: bf16 through shrew-server v0.5.0 (final guard build) on 8 MI100. v0.4: bf16 through shrew-server v0.4.0, from the v0.4 release run; re-scoring its outputs with today's harness reproduces the column. MinerU, PaddleOCR and v0.3 come from that same earlier run. Same queries, chunking and embedder for every column. INT8 parity numbers are on the GPTQ card, from earlier releases.
Retrieval, OHR-Bench
Hit@5 / MRR@10 on OHR-Bench (ICCV 2025): 1,261 PDFs, 8,561 pages, 7 domains. Bold marks the best model or tool per metric; ground truth is a reference, not ranked.
| evidence type | MinerU | PaddleOCR | shrew v0.5 (bf16) | shrew v0.4 (bf16) | shrew v0.3 weights, same server | Ground truth (reference, upper bound) |
|---|---|---|---|---|---|---|
| plain text | .890 / .804 | .956 / .875 | .958 / .871 | .957 / .871 | .960 / .873 | .975 / .892 |
| multi-evidence | .956 / .873 | .963 / .854 | .985 / .848 | .970 / .864 | .970 / .855 | .978 / .883 |
| table | .927 / .795 | .921 / .795 | .931 / .811 | .918 / .799 | .913 / .787 | .958 / .842 |
| formula | .925 / .835 | .933 / .858 | .955 / .870 | .949 / .859 | .944 / .848 | .969 / .886 |
| chart | .659 / .540 | .668 / .520 | .787 / .660 | .765 / .630 | .757 / .636 | .878 / .751 |
| vision | .634 / .488 | .737 / .579 | .784 / .648 | .767 / .613 | .754 / .599 | .797 / .588 |
| reading orderโ | .942 / .866 | .941 / .848 | .796 / .744 | .754 / .692 | .476 / .450 | .930 / .839 |
| all 8,414 queries | .882 / .779 | .910 / .808 | .917 / .819 | .906 / .806 | .881 / .782 | .952 / .852 |
Our own harness over the ~8.5k human-verified Q&A pairs (all systems share chunking and nvidia/llama-nemotron-embed-vl-1b-v2), not official OHR-Bench LCS/F1. Exact kNN, so not comparable to the v0.2 card. The reference is not a strict ceiling: MinerU and PaddleOCR exceed it on reading order, shrew on vision MRR. โ Reading-order queries are almost all dense Chinese broadsheets: on v0.4, pages failed 3.8 % vs 11.8 % for v0.3.
| paired read, top-5 hits only in one arm | A | B | p |
|---|---|---|---|
| v0.5 (A) vs v0.4 (B), all queries | 298 | 207 | 6ร10โปโต |
| same, reading order | 64 | 35 | 0.005 |
| same, the other 7,723 queries | 234 | 172 | 0.002 |
| v0.4 (A) vs v0.3 weights (B), all queries | 386 | 175 | 3ร10โปยนโน |
| same, reading order on dense Chinese | 225 | 25 | |
| same, the other 7,723 queries | 189 | 170 | 0.34 |
| recipe alone on v0.3 weights, .881 โ .893 | 265 | 165 | 2ร10โปโถ |
| v0.4 weights at the same recipe, .893 โ .906 | 325 | 214 | 2ร10โปโถ |
- The v0.3 card's earlier server scored the same weights .844 pooled and .098 on reading order.
- Overlap found 2026-09-28: 122 OHR-Bench 2024 NYT pages were in training from v0.2 through v0.4.1. Excluded here, for v0.5 as well: 280 queries (274 plain-text, 6 vision; 8,414 of 8,694 remain) and 10 localization pages (541 of 551 remain). Not re-checked for v0.5.
- That exclusion moves pooled hit@5 by 0.002โ0.003, and every paired read holds. On those 280 queries shrew scores no better than systems that never saw them (hit@5 .971 vs MinerU .975, PaddleOCR .961; ground-truth reference .982).
- Overlap found 2026-11-05: training contains 1,040 evaluation pages of public handwriting benchmarks (Peter 640/640, Rukopys 369/370, Bentham 31/31); the RL stage may have sampled 112. No score on them is reported or valid as held-out. Our handwriting checks use a separate 300-page set. Not re-checked for v0.5.
Reliability, OHR-Bench 8,561 pages
| arm | failed pages | rescued by retry |
|---|---|---|
| v0.5, current recipe (98.8 % valid on first pass) | 42 (0.49 %) | 64 |
| v0.4, current recipe (99.3 % valid on first pass) | 27 (0.32 %) | 32 |
| v0.3 weights, greedy, same server | 86 (1.00 %) | 28 by re-roll, 33 by enforced retry |
| v0.3 card, earlier server | 2.53 % |
Reliability is worse than v0.4 on this benchmark: guard aborts for repetition 99 vs 51 and failed pages 42 vs 27. Most of the increase is on manuals and textbooks (repetition aborts 31 vs 5 and 24 vs 11). Retrieval is still higher overall.
Decoding
The sampling settings and retry policy are unchanged from v0.4.1 (the output budgets changed) and the reference server ships them. Never greedy.
- First pass:
temperature0.4,presence_penalty0.3, a fixed per-page seed, no top_p / top_k.max_tokensis min(bucket budget, 30,000 โ prompt tokens) (below). - On failure (guard abort, truncation, invalid or off-schema JSON): one retry, same settings, fresh seed, no grammar enforcement.
- Then a fallback extractor if configured, otherwise a failed page. Do not loop further.
- Output budgets in shrew-server v0.5.0, in output tokens: B0 6,500, B1 25,500, B2 29,500, B3 26,500, text mode 7,000. These are caps: the server sends
max_tokens= min(budget, 30,000 โ prompt tokens). Prompt tokens measured on a full page image are about 1,500 (B0), 1,950 (B1), 3,700 (B2) and 7,200 (B3), so the effective value at a full B2 image is about 26,300 and at B3 about 22,800. With--max-model-len 32768a request that adds prompt and budget past 32,768 is rejected, so direct callers should apply the same minimum. generation_config.jsoncarriesdo_sampletrue,temperature0.4,eos_token_id100257 andmax_new_tokens29500 (the largest bucket budget); setpresence_penaltyper request. The top-levelconfig.jsonhas noeos_token_id(it lives intext_config), so Hugging Facegeneraterelies on the one ingeneration_config.json. Context length โฅ 32768.- Do not use frequency or repetition penalties,
no_repeat_ngram_sizeor grammar enforcement: they degrade required schema tokens or tables.
The measurements below are from earlier releases and were not re-run for v0.5.
| measurement | release | result |
|---|---|---|
| sampled 0.4 vs greedy, first-pass clean, 228 dense Chinese pages | v0.4 | 91.7 % (majority of three seeds) vs 75.0 %; recall inside a 0.05 bound |
| un-enforced vs enforced retry, 65 first-pass failures | v0.4-era RL checkpoints | recovered 69 % vs 15 % |
presence_penalty 0.3 at first pass, 900 runs (150 pages ร 6 arms) |
v0.3 or earlier | success 0.880 โ 0.887; precision 0.949 โ 0.949 |
| penalized retry at 0.6 on loop-failed Latin broadsheets | v0.3 or earlier | 12/12 recovered; dense non-Latin did not, at any penalty |
| grammar at first pass, table one-shot / fidelity | v0.3 or earlier | 0.975 โ 0.225 / 0.483 โ 0.280 |
- About a quarter of hard pages flip across seeds; the server derives a fixed per-page seed so reruns reproduce.
- 0.3โ0.6 both measured fidelity-neutral on healthy pages.
- A penalized retry on an unsupported script can turn a loop into fluent hallucination: gate retries the same way and mark them lower confidence.
- Which pages loop varies with the serving configuration; the loop rate does not. Compare deployments by rate over a fixed page set.
Direct integration without shrew-server
- System prompt: the literal string
structured_extraction. The model was trained on this prompt only, with the page image as the whole user turn (no user text). - Page preprocessing (glyph-height bucket routing and image enhancement) is done by shrew-server v0.5.0, which is the reference implementation; direct callers should reproduce its routing, which its documentation describes.
- Serve with
--dtype bfloat16. On small GPUs keep--max-num-batched-tokensโค 2048: large budgets can OOM the tower. - Streaming repetition guard: see the shrew-server v0.5.0 documentation.
{
"model": "shrew-ocr-preview",
"temperature": 0.4,
"presence_penalty": 0.3,
"max_tokens": 25500,
"messages": [
{"role": "system", "content": "structured_extraction"},
{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": "data:image/png;base64,<base64 of the page image>"}}]}
]
}
max_tokens above is the B1 budget; use the budget of the bucket you chose. Text mode. Same envelope, max_tokens 7000, with the raw content as a single text part and no wrapper. Born-digital input only; scans go through the image path.
Loop edit (v0.5)
- One direction, steady repetition, estimated from this checkpoint's own activations (teacher-forced on 61 looping generations from 32 distinct pages; best cross-validated separation at layer 25, AUROC .93), removed from
down_projcolumns: W[:,j] โ W[:,j] โ 0.5ยทd(dยทW[:,j]), computed in fp32, stored as bf16. - 256 columns in each of layers 24, 25, 34 and 37: 1,024 columns, 2,621,440 weights, 0.066 % of 3,997,206,464 parameters. Everything else is byte-identical to v0.5 without the loop edit.
- Fit only on a 115-page fit subset of a held-out set of looping CJK pages the training lineage had not seen. The other 113 pages (the evaluation subset) were scored once, after the edit was frozen. Four edits were compared (loop-only at strength 0.5, which is the shipped edit, and 1.0; loop-plus-entry at 0.5 and 1.0), plus earlier candidates. The shipped edit was chosen by a pre-registered rule, the smallest edit among those not worse than v0.5 without the loop edit, not by best number.
- Not guaranteed to transfer to a later checkpoint; the edit is rebuilt per checkpoint.
The effect is not statistically significant on held-out pages. Looping page-seeds at the production decode (temperature 0.4, presence_penalty 0.3), 3 seeds ร 113 pages:
| set | v0.5 without loop edit | v0.5 | paired difference, 95 % CI |
|---|---|---|---|
| held-out evaluation subset, 113 pages | 32/339 (9.4 %) | 27/339 (8.0 %) | โ1.5 points [โ5.0, +1.8]; 22 fixed, 17 broken |
| same, excluding 50 pages whose images render broken (vertical-writing font maps) | 29/261 | 21/261 | โ3.1 points [โ7.3, +1.2] |
| fit subset, 115 pages (used to choose the edit) | 26/345 | 19/345 | โ2.0 points [โ5.2, +1.2] |
Every interval includes zero. Normal pages: on 102 held-out normal pages at greedy decoding, one page that was clean without the edit loops with it; at the production recipe that page is clean in 3 of 3 seeds for both. Mean text similarity to the output without the edit is .985.
For comparison, the v0.4.1 edit (two directions, 1,925 columns) measured on a different held-out set of 40 looping pages: 27/40 โ 4/40 at greedy and 15/40 โ 3/40 at the shipped recipe of that release.
Limitations
- Dense broadsheets, low-resolution dense scans and pages the vision tower cannot resolve can still loop or stop early; the guard makes these detectable failures.
- Large tables (and occasionally figures) are split into stacked strips more often than in v0.4.x (4 pages with three or more strips, vs 2; table strips 32 โ 45), and far false positives are up against v0.4.1 on the held-out set above (figure 36 โ 61, table 16 โ 31). Figure recall fell by 7.7 points versus v0.4.1.
- The loop edit is not statistically significant on held-out pages (CIs above); do not count on it to remove loops.
- On OHR-Bench, v0.5 fails more pages than v0.4 (42 vs 27 of 8,561), mostly from repetition on manuals and textbooks. INT8 parity was not re-run for v0.5.
- Character accuracy on dense Chinese is below Latin print. Cyrillic and Arabic were not evaluated: treat as unsupported. Handwriting has limited evaluation (a separate 300-page check) and no published score.
- Boxes come from model-generated labels: pad them outward when cropping; edges are not pixel-exact.
- One page per request: no cross-page state, so assemble multi-page documents downstream.
- Reading order on dense broadsheets is .796 hit@5 on v0.5 (.754 on v0.4, .098 in v0.3), still trailing MinerU (.942).
Lineage
- LoRA on Granite โ bucket base (v0.2) โ v0.3 merge โ unpublished epoch merge โ RL merge (v0.4) โ weight edit (v0.4.1) โ two LoRA SFT stages (language model r=512, vision r=64), merged โ weight edit (v0.5). The weights are a full-weight fork; merged-only since v0.4.
- The v0.4 RL stage trained the language model only, on a repetition penalty, content structure (TEDS, cell F1, metadata, code/formula), a retrieval anchor and a summary floor. Box geometry was not rewarded.
- The v0.5 SFT stages changed the vision tower and projectors as well as the language model; see Training.
Versions
| variant | precision | size | notes |
|---|---|---|---|
| shrew-ocr-preview | bf16 (this repo) | 8.0 GB | reference quality (v0.5) |
| shrew-ocr-preview-GPTQ-8bit | INT8 LM / bf16 vision | 5.1 GB | ~1.8ร serving throughput (measured on an earlier release); serve with --dtype half; v0.5 boot check on the card |
| shrew-ocr-preview-FP8 | W8A8 FP8 LM / bf16 vision | 4.9 GB | vLLM; validated only on one NVIDIA RTX PRO 6000; calibrated on 480 real pages; measured only for boot, throughput and similarity to bf16 (see its card) |
| shrew-ocr-preview-GGUF | Q8_0 or f16 LM / f16 vision | 3.6 GB (Q8_0), 6.8 GB (f16) | llama.cpp; full context per slot required (-c = N ร 32768) |
| shrew-ocr-preview-lora | LoRA adapter (r=256, bf16) | 2.0 GB | v0.3 adapter, final โ applies only to the v0.2 bucket base; no adapter is released after it (see Lineage) |
Checksums: SHA256SUMS.v0.5.txt in this repo.
Changelog
- Correction: the CDM gain printed on the v0.4.1 card (84.21 โ 85.19) does not reproduce on re-score (84.18 โ 84.24); v0.4 figure recall in the v0.5 localization table is .655, from a different run than the .678/.688 quoted for v0.3/v0.4 below.
- v0.5 (2026-11-14): two LoRA SFT stages from v0.4.1 (the second on a revised cut of 480,513 rows), then the loop edit (1,024 MLP columns, 0.066 % of weights); OmniDocBench overall 77.12 โ 83.45, table TEDS 60.79 โ 75.25, text edit distance .1367 โ .1084; table localization recall up, figure recall down, far false positives up (v0.4.x scored on different GPUs and a different server build); loop edit not statistically significant. New contract: larger per-bucket output budgets, quadrilateral boxes, a long-window stream guard. Requires shrew-server โฅ 0.5.0; model and server versions are numbered independently.
- v0.4.1 (2026-11-06): loop direction edited out of 1,925 MLP columns (4.93M weights, 0.123 %), no retraining. Requires shrew-server โฅ 0.4.0.
- v0.4 (2026-09-28): RL stage merged onto an unpublished full-epoch merge of v0.3, plus the sampled recipe; retrieval p = 2ร10โปยนโน, reading order .476 โ .754, failed pages 1.00 % โ 0.32 %, figure recall .678 โ .688. Requires shrew-server โฅ 0.4.0.
- v0.3 (2026-09-14): r=256 LoRA, 1 epoch / 1,343 steps, eval_loss 0.1408 vs 0.1421; OmniDocBench newspaper loops 66 % โ 46 %, figure far false positives 156 โ 51; six pre-registered criteria passed; INT8 gated on a 215-page parity card (text similarity median 1.000). Requires shrew-server โฅ 0.3.14.
- v0.2: glyph-routed tile buckets; first GPTQ-8bit and GGUF releases.
- Correction 2026-09-28: 122 OHR-Bench news documents (2024 NYT front pages) were in training from v0.2 on; retrieval and localization tables exclude them (hit@5 .908 โ .906).
Attribution
Text from the following OpenStax books (Rice University and named co-authors), each retrieved 2026-10-11 under the Creative Commons Attribution 4.0 International licence (https://creativecommons.org/licenses/by/4.0/), is used as prose in some synthetic document renders, and as original page images in a tables-and-formulas training set. Only the books whose licence was CC BY 4.0 at retrieval were used; the CC BY-NC-SA books were never fetched. No OpenStax photographs or third-party credited figures are used; OpenStax's own graphs, diagrams and tables appear only as printed on the original pages described in the Changes note.
Changes: some rows use passages excerpted and re-typeset into synthetic document renders (tables, layout and surrounding content are our own); other rows are original OpenStax book pages (CC BY 4.0), full pages rasterised at 200 dpi and otherwise unmodified, with the page text, tables, graphs, diagrams and illustrations as printed, each such row carrying its own attribution (book title, licence, licence URL, page); for every row the chunk boundaries and titles, summaries, keywords and descriptions are ours; photographs and figures credited to a third party (courtesy, flickr/wikimedia/getty-style credits) are refused and not used, while OpenStax's own tables and graphs that cite a data source are kept as printed with that citation.
- Algebra and Trigonometry, OpenStax, Rice University (openstax.org/details/books/algebra-and-trigonometry) - CC BY 4.0
- American Government, OpenStax, Rice University (openstax.org/details/books/american-government) - CC BY 4.0
- American Government 2e, OpenStax, Rice University (openstax.org/details/books/american-government-2e) - CC BY 4.0
- American Government 3e, OpenStax, Rice University (openstax.org/details/books/american-government-3e) - CC BY 4.0
- Anatomy and Physiology, OpenStax, Rice University (openstax.org/details/books/anatomy-and-physiology) - CC BY 4.0
- The AP Physics Collection, OpenStax, Rice University (openstax.org/details/books/college-physics-ap-courses) - CC BY 4.0
- Astronomy, OpenStax, Rice University (openstax.org/details/books/astronomy) - CC BY 4.0
- Biology, OpenStax, Rice University (openstax.org/details/books/biology) - CC BY 4.0
- Chemistry, OpenStax, Rice University (openstax.org/details/books/chemistry) - CC BY 4.0
- Chemistry: Atoms First, OpenStax, Rice University (openstax.org/details/books/chemistry-atoms-first) - CC BY 4.0
- College Algebra, OpenStax, Rice University (openstax.org/details/books/college-algebra) - CC BY 4.0
- College Algebra with Corequisite Support, OpenStax, Rice University (openstax.org/details/books/college-algebra-corequisite-support) - CC BY 4.0
- College Physics, OpenStax, Rice University (openstax.org/details/books/college-physics) - CC BY 4.0
- Elementary Algebra, OpenStax, Rice University (openstax.org/details/books/elementary-algebra) - CC BY 4.0
- Intermediate Algebra, OpenStax, Rice University (openstax.org/details/books/intermediate-algebra) - CC BY 4.0
- Introduction to Intellectual Property, OpenStax, Rice University (openstax.org/details/books/introduction-intellectual-property) - CC BY 4.0
- Introduction to Sociology 2e, OpenStax, Rice University (openstax.org/details/books/introduction-sociology-2e) - CC BY 4.0
- Introductory Business Statistics, OpenStax, Rice University (openstax.org/details/books/introductory-business-statistics) - CC BY 4.0
- Introductory Statistics, OpenStax, Rice University (openstax.org/details/books/introductory-statistics) - CC BY 4.0
- Physics, OpenStax, Rice University (openstax.org/details/books/physics) - CC BY 4.0
- Prealgebra, OpenStax, Rice University (openstax.org/details/books/prealgebra) - CC BY 4.0
- Precalculus, OpenStax, Rice University (openstax.org/details/books/precalculus) - CC BY 4.0
- Principles of Economics, OpenStax, Rice University (openstax.org/details/books/principles-economics) - CC BY 4.0
- Principles of Economics 2e, OpenStax, Rice University (openstax.org/details/books/principles-economics-2e) - CC BY 4.0
- Principles of Macroeconomics, OpenStax, Rice University (openstax.org/details/books/principles-macroeconomics) - CC BY 4.0
- Principles of Macroeconomics 2e, OpenStax, Rice University (openstax.org/details/books/principles-macroeconomics-2e) - CC BY 4.0
- Principles of Macroeconomics for APยฎ Courses, OpenStax, Rice University (openstax.org/details/books/principles-macroeconomics-ap-courses) - CC BY 4.0
- Principles of Macroeconomics for APยฎ Courses 2e, OpenStax, Rice University (openstax.org/details/books/principles-macroeconomics-ap-courses-2e) - CC BY 4.0
- Principles of Microeconomics, OpenStax, Rice University (openstax.org/details/books/principles-microeconomics) - CC BY 4.0
- Principles of Microeconomics 2e, OpenStax, Rice University (openstax.org/details/books/principles-microeconomics-2e) - CC BY 4.0
- Principles of Microeconomics for APยฎ Courses, OpenStax, Rice University (openstax.org/details/books/principles-microeconomics-ap-courses) - CC BY 4.0
- Principles of Microeconomics for APยฎ Courses 2e, OpenStax, Rice University (openstax.org/details/books/principles-microeconomics-ap-courses-2e) - CC BY 4.0
- Psychology, OpenStax, Rice University (openstax.org/details/books/psychology) - CC BY 4.0
- Statistics, OpenStax, Rice University (openstax.org/details/books/statistics) - CC BY 4.0
- Fรญsica universitaria volumen 1, OpenStax, Rice University (openstax.org/details/books/fรญsica-universitaria-volumen-1) - CC BY 4.0
License and citation
Apache-2.0. Base: ibm-granite/granite-vision-4.1-4b (Apache 2.0).
@misc{shrew-ocr-preview,
title = {shrew-ocr-preview: page image to structured JSON},
author = {btbtyler09},
year = {2026},
howpublished = {\url{https://huggingface.co/btbtyler09/shrew-ocr-preview}},
note = {Version v0.5}
}
- Downloads last month
- 106