Spaces:
Running
Running
Download analysis.html from thanujhf/akshara-ocr-studio: direct link, hf CLI and curl.
- Browser
- Download file 11.4 kB
-
https://huggingface.co/spaces/thanujhf/akshara-ocr-studio/resolve/main/analysis.html
- Command line
-
hf download hf://spaces/thanujhf/akshara-ocr-studio/analysis.html
-
curl -L -o analysis.html https://huggingface.co/spaces/thanujhf/akshara-ocr-studio/resolve/main/analysis.html
11.4 kB
| <html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1"><link rel="icon" href="favicon.svg" type="image/svg+xml"><title>Akshara · Research results & comparisons</title><meta name="description" content="Measured Telugu word-crop OCR results, paired uncertainty and benchmark-specific comparisons."><link rel="stylesheet" href="style.css?v=20261015.2"><link rel="stylesheet" href="analysis.css?v=20261015.2"></head> | |
| <body class="analysis-page"><div class="analysis-shell"> | |
| <nav class="analysis-nav" aria-label="Research navigation"><a class="analysis-brand" href="/"><span class="analysis-mark" lang="te">అ</span><strong>akshara</strong><span class="research-label">/ research</span></a><div><a href="#controlled">Our study</a><a href="#external">Published benchmarks</a><a href="progress.html">Study dashboard ↗</a><a class="button" href="/">Open studio ↗</a></div></nav> | |
| <main class="analysis-main"> | |
| <section class="analysis-hero"><div><div class="eyebrow">TELUGU HANDWRITING · COMPLETED LOCAL STUDY</div><h1>Our OCR,<br><em>measured.</em></h1><p>One capped training epoch. An untouched final set until selection. A direct comparison with the same original IndicOCR checkpoint.</p><p id="snapshotDate" class="muted">Loading the verified results snapshot…</p></div><aside class="rank-card"><span class="eyebrow">WITHIN THIS STUDY</span><strong id="localRank">—</strong><p>By final grapheme CER, among three tested variants.</p><div class="rank-rule"></div><span class="eyebrow">WORLDWIDE POSITION</span><b>Not externally ranked</b><p>A common external benchmark submission is required to establish that position.</p></aside></section> | |
| <p id="analysisError" role="alert" hidden></p> | |
| <div class="analysis-kpis" aria-label="Selected-model final results"><article><small>Final exact-word match</small><strong id="finalExact">—</strong><span id="exactCount">Against source-provided labels</span></article><article><small>Final grapheme CER ↓</small><strong id="finalCER">—</strong><span>Raw and NFC reported separately below</span></article><article><small>Exact gain over original</small><strong id="exactGain">—</strong><span id="exactCI">Paired 95% interval</span></article><article><small>Original / selected API parity</small><strong id="parityCount">—</strong><span>32 predeclared validation crops · 85 software tests</span></article></div> | |
| <section class="analysis-section" id="controlled"><div class="section-heading"><div><div class="eyebrow">IDENTICAL INPUTS · DETERMINISTIC DECODING</div><h2>Original weights. Pilot. Selected adapter.</h2></div><label class="metric-picker">Compare<select id="comparisonMetric"><option value="nfc_micro_cer">NFC grapheme CER ↓</option><option value="nfc_micro_wer">NFC WER ↓</option><option value="raw_exact_match">Raw exact word ↑</option></select></label></div><p>The candidate was locked using validation only. Final outcomes did not change that choice. This ranking is descriptive of these three variants on the same 3,000 word crops.</p><div id="modelChart" class="model-chart" aria-live="polite"></div><p id="chartScope" class="chart-scope"></p><div class="table-wrap" tabindex="0" aria-label="Complete raw and NFC final metrics"><table><thead><tr><th>Variant</th><th>Crops</th><th>Raw CER</th><th>NFC CER</th><th>Raw WER</th><th>NFC WER</th><th>Raw exact</th><th>NFC exact</th></tr></thead><tbody id="finalTable"></tbody></table></div><p class="table-help">Swipe tables horizontally on a narrow screen. Lower CER/WER and higher exact match are better.</p><div class="protocol-strip"><div><small>TRAIN</small><strong id="trainN">—</strong><span>One effective epoch · 6,250 updates</span></div><div><small>VALIDATE</small><strong id="valN">—</strong><span>Initialization and candidate selection</span></div><div><small>TEST</small><strong id="testN">—</strong><span>Sealed until selection · same inputs</span></div></div><p id="validationGap" class="study-note"></p></section> | |
| <section class="analysis-section"><div class="eyebrow">PAIRED EVIDENCE</div><h2>Improvement, with uncertainty.</h2><p>2,000 bootstrap repetitions, seed 42, across 128 source-document / duplicate components. These groups are not verified independent writers. Positive error reductions and exact gains favor the already-selected model.</p><div class="table-wrap" tabindex="0" aria-label="Paired uncertainty intervals"><table><thead><tr><th>Original → selected</th><th>Difference (pp)</th><th>Paired 95% interval (pp)</th></tr></thead><tbody id="intervalTable"></tbody></table></div><div class="detail-grid"><details><summary>Input-defined subgroup results</summary><p>Original versus selected. Output-dependent error-flag groups are excluded from this matched subgroup table.</p><div class="table-wrap" tabindex="0" aria-label="Subgroup results"><table><thead><tr><th>Subgroup</th><th>N</th><th>Original CER</th><th>Selected CER</th><th>Original WER</th><th>Selected WER</th><th>Original exact</th><th>Selected exact</th></tr></thead><tbody id="subgroupTable"></tbody></table></div></details><details><summary>Output diagnostics and resource measurements</summary><div class="table-wrap" tabindex="0" aria-label="Diagnostics and resource observations"><table><thead><tr><th>Variant</th><th>Median generation</th><th>p95</th><th>Peak allocated</th><th>Repetition flags</th><th>Truncations</th></tr></thead><tbody id="resourceTable"></tbody></table></div><p>Latency measures generation, excluding upload/HTTP and input transfer. Original peak VRAM covers its remaining 773-example process; pilot and selected peaks cover their complete 3,000-example passes. Hardware and thermal conditions varied.</p><p id="diagnosticNote"></p></details></div></section> | |
| <section class="analysis-section" id="external"><div class="eyebrow">PUBLISHED CONTEXT · CHECKED 15 OCTOBER 2026</div><h2>Benchmarks have boundaries.</h2><p>External tables below use their publishers’ datasets, units and protocols. Akshara was not evaluated on those exact external test sets. Scores from different tables cannot form one worldwide leaderboard.</p> | |
| <div class="benchmark-grid"><article class="benchmark-panel"><h3>IndicOCR-HW · Telugu</h3><p>Vendor score: 100 × (1 − WER). <a href="https://huggingface.co/bodhan-ai/indic-ocr#indicocr-hw-handwriting-accuracy-by-language-higher-is-better" target="_blank" rel="noopener">Model-card source ↗</a></p><div class="table-wrap" tabindex="0"><table><thead><tr><th>Order in this row</th><th>Model</th><th>Score ↑</th></tr></thead><tbody id="vendorTable"></tbody></table></div><p>Akshara: not evaluated on this benchmark. Its 76.20% local exact match must not be inserted into this table.</p></article><article class="benchmark-panel"><h3>IndicOCR’s published benchmark families</h3><div class="table-wrap" tabindex="0"><table><thead><tr><th>Published task</th><th>IndicOCR</th><th>Unit</th></tr></thead><tbody id="familyTable"></tbody></table></div><p>Vendor H100 80GB page pipeline: 6.34 pages/s at concurrency 128. Our RTX 3050 word timings measure a different workload. <a href="https://huggingface.co/bodhan-ai/indic-ocr#performance" target="_blank" rel="noopener">Model-card results ↗</a></p></article></div> | |
| <details class="published-detail"><summary>OmniDocBench v1.6_full · published top five</summary><p>Page-parsing composite across text, formulas and tables; higher is better. This is not Telugu word exact match. Table version retained as labeled by the repository. <a href="https://github.com/opendatalab/OmniDocBench#end-to-end-evaluation" target="_blank" rel="noopener">Official benchmark repository ↗</a></p><div class="table-wrap" tabindex="0"><table><thead><tr><th>Order by this score</th><th>Model</th><th>Overall ↑</th></tr></thead><tbody id="omniTable"></tbody></table></div></details> | |
| <details class="published-detail"><summary>ICDAR 2024 HWD · Telugu isolated words</summary><p>Competition teams, ranked here by published Telugu WRR. Different test inputs and recognition-rate definitions; these are not our grapheme-CER results. “IndicOCR Group” is a competition team name, not an identification of bodhan-ai’s checkpoint. <a href="https://ilocr.iiit.ac.in/icdar_2024_hwd/leaderboard.html" target="_blank" rel="noopener">Organizer leaderboard ↗</a></p><div class="table-wrap" tabindex="0"><table><thead><tr><th>Order by WRR</th><th>Team</th><th>CRR ↑</th><th>WRR ↑</th></tr></thead><tbody id="icdarTable"></tbody></table></div></details> | |
| <details class="published-detail"><summary>Published Telugu HTR · in-domain and unconstrained</summary><p>Paper-reported WER/CER on its IIIT dataset splits. Training budgets, unlabeled-data use and evaluation definitions differ from this capped study. No shared Akshara rank follows. <a href="https://cvit.iiit.ac.in/images/ConferencePapers/2025/Self_supervised_approach_for_improving_Indic_handwritten_OCR.pdf#page=6" target="_blank" rel="noopener">SemiHastakshar paper, Tables 4–5 ↗</a></p><div class="table-wrap" tabindex="0"><table><thead><tr><th>Domain</th><th>System</th><th>WER ↓</th><th>CER ↓</th></tr></thead><tbody id="paperTable"></tbody></table></div></details></section> | |
| <section class="analysis-section" id="reproducibility"><div class="eyebrow">REPRODUCIBILITY & LIMITS</div><h2>Keep the evidence with the model.</h2><div class="detail-grid"><div><h3>What is verified</h3><ul><li>Original 31-file checkpoint and frozen manifest hashes preserved.</li><li>Native BF16 recovery smoke reproduced uninterrupted adapter weights.</li><li>Separate latest resumable, best-validation and versioned selected artifacts.</li><li>Checksum/ZIP integrity, 85 software tests, actual 64/64 API parity and headed desktop/mobile checks.</li></ul></div><div><h3>What remains unknown</h3><ul><li>Writer identity and original-model pretraining overlap.</li><li>Complete human semantic verification of source labels.</li><li>Other datasets, camera-image robustness and whole-page accuracy.</li><li>Between-seed robustness; this study used one seed and one capped epoch.</li></ul></div></div><p>Selected output still has 714 non-exact final crops. Diagnostic flags are heuristics, not calibrated confidence. Constant-pixel abstention does not prove general blank-image detection. Backups are verified locally on D:; the original study did not establish an offsite backup. Hosting this report does not deploy GPU inference.</p><div class="download-row"><a class="button primary" href="akshara-study-report-20261015.md" download>Download full report ↓</a><a class="button" href="capped-study-metrics-20261015.csv" download>Final metrics CSV ↓</a><a class="button" href="capped-study-results-20261015.json" download>Evidence snapshot JSON ↓</a></div><details><summary>Frozen artifact identities</summary><pre id="artifactHashes"></pre></details></section> | |
| </main><footer class="analysis-footer"><span>Akshara · Word-crop research · Local Windows measurements</span><span>Handwriting data: <a href="https://ilocr.iiit.ac.in/dataset/23/" target="_blank" rel="noopener">IIIT Hyderabad</a> · <a href="https://creativecommons.org/licenses/by/4.0/" target="_blank" rel="noopener">CC BY 4.0</a>. Original model has separate license terms.</span></footer></div><script src="analysis.js?v=20261015.1"></script></body></html> | |