File size: 17,957 Bytes
f9bc0c1 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 | # Nepal Multilingual Document OCR Goal
This is the research target after the steering decision on 2026-06-16. The
project is now a PaddleOCR-centered multilingual document OCR system for
languages used in Nepal, not a narrow Nepali/English OCR experiment.
## Goal
Build and validate a PaddleOCR-based document OCR engine for at least ten
languages spoken in Nepal, with Limbu first, that supports:
- page/PDF/image parsing
- text OCR
- reading order
- table reconstruction
- figure and caption extraction
- Markdown export
- structured JSON export
- selective PaddleOCR-VL/VLM fallback for hard pages
The claim is valid only if the engine beats strong baselines on predefined real
and held-out evaluations. Synthetic-only results are not enough.
## Research Claim Shape
The paper should not claim "best OCR engine overall." The defensible claim is:
> A script-aware PaddleOCR adaptation framework can substantially improve
> document OCR and structured extraction for low-resource Nepal languages,
> including scripts not covered well by stock OCR systems, while preserving
> Nepali/English performance and using selective VLM fallback only when the
> local pipeline is uncertain.
The core contribution is the system and validation recipe:
1. Language/script registry for Nepal document OCR.
2. Gorkhapatra/Naya Nepal real-publication validation pack.
3. Unicode/font/renderability audit before synthesis.
4. Real-text synthetic data generation for lines, tables, captions, and pages.
5. PaddleOCR recognizer fine-tuning by script family.
6. PP-Structure-style document extraction.
7. Claim review that rejects weak SOTA statements.
## Primary Engine Direction
Use PaddleOCR as the engine family:
- **PP-OCRv6** for the modern small/local OCR baseline and fine-tuning target.
- **PP-StructureV3** for layout, reading order, tables, and document structure.
- **PaddleOCR-VL-1.6** as fallback/teacher for charts, figures, hard layouts,
seals, and low-confidence pages.
Do not route everything to a VLM and call that the engine. The local PaddleOCR
path must be the default path; fallback usage must be measured and reported.
## First Ten Language Targets
The first target set should be driven by actual Gorkhapatra/Naya Nepal
availability plus local corpus availability. OCR support is tracked by script,
Unicode, fonts, and evaluation data, not by language name alone.
| Priority | Language | First script target | Why first | Notes |
| --- | --- | --- | --- | --- |
| 1 | Limbu/Yakthung | Limbu/Sirijonga, U+1900..U+194F | Has own Unicode block and local Limbu resources | First non-Devanagari research target |
| 2 | Nepali | Devanagari | Core language and baseline | Must not regress |
| 3 | English | Latin | Mixed documents and forms | Must not regress |
| 4 | Tamang | Devanagari first; Tamyig audit later | Local NepTam corpus and dictionary | Start with Devanagari Tamang |
| 5 | Magar | Devanagari first | Gorkhapatra item confirmed; local dictionary | Own/community script requires separate audit |
| 6 | Wambule | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 7 | Bantawa/Rai | Devanagari or Kirat Rai depending source | Gorkhapatra item confirmed; Rai resources on remote/local projects | Kirat Rai Unicode support is newer; font audit required |
| 8 | Kulung | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 9 | Yakkha | Devanagari first | Gorkhapatra item confirmed | Rai-family validation candidate |
| 10 | Tharu/Rana Tharu | Devanagari first | Gorkhapatra item confirmed | Multiple varieties; keep labels explicit |
Additional Gorkhapatra-visible candidates include Sanskrit, Magahi, Bajjika,
Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami, Sherpa, Achhami, Bote,
Rajbanshi, Hayu, Santhali, Dhimal, Bhote, Uraon, Kumal, Gurung, Khash Jumli,
and Meche. These should enter the registry after script/corpus/font checks.
## Source Findings So Far
### Gorkhapatra Naya Nepal
The category page is useful for language-specific validation:
- `https://gorkhapatraonline.com/categories/new-nepal`
- `https://gorkhapatraonline.com/categories/new-nepal?page=2`
Confirmed visible language-specific entries include Magar, Wambule, Rana Tharu,
Magahi, Bajjika, Sunuwar, Danuwar, Majhi, Lhomi, Bhojpuri, Kisan, Thami,
Sherpa, Achhami, Bote, Rajbanshi, Hayu, Santhali, Kulung, Yakkha, Meche, Khash
Jumli, Dhimal, Tharu, Bhote, Uraon, Bantawa, Kumal, Gurung, and others.
The blog/category pages expose image URLs and article URLs. They are useful for
real-world OCR samples, but they should not be trusted as clean text references
without manual or tool-assisted verification.
### Gorkhapatra Epaper
The epaper list endpoint exposes viewer links:
```text
https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra
```
The visible links look like:
```text
https://epaper.gorkhapatraonline.com/pdf?file=/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf
```
That `pdf?...` URL returns a PDF.js viewer shell. The real PDF is downloaded by
fetching the file parameter directly:
```text
https://epaper.gorkhapatraonline.com/uploads/file/2026/6/gorkhapatra/2026-06-14-11-49-50-2083-03-01.pdf
```
On `k2`, this was verified as a real PDF:
```text
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf
PDF document, version 1.6
size: 48M
```
The `k2` audit produced:
```text
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/audit.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/new-nepal-language-items.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/epaper-assets.txt
/Users/k2/ocr-tech/outputs/gorkhapatra-source-audit/sample.pdf
```
`k2` does not currently expose `pdfinfo` or `pdftotext`. `pymupdf` was installed
into `/Users/k2/ocr-tech/.venv` on 2026-06-16 as the immediate PDF audit tool.
The sample epaper PDF opens successfully with PyMuPDF:
```text
page_count: 26
total extracted text chars: 309924
preview image: outputs/gorkhapatra-source-audit/sample-page-1-preview.jpg
audit: outputs/gorkhapatra-source-audit/sample-pymupdf-audit.json
text: outputs/gorkhapatra-source-audit/sample-pymupdf-text.txt
```
The extracted text layer is not reliable reference text. The first-page preview
contains mojibake/transliterated-looking output for Nepali content rather than
clean Unicode Devanagari. Therefore the epaper PDFs are useful as real page
images/layout sources, but their embedded text layer must be audited per file
and must not be treated as ground truth without verification.
### Local And Remote Corpus Hints
Local machine:
- `~/github/llm/ltk/tamang`: NepTam train/test, NepTam80K, dictionary, TTS
lexicon.
- `~/github/llm/ltk/limbu`: Limbu dictionary, Pangloss IGT, Limbu G2P.
- `~/github/llm/ltk/magar`: Western Magar dictionary and resource report.
Remote `cdjk@100.110.148.43`:
- `/Volumes/TRANSCEND` has about 2.0 TiB free.
- `~/github/llm` contains additional language projects:
`dhimal-toolkit`, `gurung-language-toolkit`, `limbu-speech-toolkit`,
`magar-toolkit`, `rai-speech-toolkit`, `nllb-better`, `nepa-newa-text-frontend`,
`new-place`, and `loksewa.zip`.
- `nllb-better` has Rai-family generated/validation artifacts and a language
registry that should be inspected before building the OCR registry.
Reproducible inventory gate:
- `ocrtech audit-synthesis-resources` records file counts, suffix counts,
sample hashes, and Unicode block hints before any converter is written.
- The first local LTK audit is documented in
`docs/SYNTHESIS_RESOURCE_AUDIT.md`.
- A source-only copy was synced to `/tmp/ocr-tech` on
`cdjk@100.110.148.43`; remote toolkit and `/Volumes/TRANSCEND` audits are
documented in `docs/SYNTHESIS_RESOURCE_AUDIT.md`.
- The first schema-specific converter, `ocrtech prepare-limbu-limdic-text`,
emits source-hashed Limbu Limdic synthesis-text JSONL after verifying actual
JSON/TSV/Firebase schemas.
- `ocrtech render-synthesis-text-lines` turns verified synthesis-text JSONL into
PaddleOCR-style line PNGs and label files with font SHA provenance.
- `ocrtech audit-rendered-synthesis-lines` verifies rendered synthetic line
manifests before training bundles: image hashes, PaddleOCR labels, source
schema provenance, font provenance when required, and
`claim_evidence_eligible=false`.
## Data Plan
### Real Validation Data
Use Gorkhapatra/Naya Nepal as the central real-publication validation source.
Create a scraper that emits:
- article URL
- article title
- language label extracted from title
- article image URLs
- epaper date
- epaper PDF URL
- direct PDF file URL
- image/PDF hashes
- source timestamp
- license/terms note
- manual-review status
Then create a hand-labeled validation pack:
- at least 10 languages
- at least 20 pages or page crops per language for v1 if available
- Limbu set first
- no synthetic samples in the final real validation pack
- references include text, line boxes, reading order, table cells, captions, and
Markdown where applicable
### Synthetic Training Data
Use real text when synthesizing:
- Limbu dictionary/Pangloss/IGT text first.
- Tamang NepTam and dictionary text.
- Devanagari text from Gorkhapatra/Loksewa/government-like documents where
licensing permits.
- Rai-family text from verified local/remote corpora.
Synthetic generation must produce:
- line crops
- word crops
- table cell crops
- captions
- full pages
- degraded scan/photo variants
- PaddleOCR recognition labels
- document JSON/Markdown references
Every synthetic sample must carry:
- language
- script
- Unicode ranges observed
- font
- source text hash
- rendered image hash
- template id
- degradation id
- train/eval split id
### Handwriting Synthesis
Handwriting is not a v1 blocker. It can be a later ablation.
Possible direction:
- Use available fonts and handwritten-style fonts for cheap diversity.
- Explore diffusion/DM-based handwriting generation only after print OCR is
stable.
- For Limbu, do not synthesize handwriting until we have real handwritten or
community-reviewed glyph examples. Otherwise we risk generating plausible but
wrong orthography.
If we explore One-DM or related diffusion methods, it should be framed as
optional augmentation, not the main claim.
## Validation Gates
### Gate 0: Source Audit
Pass only if:
- Gorkhapatra article image URLs are downloadable.
- Epaper direct PDFs are downloadable.
- PDF text/image extraction path is implemented and audited.
- Every source artifact is hashed.
- Source license/terms risk is recorded.
### Gate 1: Language Registry
Pass only if every target language has:
- language name
- ISO code if known
- script name
- Unicode range
- font list
- corpus source
- renderability audit
- normalization policy
- PaddleOCR dictionary path
- evaluation availability
### Gate 2: Limbu First
Pass only if:
- Limbu Unicode rendering works with selected fonts.
- Limbu PaddleOCR dictionary exists.
- Limbu synthetic crop/page generator emits valid labels.
- Limbu held-out real or manually reviewed eval exists.
- Stock baselines are recorded before fine-tuning.
- Claim-grade recognizer bundles use `--require-eval-real`; rendered synthetic
Limbu rows may train/ablate but must not appear in held-out eval evidence.
- Gorkhapatra language-page candidates are promoted through
`ocrtech finalize-gorkhapatra-language-page-review` only after accepted review
rows have verified structured reference JSON with text, reading order, tables,
and figures.
- `ocrtech audit-gorkhapatra-language-page-review` must pass before
finalization: exactly one page candidate is accepted per
publication/language/date group, and accepted rows point to reference files.
Use `--require-verified-references` for the final pre-claim gate.
- `ocrtech prepare-gorkhapatra-language-page-reference-templates` creates draft
structured references for reviewers without making them claim-eligible.
### Gate 3: PaddleOCR Baselines
Run before any claim:
- stock PP-OCRv6
- stock PP-OCRv5 if available
- Tesseract where traineddata exists
- Surya
- GLM-OCR
- PaddleOCR-VL-1.6
### Gate 4: Fine-Tuned Candidate
The candidate must beat baselines on:
- CER/WER per language
- strict and normalized Unicode CER
- exact-line accuracy
- reading order
- table text and structure
- figure/caption detection
- Markdown reconstruction
- JSON artifact completeness
English and Nepali must not regress versus the chosen stock baseline.
### Gate 5: Claim Review
The paper claim is allowed only if:
- all public/real validation rows have complete artifacts
- fallback usage is reported separately
- synthetic-only improvements are not used as SOTA evidence
- failures are included in error analysis
- a reproducibility bundle can rerun the benchmark
## Immediate Implementation Steps
1. Add `ocrtech audit-gorkhapatra-source`. **Implemented:** the command audits
Naya Nepal article images and Gorkhapatra epaper PDF links, resolves PDF.js
viewer URLs to direct `/uploads/file/...pdf` URLs, optionally downloads and
hashes assets, and records PyMuPDF text-layer metadata when available.
2. Add `ocrtech prepare-gorkhapatra-pack`. **Implemented:** the command turns a
source-audit JSON file into a pending-reference validation manifest, copies
audited article images, renders audited epaper PDF pages when PyMuPDF is
available, writes placeholder reference JSON files, and marks samples as
`reference_status=pending_manual_label` and
`claim_evidence_eligible=false` until references are verified.
3. Add `languages/nepal-v1.json` registry with Limbu first and at least ten
target languages. **Implemented:** the registry tracks Limbu/Sirijonga first,
more than ten Nepal-language targets, English as a non-counted regression
language, script/Unicode ranges, font candidates, corpus sources, Gorkhapatra
labels, validation status, and PaddleOCR dictionary targets. The
`ocrtech audit-language-registry` command validates the registry and writes
JSON/Markdown audit artifacts.
4. Add renderability audit for Unicode ranges and fonts. **Implemented:** the
`ocrtech audit-font-renderability` command reads TTF/OTF/TTC cmap tables,
checks registered font candidates against each script's Unicode ranges, and
writes JSON/Markdown evidence. Raster sample rendering is supported when
Pillow is installed. Current cmap evidence verifies Limbu, Devanagari,
Santhali/Ol Chiki, and Latin coverage when the registered runtime font assets
are present.
5. Add target-language readiness audit. **Implemented:** the
`ocrtech audit-language-readiness` command combines `languages/nepal-v1.json`
with passed `synthesis-text-manifest-audit.json` evidence and writes a
per-language checklist. Current run:
```bash
uv run ocrtech audit-language-readiness \
languages/nepal-v1.json \
--out outputs/language-readiness-audit-2026-06-16 \
--synthesis-text-audit outputs/limbu-unicode-text-smoke-audit-2026-06-16/synthesis-text-manifest-audit.json \
--synthesis-text-audit outputs/tamang-text-smoke-audit-2026-06-16/synthesis-text-manifest-audit.json \
--synthesis-text-audit outputs/magar-text-smoke-audit-2026-06-16/synthesis-text-manifest-audit.json \
--synthesis-text-audit outputs/remote-bible-brain-readiness-2026-06-16/synthesis-text-manifest-audit.json \
--synthesis-text-audit outputs/remote-bible-brain-readiness-extra-2026-06-16/synthesis-text-manifest-audit.json
```
Result: `passed=True target_languages=17 synthesis_ready=11/1`. Limbu is
synthesis-text ready with 200 audited samples, Tamang is synthesis-text ready
with 300 audited Devanagari samples, and Magar is synthesis-text ready with
200 audited lexicon-derived samples. Remote Bible Brain adds 80 audited
Devanagari samples each for Nepali, Rana Tharu, and Kulung, plus 60 audited
samples each for Gurung, Dhimal, Sunuwar, Rajbanshi, and Sherpa. Every other
target language is still a source-candidate or registry-only item until a
converter and
`audit-synthesis-text-manifest` run exists.
6. Add Limbu synthetic line/page generator.
7. Add PP-OCRv6 stock baseline runner.
8. Build the first validation pack through Gorkhapatra/Naya Nepal publication
date alignment: extract Naya Nepal card dates, match the same-date
Gorkhapatra epaper PDF, locate the full `भाषा पृष्ठ` page, render that PDF
page, and keep it pending manual reference labeling. See
[Gorkhapatra Language Page Alignment](GORKHAPATRA_LANGUAGE_PAGE_ALIGNMENT.md).
9. Keep synthesis resource audits current on `cdjk@100.110.148.43`, especially
the 4TB SSD paths `/Volumes/TRANSCEND` and `~/github/llm`, before creating
real-text synthesis manifests.
10. Use `audit-synthesis-resources` output as the inventory gate; then build
schema-specific converters for Limbu, Tamang, Magar, and remote Bible Brain
resources only after actual fields are verified.
## External References
- PaddleOCR docs: <https://paddlepaddle.github.io/PaddleOCR/main/en/index.html>
- PaddleOCR repo: <https://github.com/PaddlePaddle/PaddleOCR>
- PP-StructureV3 docs:
<https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/pipeline_usage/PP-StructureV3.html>
- PaddleOCR-VL-1.6 docs:
<https://paddlepaddle.github.io/PaddleOCR/main/en/version3.x/algorithm/PaddleOCR-VL/PaddleOCR-VL-1.6.html>
- Gorkhapatra Naya Nepal category:
<https://gorkhapatraonline.com/categories/new-nepal>
- Gorkhapatra Naya Nepal publication:
<https://gorkhapatraonline.com/publications/naya-nepal>
- Gorkhapatra epaper list example:
<https://epaper.gorkhapatraonline.com/epaper-list?pdfdaterange1=2026-06-09&pdfdaterange2=2026-06-16&slug=gorkhapatra>
- Unicode Limbu chart: <https://www.unicode.org/charts/PDF/U1900.pdf>
- Kirat Rai Unicode proposal:
<https://www.unicode.org/L2/L2022/22043-kirat-rai.pdf>
|