docs(audit): complete academic precision and reproducibility cleanup

ACADEMIC_AUDIT.md

@@ -120,13 +120,13 @@ To adhere to rigorous academic scholarship, multiple unverified or hyperbolic as
 To situate this project appropriately within existing scientific literature, a dedicated related work section and bibliography were added citing seminal contributions:
 
 1. **Nguyen, M.-T., Schweyer, A.-V., Le, T.-L., Tran, T.-H., & Vu, H. (2019a)**: *Preliminary Results on Ancient Cham Glyph Recognition from Cham Inscription Images*. In 2019 International Conference on Multimedia Analysis and Pattern Recognition (MAPR 2019), IEEE, pp. 1–6. DOI: 10.1109/MAPR.2019.8743540. (Evaluated HOG, NPW, and CNN-derived features with k-NN and linear SVM).
-2. **Nguyen, M.-T., Schweyer, A.-V., Le, T.-L., Tran, T.-H., & Vu, H. (2019b)**: *Improving Ancient Cham Glyph Recognition Using Data Augmentation and Transfer Learning*. In International Conference on Image Analysis and Processing (ICIAP 2019 Workshops: PatReCH), Springer, Cham, LNCS 11808, pp. 115–125. DOI: 10.1007/978-3-030-30754-7_12. (Evaluated data augmentation and transfer learning from rendered synthetic glyph variants to weathered stone inscriptions).
+2. **Nguyen, M.-T., Schweyer, A.-V., Le, T.-L., Tran, T.-H., & Vu, H. (2019b)**: *Improving Ancient Cham Glyph Recognition Using Data Augmentation and Transfer Learning*. In International Conference on Image Analysis and Processing (ICIAP 2019 Workshops: PatReCH), Springer, Cham, LNCS 11808, pp. 115–125. DOI: 10.1007/978-3-030-30754-7_12. (Evaluated data augmentation and transfer learning from scripts of similar or the same language family to ancient stone inscription glyphs).
 3. **Nguyen, T.-N. (2023)**: *Segmentation, Recognition and Indexing of Cham characters in Cham documents* (*Segmentation, reconnaissance et indexation des caractères Cham dans les documents Cham*). Ph.D. Dissertation, Université de La Rochelle, France.
 4. **Nguyen, T.-N., Burie, J.-C., Le, T.-L., & Schweyer, A.-V. (2022)**: *An effective method for text line segmentation in historical document images*. In 26th International Conference on Pattern Recognition (ICPR 2022), IEEE, pp. 2686–2692. DOI: 10.1109/ICPR56361.2022.9956617.
 5. **Nguyen, T.-N., Burie, J.-C., Le, T.-L., & Schweyer, A.-V. (2023)**: *A Two-Step Sequence Transformer Based Method for Cham to Latin Script Transliteration*. In Proceedings of the 7th International Workshop on Historical Document Imaging and Processing (HIP@ICDAR 2023), ACM, pp. 25–30. DOI: 10.1145/3604951.3605525.
 6. **Nguyen, T.-N., Burie, J.-C., Le, T.-L., & Schweyer, A.-V. (2025)**: *Text line segmentation approach combining deep learning model and traditional image processing techniques - application to transliteration of Cham manuscripts*. *Multimedia Tools and Applications*, Springer. DOI: 10.1007/s11042-025-20602-x. (Evaluates line segmentation and transliteration across 627 images, ~8,300 lines).
 7. **ANR CHAMDOC Project (2019–2024)**: *Cham Documentation (CHAMDOC)*, Project ANR-19-CE27-0018, Agence Nationale de la Recherche, France.
-8. **Digitization Archives**: *Historic Cham Manuscripts of Vietnam* collection (Southeast Asia Digital Library / SEADL, Northern Illinois University, preserving 977 digitized manuscripts, >57,800 page scans) and Cham community collections (~3,000 physical manuscripts across Vietnam and Cambodia).
+8. **Digitization Archives**: The Historic Cham Manuscripts of Vietnam collection in the Southeast Asia Digital Library (SEADL) contains 977 digitized manuscripts comprising more than 57,800 page scans. Cham community collections preserve an estimated 3,000 physical manuscripts across Vietnam and Cambodia.
 
 ---
 
@@ -134,9 +134,9 @@ To situate this project appropriately within existing scientific literature, a d
 
 The project openly documents four major technical limitations:
 
-1. **Synthetic-to-Real Domain Gap**: The neural recognizer is trained primarily on synthetic textline images rendered with digital TrueType fonts (*Noto Sans Cham*, *Cham Roman*, *EFEO Cham*). Real manuscripts feature idiosyncratic scribal handwriting, varying pen pressure, and ink bleed that may degrade recognition accuracy in archival field settings.
+1. **Synthetic-to-Real Domain Gap**: The neural recognizer is trained predominantly on synthetic textline images rendered with bundled *Noto Sans Cham* (SIL OFL 1.1). Real manuscripts feature idiosyncratic scribal handwriting, varying pen pressure, and ink bleed that may degrade recognition accuracy in archival field settings.
 2. **Absence of a Large-Scale Annotated Real Manuscript Benchmark**: To date, there is no publicly available, character-level annotated historical Cham manuscript dataset of sufficient volume. Curating such a corpus requires close, sustained collaboration with native Cham scholars, paleographers, and community elders.
-3. **Extreme Biological Degradation & Epigraphy**: Inscription rubbings and severely mold-damaged palm-leaves (where $>40\%$ of glyph strokes are eroded) cannot be resolved by visual CTC sequence decoders alone without integrating language modeling priors (MLMs).
+3. **Extreme Biological Degradation & Epigraphy**: Inscription rubbings and severely mold-damaged palm-leaves (where character ink has substantially eroded) cannot be resolved by visual CTC sequence decoders alone without integrating language modeling priors (MLMs).
 4. **Interlinear Overlap Breaking Point (Level 5)**: When sinusoidal document warping exceeds line spacing ($\text{Amplitude} > \text{Gap}$), 1D projection and horizontal bounding boxes break down, necessitating future research into 2D polygonal baseline tracking.
 
 ---
@@ -154,16 +154,19 @@ The project openly documents four major technical limitations:
 | File Audited | Changes Applied | Status | Verification Mechanism |
 | :--- | :--- | :---: | :--- |
 | `LICENSE` | Clean MIT License for code; pointers to DATA_PROVENANCE.md | **Verified** | Standard legal wording |
-| `DATA_PROVENANCE.md` | Documented text corpora, fonts (*Noto Sans Cham*, *EFEO Cham*), rights disclaimer, and synthetic splits | **Verified** | Created & linked across docs |
-| `RESEARCH.md` | Reconciled title, scope, Unicode order, citations (DOIs, ICPR 2022, 2025 paper), detector parameters, manuscript counts, 25/20/15/15/15/10 splits | **Verified** | 100% matched with benchmarks |
-| `ocr-studio/research.html` | Verbatim synchronization with RESEARCH.md; fixed pre-vowels (O/AI only); updated tables and citations | **Verified** | `node --check` static JS syntax validated (Code 0) |
+| `DATA_PROVENANCE.md` | Documented text corpora, fonts (*Noto Sans Cham*, external reference fonts), rights disclaimer, and dual-generator architecture | **Verified** | Created & linked across docs |
+| `RESEARCH.md` | Reconciled title, scope, Unicode order, citations (DOIs, ICPR 2022, 2025 paper), detector parameters, manuscript counts, dual generators (Pipeline A baseline 25/20/15/15/15/10 and Pipeline B V25 5-pillar) | **Verified** | 100% matched with benchmarks |
+| `ocr-studio/research.html` | Verbatim synchronization with RESEARCH.md; updated title, tag pill, abstract grammar, citations, and dual generators | **Verified** | `node --check` static JS syntax validated (Code 0) |
 | `ocr-studio/index.html` | Updated model picker, Tab 1 Unicode, Tab 5 empirical benchmarks | **Verified** | `node --check` static JS syntax validated (Code 0) |
-| `README.md` & `README_VI.md` | Standardized title, qualified scope to Eastern Cham, updated synthetic splits and font classifications | **Verified** | Synchronized across languages |
+| `README.md` & `README_VI.md` | Standardized title, qualified scope to Eastern Cham, updated synthetic pipeline to dual generators | **Verified** | Synchronized across languages |
 | `generate_data.py` | Fixed PRE_SIGNS (`ꨯꨰ`), MEDIAL_SIGNS (`ꨴꨵꨳꨶ`), cleaned VOWEL_DIACRITIC_SIGNS, enforced canonical Unicode order | **Verified** | Unit tests passing |
+| `generate_data_v25.py` | Fixed docstring (140k/10k), renamed `MINIMAL_PAIRS_UE_WA` (`ꨲ` UE vs `ꨶ` WA), synchronized with manifest | **Verified** | Code inspection & syntax clean |
+| `v25_dataset_manifest.json` | Reconciled batch size (32/card, 64 global), stages (1-12, 13-22, 23-40), and diacritic naming with `rec_cham_v25.yml` | **Verified** | JSON syntax validated |
+| `TRAINING_ROADMAP_V25.md` | Aligned data distribution table to 5 pillars, updated execution plan stages to 1-12, 13-22, 23-40, fixed diacritic naming | **Verified** | Documented & reconciled |
 | `test_vol_roundtrip.py` | Added comprehensive test suites for VA initial/final, medials, pre-vowels, dependent vowels, stanzas | **Verified** | 100% test pass rate |
 | `validate_unicode_source_labels.py` | Fixed PRE_SIGNS, MEDIAL_SIGNS, and regex for diacritic validation | **Verified** | Script execution clean |
 | `paddleocr_cham_finetune.ipynb` | Synchronized character sets and canonical cluster reordering with generate_data.py | **Verified** | JSON validated |
-| `ACADEMIC_AUDIT.md` | Internal Documentation Audit & Reconciliation Log documenting all items | **Verified** | Fully documented |
+| `ACADEMIC_AUDIT.md` | Documentation Audit & Reconciliation Log tracking known issues and empirical uncertainties | **Verified** | Fully documented |
 
 ---
 
@@ -193,4 +196,4 @@ To maintain full transparency with academic collaborators and external reviewers
 
 ---
 
-**Conclusion**: The repository `phucsd/Cham-OCR` is in full compliance with rigorous academic, scientific, and paleographic standards. Zero ungrounded claims remain in the codebase.
+**Summary of Documentation & Verification Status**: The documentation across `phucsd/Cham-OCR` has been calibrated against actual code artifacts and empirical benchmark logs. Open empirical uncertainties and ongoing data collection needs remain cataloged above to guide future research and collaborative fieldwork.

DATA_PROVENANCE.md

@@ -14,10 +14,10 @@ This document provides a transparent, verifiable record of the provenance, right
 
 | Source / Title | Author / Editor / Institution | Web Reference / Archive | Acquisition Method | Rights / License Status | Redistributed in Repo? |
 | :--- | :--- | :--- | :--- | :--- | :---: |
-| *Akayet Inra Patra* (Classical verse epic) | Traditional Cham scribal transmission; modern editorial transliterations by Cham scholars | Academic & community preservation publications | Curated and transcribed from public domain cultural heritage texts | Traditional cultural heritage; public domain for classical text | Yes (as normalized Unicode textlines in `cham_text.txt`) |
-| *Ariya Po Pareng* (Classical narrative poem) | Traditional Cham scribal transmission | Cham cultural preservation archives | Curated and transcribed from traditional manuscripts | Traditional cultural heritage; public domain for classical text | Yes (normalized Unicode textlines in `cham_text.txt`) |
-| Historical chronicles & folk narratives (*Dalikal*) | Cham traditional oral and written literature | Cham community collections and regional cultural publications | Digitized and normalized into Unicode | Traditional cultural heritage | Yes (normalized Unicode textlines in `cham_text.txt`) |
-| Cham-Vietnamese-French Lexical Compendia | Historical missionary and linguistic wordlists (e.g. Aymonier & Cabaton 1906; Moussay 1971; modern pedagogical materials) | Academic library collections & SEADL references | Digital normalization into standard Unicode 16.0 Cham | Historical reference works / non-exclusive educational use | Yes (vocabulary tokens merged into `cham_text.txt`) |
+| *Akayet Inra Patra* (Classical verse epic) | Traditional Cham scribal transmission; modern editorial transliterations by Cham scholars | Academic & community preservation publications | Curated and transcribed from traditional cultural heritage texts | Traditional cultural heritage / specific edition rights unverified / non-commercial educational & preservation use (line-by-line edition mapping subject to archival verification) | Yes (as normalized Unicode textlines in `cham_text.txt`) |
+| *Ariya Po Pareng* (Classical narrative poem) | Traditional Cham scribal transmission | Cham cultural preservation archives | Curated and transcribed from traditional manuscripts | Traditional cultural heritage / specific edition rights unverified / non-commercial educational & preservation use (line-by-line edition mapping subject to archival verification) | Yes (normalized Unicode textlines in `cham_text.txt`) |
+| Historical chronicles & folk narratives (*Dalikal*) | Cham traditional oral and written literature | Cham community collections and regional cultural publications | Digitized and normalized into Unicode | Traditional cultural heritage / specific edition rights unverified / non-commercial educational & preservation use (line-by-line edition mapping subject to archival verification) | Yes (normalized Unicode textlines in `cham_text.txt`) |
+| Cham-Vietnamese-French Lexical Compendia | Historical missionary and linguistic wordlists (e.g. Aymonier & Cabaton 1906; Moussay 1971; modern pedagogical materials) | Academic library collections & SEADL references | Digital normalization into standard Unicode 16.0 Cham | Historical reference works / specific edition rights unverified / non-commercial educational & preservation use (line-by-line edition mapping subject to archival verification) | Yes (vocabulary tokens merged into `cham_text.txt`) |
 
 ### 1.2 Corpus Curation & Linguistic Preprocessing
 * **Orthographic Normalization**: Conversion into canonical Unicode 16.0 Cham representation (`U+AA00`–`U+AA5F`).
@@ -56,10 +56,10 @@ The following typefaces were evaluated as reference designs during research expe
 
 ## 3. Synthetic Document & Textline Datasets
 
-Due to the acute scarcity of character-level annotated historical Cham manuscript datasets, training and benchmark datasets were synthesized programmatically from `cham_text.txt`:
+Due to the acute scarcity of character-level annotated historical Cham manuscript datasets, training and benchmark datasets were synthesized programmatically from `cham_text.txt`. The repository maintains two distinct synthetic generation pipelines:
 
-### 3.1 150,000 Textline Synthesis Dataset (`scripts/generate_data.py`)
-The generator partitions the 150,000 synthetic corpus into **15% evaluation/locked splits** (22,500 lines) and **85% training split** (127,500 lines):
+### 3.1 Pipeline A: Baseline General Generator (`scripts/generate_data.py`)
+The baseline generator produces a 150,000 textline corpus partitioned into **15% evaluation/locked splits** (22,500 lines) and **85% training split** (127,500 lines):
 * **Evaluation & Locked Splits (15% total / 22,500 lines)**:
   - `val_clean_short` (3% / 4,500 lines)
   - `val_clean_long` (3% / 4,500 lines)
@@ -74,14 +74,22 @@ The generator partitions the 150,000 synthetic corpus into **15% evaluation/lock
   - `train_noisy_long` (**15%** / 19,125 lines): Long verse lines with severe photometric degradation and spatial elastic distortion.
   - `train_hard_examples` (**10%** / 12,750 lines): Adversarial minimal pairs (`ꨲ` vs `ꨶ`, `꩝꩝` vs `꩝`), verse numerals (1–99) with section marks (`꩑꩞`), and stacked subjoined clusters.
 
-### 3.2 Controlled 200-Page Synthetic Document Stress-Test (`ocr-benchmark/`)
+### 3.2 Pipeline B: Experimental V25 Generator (`scripts/generate_data_v25.py`)
+The V25 generator synthesizes 150,000 textline images (140,000 train + 10,000 val) synchronized with `configs/v25_dataset_manifest.json` across 5 core pillars:
+1. **Canonical Cham Literature (43.3% / 65,000 lines)**: Classical literary vocabulary (*Po Klong Garai*, 57-stanza verse).
+2. **Anti-Motion Blur Base (20.0% / 30,000 lines)**: Clean base images paired with dynamic in-RAM directional motion blur ($7\times 7$ to $13\times 13$) and defocus.
+3. **Inline Bilingual Code-Switching (16.7% / 25,000 lines)**: Intra-line mixed phrases with Vietnamese diacritics and Latin annotations.
+4. **Stanza Numerals & Boundary Punctuation (12.0% / 18,000 lines)**: Verse numbers $1$–$99$ (`꩑꩞`..`꩙꩙꩞`) and boundary marks (`꩞`, `:`, `–`).
+5. **Adversarial Minimal Pairs (8.0% / 12,000 lines)**: Micro-stroke pairs (`ꨲ` U+AA32 Vowel Sign UE vs `ꨶ` U+AA36 Medial WA, variable Double Danda `꩝꩝` spacing 2–8px, 3-tier diacritic stacks).
+
+### 3.3 Controlled 200-Page Synthetic Document Stress-Test (`ocr-benchmark/`)
 * 200 full-page synthetic documents comprising 903 ground-truth textlines across 5 difficulty levels (Standard, Aged Paper, Narrow Gap, Wavy Sinusoid, Extreme Overlap).
 * Generated by `ocr-benchmark/scripts/generate_benchmark_200.py` to establish the mathematical breaking point of line segmentation and CTC recognition.
 
-### 3.3 50-Test Stratified Diagnostic Suite (`ocr-studio/data/benchmark_50_tests_results.json`)
+### 3.4 50-Test Stratified Diagnostic Suite (`ocr-studio/data/benchmark_50_tests_results.json`)
 * 50 targeted synthetic image crops covering 10 distinct failure modes (noise, tilt, blur, diacritic confusion, low resolution).
 
-### 3.4 Rights Status of Synthetic Outputs
+### 3.5 Rights Status of Synthetic Outputs
 Synthetically rendered document images and transcriptions generated by repository scripts inherit the rights and usage status of the underlying linguistic corpora and fonts. The synthetic generator code and annotations produced by the project maintainers are provided under the project's [MIT License](LICENSE), while redistribution of generated textline images remains subject to the rights status of the respective underlying source texts.
 
 ---

README.md

@@ -38,10 +38,12 @@ An end-to-end deep learning research platform and paleographic transcription sys
    - Dual-head multi-task learning with **Connectionist Temporal Classification (CTC)** sequence alignment and **NRTR Multi-Head Cross-Attention** auxiliary decoding.
 3. **Canonical Unicode & Logical Order Normalization**:
    - Automatically parses recognized grapheme clusters via `normalize_unicode` to reconstruct standard **Brahmic Logical Order** (`Base Consonant + Medials + Pre-Vowels + Dependent Vowels + Vowel Lengthener AA + Finals / Signs / Final VA`), resolving local visual permutation artifacts.
-4. **150,000 Textline Synthetic Pipeline**:
+4. **150,000 Textline Synthetic Pipelines**:
    - Font map validation via `fontTools` cmap parsing, eliminating tofu/missing glyph artifacts.
    - Dynamic In-RAM augmentation applying directional motion blur on clean disk renders to prevent double-blur degradation.
-   - Calibrated dataset generator (`scripts/generate_data.py`) partitioning 150,000 textlines into 15% evaluation/locked splits and 85% training across 6 categories: clean short (25%), clean medium (20%), clean long (15%), noisy short (15%), noisy long (15%), and hard adversarial examples (10%).
+   - **Dual-Generator Architecture**:
+     - *Pipeline A (Baseline: `scripts/generate_data.py`)*: 150,000 textlines partitioned into 15% evaluation/locked splits and 85% training across 6 categories: clean short (25%), clean medium (20%), clean long (15%), noisy short (15%), noisy long (15%), and hard adversarial examples (10%).
+     - *Pipeline B (Experimental V25: `scripts/generate_data_v25.py`)*: 150,000 textlines (140,000 train + 10,000 val) synchronized with `configs/v25_dataset_manifest.json` across 5 core pillars: canonical Cham literature (43.3%), anti-blur base (20.0%), inline bilingual (16.7%), stanza numerals & punctuation (12.0%), and minimal pairs (8.0%).
 5. **Distributed Multi-GPU Cloud Training**:
    - Adaptive data-parallel execution (`paddle.distributed.launch --gpus '0,1'`) on Kaggle Nvidia Tesla T4x2 and Lightning Cloud A100 SXM4.
    - 3-Stage Checkpoint & Resume protocol ensuring uninterrupted progress across Kaggle's 12-hour session timeout limits.
@@ -172,13 +174,13 @@ Logged in `ocr-studio/data/benchmark_v24_vs_v25_results.json`:
 If you utilize Cham-OCR, our synthetic dataset pipeline, or the OCR Studio in your research, please cite:
 
 ```bibtex
-@software{cham_ocr_studio,
-  author = {Nguyen, Phuc},
-  title = {Cham OCR Studio: Deep Learning Pipeline and Paleographic Transcription Workbench for Historical Cham Manuscripts},
+@software{cham_ocr_pipeline,
+  author = {Nguyen, Phuc H.},
+  title = {Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition},
   year = {2026},
   publisher = {GitHub},
   url = {https://github.com/phucsd/Cham-OCR},
-  note = {Live web service: https://ocr.cham.asia}
+  note = {Production web service: https://ocr.cham.asia}
 }
 ```
 

README_VI.md

@@ -83,9 +83,11 @@ Cham-OCR/
    - Mạng nơ-ron **PP-OCRv4 SVTR-LCNet** mở rộng tensor đầu vào lên `[3, 48, 480]`.
    - Cơ chế giải mã kép: CTC Loss kết hợp NRTR Multi-Head Cross-Attention.
    - Chuẩn hóa thứ tự gõ Logic Chăm `normalize_unicode` (Brahmic Logical Order: Phụ âm cơ sở + Dấu phụ dưới chân/quấn RA/LA + Dấu phụ YA/WA + Nguyên âm đứng trước O/AI + Các nguyên âm phụ thuộc khác + Dấu kéo dài âm AA + Phụ âm cuối/Dấu ngắt).
-3. **Sinh Dữ liệu Tổng hợp & Tăng cường**:
-   - 150,000 dòng dữ liệu tổng hợp qua 4 gói phân tầng.
+3. **Hệ thống Sinh Dữ liệu Tổng hợp 150.000 dòng**:
    - Cơ chế lọc chữ lỗi ô vuông (tofu) bằng `fontTools` cmap và làm mờ động học trong RAM (*On-the-Fly Motion Blur*).
+   - **Kiến trúc Hai Bộ Sinh Dữ liệu (Dual Generators)**:
+     - *Pipeline A (Bộ sinh chuẩn: `scripts/generate_data.py`)*: 150.000 dòng chia 15% kiểm thử/đóng băng và 85% huấn luyện qua 6 nhóm: ngắn sạch (25%), vừa sạch (20%), dài sạch (15%), ngắn nhiễu (15%), dài nhiễu (15%), cặp đối kháng (10%).
+     - *Pipeline B (Bộ sinh thực nghiệm V25: `scripts/generate_data_v25.py`)*: 150.000 dòng (140.000 train + 10.000 val) đồng bộ trực tiếp với `configs/v25_dataset_manifest.json` qua 5 trụ cột: ngữ liệu chuẩn (43,3%), chống mờ rung tay (20,0%), song ngữ kẹp dòng (16,7%), số khổ & dấu câu (12,0%), cặp đối kháng vi mô (8,0%).
 4. **Triển khai Trực tuyến**:
    - Đóng gói container Docker trên **Hugging Face Spaces** (`phucsd/cham-ocr-studio`).
    - Tên miền dịch vụ chính thức: **[https://ocr.cham.asia](https://ocr.cham.asia)**.
@@ -115,13 +117,13 @@ Mọi số liệu công bố đều được đối chiếu trực tiếp từ c
 Nếu bạn sử dụng tài liệu, bộ sinh dữ liệu hoặc công cụ Cham OCR Studio trong nghiên cứu, vui lòng trích dẫn:
 
 ```bibtex
-@software{cham_ocr_studio,
-  author = {Nguyen, Phuc},
-  title = {Cham OCR Studio: Deep Learning Pipeline and Paleographic Transcription Workbench for Historical Cham Manuscripts},
+@software{cham_ocr_pipeline,
+  author = {Nguyen, Phuc H.},
+  title = {Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition},
   year = {2026},
   publisher = {GitHub},
   url = {https://github.com/phucsd/Cham-OCR},
-  note = {Live web service: https://ocr.cham.asia}
+  note = {Production web service: https://ocr.cham.asia}
 }
 ```
 

RESEARCH.md

@@ -115,7 +115,7 @@ However, pre-vowels such as `ꨯ` (Vowel Sign O, U+AA2F) and `ꨰ` (Vowel Sign A
 ### 2.1 Early Epigraphic & Glyph Recognition
 Automated processing of Cham script originated in the analysis of stone inscriptions from ancient Champa sanctuaries (Mỹ Sơn, Po Nagar, Đồng Dương). Nguyen, Schweyer, Le, Tran, & Vu (2019a) presented preliminary results on recognizing ancient Cham glyphs extracted from stone rubbings and photographs using Histogram of Oriented Gradients (HOG), Non-Parametric Weighted (NPW) features, and CNN-derived feature representations coupled with $k$-NN and linear SVM classifiers at the IEEE MAPR 2019 conference. Due to severe stone weathering, erosion, and irregular surface texture, isolated glyph classification across noisy background substrates was evaluated.
 
-To mitigate extreme sample scarcity in epigraphic corpora, Nguyen, Schweyer, Le, Tran, & Vu (2019b) evaluated data augmentation strategies and transfer learning from rendered synthetic character variants to ancient stone inscription glyphs at the ICIAP 2019 Workshops (PatReCH), demonstrating quantitative gains on synthetic-to-real transfer across weathered stone carvings.
+To mitigate extreme sample scarcity in epigraphic corpora, Nguyen, Schweyer, Le, Tran, & Vu (2019b) evaluated data augmentation strategies and transfer learning from scripts of similar or the same language family to ancient stone inscription glyphs at the ICIAP 2019 Workshops (PatReCH), demonstrating quantitative gains on script-to-inscription transfer across weathered stone carvings.
 
 ### 2.2 Deep Learning for Cham Epigraphy & Transliteration
 In his doctoral dissertation, Nguyen (2023) developed comprehensive document image processing and understanding methodologies dedicated to ancient Cham documents. His work investigated stone inscription image restoration, automatic glyph segmentation under rough stone substrate conditions, and deep convolutional representations for historical Brahmic scripts.
@@ -123,7 +123,7 @@ In his doctoral dissertation, Nguyen (2023) developed comprehensive document ima
 Complementing glyph recognition, Nguyen, Burie, Le, & Schweyer (2022) proposed an effective multi-stage method for text line segmentation in historical document images presented at the 26th International Conference on Pattern Recognition (ICPR 2022), addressing line curvature and touching diacritics in historical archives. Expanding to linguistic representation, Nguyen, Burie, Le, & Schweyer (2023) introduced a two-step sequence transformer model for Cham-to-Latin script transliteration presented at the HIP@ICDAR 2023 workshop. Their system demonstrated the efficacy of sequence-to-sequence transformers in mapping recognized Cham text sequences to standardized Latin phonological romanizations (*Cam-Latin*). Most recently, Nguyen, Burie, Le, & Schweyer (2025) published a comprehensive study entitled *Text line segmentation approach combining deep learning model and traditional image processing techniques - application to transliteration of Cham manuscripts* in *Multimedia Tools and Applications*, evaluating text-line segmentation and automatic transliteration across 627 manuscript images and approximately 8,300 textlines (focusing on line segmentation and transliteration rather than end-to-end character OCR benchmarking).
 
 ### 2.3 Manuscript Preservation & Digitization Initiatives
-Beyond epigraphy on stone, historical Cham manuscripts on paper (*kertas*) and palm-leaf have been documented through preservation missions. Most notably, the *Historic Cham Manuscripts of Vietnam* collection—hosted and preserved by the Southeast Asia Digital Library (SEADL) at Northern Illinois University—preserves 977 digitized manuscripts comprising more than 57,800 high-resolution page scans. In addition, field surveys estimate approximately 3,000 physical manuscripts preserved within Cham community families and religious dignitaries across Vietnam and Cambodia. The French National Research Agency funded the CHAMDOC project (*Cham Documentation*, ANR-19-CE27-0018, 2019–2024) to support preservation, cataloging, and linguistic documentation.
+Beyond epigraphy on stone, historical Cham manuscripts on paper (*kertas*) and palm-leaf have been documented through preservation missions. The Historic Cham Manuscripts of Vietnam collection in the Southeast Asia Digital Library (SEADL) contains 977 digitized manuscripts comprising more than 57,800 page scans. In addition, field surveys estimate approximately 3,000 physical manuscripts preserved within Cham community families and religious dignitaries across Vietnam and Cambodia. The French National Research Agency funded the CHAMDOC project (*Cham Documentation*, ANR-19-CE27-0018, 2019–2024) to support preservation, cataloging, and linguistic documentation.
 
 However, despite the preservation of tens of thousands of scanned folios, almost none of these archives possess open, machine-readable character-level or line-level bounding box ground-truth annotations suitable for training supervised deep learning models. The bottleneck for Cham OCR is therefore not an absolute absence of physical manuscripts, but rather the severe scarcity of annotated, verified training ground truth.
 
@@ -139,7 +139,7 @@ Prior research has focused predominantly on either **isolated glyph recognition
 ## 3. Text Detection & Indic Line Segmentation (Cham-DBNet)
 
 ### 3.1 The Interlinear Diacritic Collision Problem
-Historical Cham manuscripts frequently exhibit tight interlinear spacing (observed in synthetic and normalized rendering spaces at standard font scale as tight as 3–12 pixels between lines). Because upper dependent vowels reach high above the core text band (ascender excursions extending up to $+18	ext{px}$ at nominal 32–36px glyph heights) and subjoined medials plunge below (descender excursions down to $-22	ext{px}$), the ascenders of line $n+1$ routinely touch or intertwine with the descenders of line $n$ under dense formatting. Standard off-the-shelf text detection models (e.g., baseline DBNet trained on natural scene Latin or CJK text) frequently encounter two severe failure modes when applied to dense historical manuscripts:
+In digital modeling and synthetic text rendering at nominal glyph heights (32–36px), interlinear clearance often narrows to 3–12 pixels between adjacent lines (with ascender excursions extending up to $+18\text{px}$ and descender excursions down to $-22\text{px}$ in normalized coordinate space). Because upper dependent vowels reach high above the core text band and subjoined medials plunge below, the ascenders of line $n+1$ routinely touch or intertwine with the descenders of line $n$ under dense formatting. Standard off-the-shelf text detection models (e.g., baseline DBNet trained on natural scene Latin or CJK text) frequently encounter two severe failure modes when applied to dense historical manuscripts:
 1. **Line Merging**: Grouping adjacent lines into a single bounding box when diacritics touch.
 2. **Diacritic Amputation**: Slicing off upper vowel signs or subjoined medials during rectangular cropping, which causes fatal downstream recognition errors.
 
@@ -211,8 +211,11 @@ def normalize_unicode(text: str) -> str:
 ### 5.1 The Data Scarcity Bottleneck
 Deep neural recognition networks require hundreds of thousands of diverse labeled textline instances. However, surviving physical Cham manuscripts lack character-level bounding box ground truth. To overcome this limitation, we engineered a synthetic generation pipeline (`scripts/generate_data.py`) capable of synthesizing 150,000 diverse textline crops from authentic literary corpora.
 
-### 5.2 Calibrated Synthetic Dataset Partitioning
-The synthetic dataset generator (`scripts/generate_data.py`) partitions the target 150,000 textline synthesis corpus into **15% evaluation/locked splits** (22,500 lines: `val_clean_short` 3%, `val_clean_long` 3%, `val_noisy_short` 3%, `val_noisy_long` 3%, `locked_test` 3%) and **85% training split** (127,500 lines) structured across 6 calibrated sub-categories:
+### 5.2 Calibrated Synthetic Dataset Partitioning & Dual-Generator Architecture
+To support both baseline reproducibility and ongoing experimental modeling, the repository provides two distinct synthetic dataset generation pipelines:
+
+#### Pipeline A: Baseline General Generator (`scripts/generate_data.py`)
+The baseline generator produces a 150,000 textline corpus partitioned into **15% evaluation/locked splits** (22,500 lines: `val_clean_short` 3%, `val_clean_long` 3%, `val_noisy_short` 3%, `val_noisy_long` 3%, `locked_test` 3%) and an **85% training split** (127,500 lines) structured across 6 calibrated sub-categories:
 
 | Training Sub-Category | Proportion | Line Count | Target Composition & Protocol | Target Failure Mode Addressed |
 | :--- | :---: | :---: | :--- | :--- |
@@ -223,6 +226,14 @@ The synthetic dataset generator (`scripts/generate_data.py`) partitions the targ
 | `train_noisy_long` | **15%** | 19,125 | Long verse lines with severe photometric degradation and spatial elastic distortion. | Prevents alignment collapse on degraded multi-word lines. |
 | `train_hard_examples` | **10%** | 12,750 | Adversarial minimal pairs (`ꨲ` vs `ꨶ`, `꩝꩝` vs `꩝`), verse numerals (1–99) with section marks (`꩑꩞`), and stacked subjoined clusters. | Eliminates CTC misclassification of numerals as letters (`꩔` vs `ꨤ`, `꩕` vs `ꨅ`) and diacritic collapsing. |
 
+#### Pipeline B: Experimental V25 Generator (`scripts/generate_data_v25.py`)
+For the V25 experimental iteration, a dedicated multi-threaded generator (`generate_data_v25.py`) synthesizes 150,000 textline images (140,000 train + 10,000 validation) directly synchronized with `configs/v25_dataset_manifest.json` across 5 core pillars:
+1. **Canonical Cham Literature (43.3% / 65,000 lines)**: Classical literary vocabulary (*Po Klong Garai*, 57-stanza verse).
+2. **Anti-Motion Blur Base (20.0% / 30,000 lines)**: Clean base images paired with dynamic in-RAM directional motion blur ($7\times 7$ to $13\times 13$) and defocus.
+3. **Inline Bilingual Code-Switching (16.7% / 25,000 lines)**: Intra-line mixed phrases with Vietnamese diacritics and Latin annotations.
+4. **Stanza Numerals & Boundary Punctuation (12.0% / 18,000 lines)**: Verse numbers $1$–$99$ (`꩑꩞`..`꩙꩙꩞`) and boundary marks (`꩞`, `:`, `–`).
+5. **Adversarial Minimal Pairs (8.0% / 12,000 lines)**: Micro-stroke pairs (`ꨲ` U+AA32 Vowel Sign UE vs `ꨶ` U+AA36 Medial WA, variable Double Danda `꩝꩝` spacing 2–8px, 3-tier diacritic stacks).
+
 ### 5.3 Digital Typefaces & TrueType cmap Tofu Glyph Safeguards
 When rendering synthetic text, digital Cham typefaces must be verified for glyph coverage. The primary bundled and reproducible typeface distributed directly within this repository is **Noto Sans Cham** (`Regular`, `Bold`, `Black`; SIL Open Font License 1.1, located in `ocr-training/data/fonts/`). External reference typefaces evaluated during research—including *Cham Roman*, *EFEO Cham*, and community fonts—are not redistributed in this repository and must be acquired independently if researchers wish to reproduce specific historical font variations.
 
@@ -324,9 +335,9 @@ if not hasattr(np, 'typeDict'): np.typeDict = {}
 
 While our pipeline demonstrates substantial gains on synthetic and semi-controlled documents, rigorous academic honesty requires acknowledging the following boundaries:
 
-1. **Synthetic-to-Real Domain Gap**: The recognizer is trained predominantly on synthetic textlines rendered with available digital TrueType fonts (*Noto Sans Cham*, *Cham Roman*, *EFEO Cham*). Historical manuscripts feature idiosyncratic scribal hands, variable ink viscosity, and non-standard ligature variations that may exhibit lower recognition confidence in the field.
+1. **Synthetic-to-Real Domain Gap**: The neural recognizer is trained predominantly on synthetic textlines rendered with bundled *Noto Sans Cham* (SIL OFL 1.1). Historical manuscripts feature idiosyncratic scribal hands, variable ink viscosity, and non-standard ligature variations that may exhibit lower recognition confidence in the field.
 2. **Absence of Large-Scale Real Manuscript Ground Truth**: Due to the acute scarcity of digitized historical Cham archives with line-level annotations, current benchmarks rely on controlled synthetic stress-tests and curated test slices. Establishing an open, expert-verified historical palm-leaf manuscript benchmark represents an urgent necessity requiring future collaborative scholarship with native Cham elders and linguists.
-3. **Severe Biological Degradation & Epigraphy**: Inscription stone rubbings and severely mold-damaged palm leaves (where over 40% of character ink has flaked away) remain beyond the capabilities of pure vision-based CTC sequence models. Integrating masked language models (MLMs) trained on historical Cham corpora is an active subject of future study.
+3. **Severe Biological Degradation & Epigraphy**: Inscription stone rubbings and severely mold-damaged palm leaves (where character ink has substantially eroded) remain beyond the capabilities of pure vision-based CTC sequence models. Integrating masked language models (MLMs) trained on historical Cham corpora is an active subject of future study.
 4. **Extreme Interlinear Overlap (Level 5)**: When severe paper wrinkling causes interlinear wave amplitude to exceed line spacing ($\text{Amplitude} > \text{Gap}$), 1D projection and horizontal bounding boxes collapse, indicating the need for 2D polygonal baseline tracking models in future iterations.
 
 ---
@@ -334,13 +345,13 @@ While our pipeline demonstrates substantial gains on synthetic and semi-controll
 ## 9. BibTeX Citation & Academic References
 
 ```bibtex
-@software{cham_ocr_studio,
-  author = {Nguyen, Phuc},
-  title = {Cham OCR Studio: Deep Learning Pipeline and Paleographic Transcription Workbench for Historical Cham Manuscripts},
+@software{cham_ocr_pipeline,
+  author = {Nguyen, Phuc H.},
+  title = {Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition},
   year = {2026},
   publisher = {GitHub},
   url = {https://github.com/phucsd/Cham-OCR},
-  note = {Official web service: https://ocr.cham.asia}
+  note = {Production web service: https://ocr.cham.asia}
 }
 ```
 

ocr-studio/research.html

@@ -3,7 +3,7 @@
 <head>
     <meta charset="UTF-8">
     <meta name="viewport" content="width=device-width, initial-scale=1.0">
-    <title>End-to-End Deep Learning Pipeline & Paleographic Transcription Workbench for Historical Cham Manuscripts — Technical Report</title>
+    <title>Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition — Technical Report</title>
     
     <!-- Google Fonts: Google Sans Flex & JetBrains Mono -->
     <link rel="preconnect" href="https://fonts.googleapis.com">
@@ -750,10 +750,10 @@
                 <span class="tag-pill">Not Peer Reviewed</span>
                 <span class="tag-pill tag-pill-green">PaddleOCR PP-OCRv4</span>
                 <span class="tag-pill tag-pill-blue">Brahmic Paleography</span>
-                <span class="tag-pill">Akhar Thrah & Cam Srak</span>
+                <span class="tag-pill">Unicode Eastern Cham (Akhar Thrah)</span>
             </div>
 
-            <h1 class="paper-title">End-to-End Deep Learning Pipeline & Paleographic Transcription Workbench for Historical Cham Manuscripts</h1>
+            <h1 class="paper-title">Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition</h1>
 
             <div class="author-box">
                 <div class="author-info">
@@ -771,7 +771,7 @@
             <section id="abstract" class="abstract-box">
                 <div class="abstract-title">Abstract</div>
                 <p class="abstract-text">
-                    Historical manuscripts of the Cham civilization—primarily preserved in Eastern Cham (<em>Akhar Thrah</em>) alongside Western Cham (<em>Cam Srak</em>) traditions—represent a fragile, millennia-old Brahmic epigraphic and literary heritage in Southeast Asia. While this project validates an end-to-end deep learning OCR pipeline specifically on Unicode Eastern Cham (<em>Akhar Thrah</em>), Western Cham (<em>Cam Srak</em>), real physical manuscripts, and epigraphic stone carvings represent active and ongoing future extensions. Automatic transcription via Optical Character Recognition (OCR) has historically faced steep barriers due to complex Brahmic abugida orthography, multi-directional dependent vowel diacritics, predominantly continuous script (<em>scriptio continua</em>) in classical manuscripts, physical carrier degradation, and tight interlinear diacritic collisions. In this technical report, we describe the design, implementation, and empirical evaluation of an end-to-end deep learning pipeline and diagnostic transcription workbench engineered for Cham documents: (1) a hybrid text detection and line segmentation engine combining a fine-tuned <strong>Mobile DBNet (PPLCNetV3)</strong> with an <strong>Indic Valley-Cut Heuristic Engine</strong> that preserves boundary strokes via mathematical set-difference masking (<code>difference_update</code>) and \(0.40 \times \text{median line height}\) vertical safety cushions; (2) a neural recognition backbone based on <strong>PP-OCRv4 SVTR-LCNet</strong> with expanded input dimensions \([3, 48, 480]\), joint Connectionist Temporal Classification (CTC) alignment, and post-OCR deterministic Unicode cluster canonicalization; and (3) a 150,000-textline synthetic dataset pipeline featuring on-the-fly motion blur, TrueType cmap tofu filtering, and hard-example adversarial mining for verse numerals (\(1\)–\(99\)) and confusing diacritic minimal pairs. Evaluations across a <strong>controlled 200-page synthetic document stress-test</strong> (903 textlines across 5 difficulty levels) and a <strong>50-test stratified failure-mode benchmark</strong> demonstrate that our line detector demonstrate robust line detection with a <strong>100.0% line detection rate</strong> across standard, aged paper, and narrow-gap folios (Levels 1–3) and graceful degradation up to Level 4 (99.44% detection), before encountering an empirical breaking point at Level 5 (61.45% detection) where interlinear wave amplitude exceeds the interlinear gap. For textline recognition, our validated baseline model (<strong>V24</strong>) achieves <strong>13.88% mean CER</strong> across the 50-test suite (reducing noise CER from 50.59% down to 8.82% over V23), while experimental checkpoints (<strong>V25</strong>) remain under active training and evaluation. Multi-GPU distributed scaling benchmarks across Kaggle Tesla T4x2, Cloud A100, L40S, and L4 characterize hardware efficiency for non-profit cultural preservation. Finally, we discuss critical limitations regarding synthetic-to-real domain gaps and provide a production reference deployed at <code>https://ocr.cham.asia</code>.
+                    Historical manuscripts of the Cham civilization—primarily preserved in Eastern Cham (<em>Akhar Thrah</em>) alongside Western Cham (<em>Cam Srak</em>) traditions—represent a fragile, millennia-old Brahmic epigraphic and literary heritage in Southeast Asia. While this project validates an end-to-end deep learning OCR pipeline specifically on Unicode Eastern Cham (<em>Akhar Thrah</em>), Western Cham (<em>Cam Srak</em>), real physical manuscripts, and epigraphic stone carvings represent active and ongoing future extensions. Automatic transcription via Optical Character Recognition (OCR) has historically faced steep barriers due to complex Brahmic abugida orthography, multi-directional dependent vowel diacritics, predominantly continuous script (<em>scriptio continua</em>) in classical manuscripts, physical carrier degradation, and tight interlinear diacritic collisions. In this technical report, we describe the design, implementation, and empirical evaluation of an end-to-end deep learning pipeline and diagnostic transcription workbench engineered for Cham documents: (1) a hybrid text detection and line segmentation engine combining a fine-tuned <strong>Mobile DBNet (PPLCNetV3)</strong> with an <strong>Indic Valley-Cut Heuristic Engine</strong> that preserves boundary strokes via mathematical set-difference masking (<code>difference_update</code>) and \(0.40 \times \text{median line height}\) vertical safety cushions; (2) a neural recognition backbone based on <strong>PP-OCRv4 SVTR-LCNet</strong> with expanded input dimensions \([3, 48, 480]\), joint Connectionist Temporal Classification (CTC) alignment, and post-OCR deterministic Unicode cluster canonicalization; and (3) a 150,000-textline synthetic dataset pipeline featuring on-the-fly motion blur, TrueType cmap tofu filtering, and hard-example adversarial mining for verse numerals (\(1\)–\(99\)) and confusing diacritic minimal pairs. Evaluations across a <strong>controlled 200-page synthetic document stress-test</strong> (903 textlines across 5 difficulty levels) and a <strong>50-test stratified failure-mode benchmark</strong> demonstrate robust textline detection with a <strong>100.0% line detection rate</strong> across standard, aged paper, and narrow-gap folios (Levels 1–3) and graceful degradation up to Level 4 (99.44% detection), before encountering an empirical breaking point at Level 5 (61.45% detection) where interlinear wave amplitude exceeds the interlinear gap. For textline recognition, our validated baseline model (<strong>V24</strong>) achieves <strong>13.88% mean CER</strong> across the 50-test suite (reducing noise CER from 50.59% down to 8.82% over V23), while experimental checkpoints (<strong>V25</strong>) remain under active training and evaluation. Multi-GPU distributed scaling benchmarks across Kaggle Tesla T4x2, Cloud A100, L40S, and L4 characterize hardware efficiency for non-profit cultural preservation. Finally, we discuss critical limitations regarding synthetic-to-real domain gaps and provide a production reference deployed at <code>https://ocr.cham.asia</code>.
                 </p>
                 <div class="keywords-strip">
                     <strong>Keywords:</strong> Optical Character Recognition (OCR), Cham Script, Akhar Thrah, Cam Srak, Brahmic Paleography, Differentiable Binarization (DBNet), SVTR-LCNet, Connectionist Temporal Classification (CTC), Indic Line Segmentation.
@@ -852,7 +852,7 @@
                     Automated processing of Cham script originated in the analysis of stone inscriptions from ancient Champa sanctuaries (Mỹ Sơn, Po Nagar, Đồng Dương). Nguyen, Schweyer, Le, Tran, & Vu (2019a) presented preliminary results on recognizing ancient Cham glyphs extracted from stone rubbings and photographs using Histogram of Oriented Gradients (HOG), Non-Parametric Weighted (NPW) features, and CNN-derived feature representations coupled with $k$-NN and linear SVM classifiers at the IEEE MAPR 2019 conference. Due to severe stone weathering, erosion, and irregular surface texture, isolated glyph classification across noisy background substrates was evaluated.
                 </p>
                 <p>
-                    To mitigate extreme sample scarcity in epigraphic corpora, Nguyen, Schweyer, Le, Tran, & Vu (2019b) evaluated data augmentation strategies and transfer learning from rendered synthetic character variants to ancient stone inscription glyphs at the ICIAP 2019 Workshops (PatReCH), demonstrating quantitative gains on synthetic-to-real transfer across weathered stone carvings.
+                    To mitigate extreme sample scarcity in epigraphic corpora, Nguyen, Schweyer, Le, Tran, & Vu (2019b) evaluated data augmentation strategies and transfer learning from scripts of similar or the same language family to ancient stone inscription glyphs at the ICIAP 2019 Workshops (PatReCH), demonstrating quantitative gains on script-to-inscription transfer across weathered stone carvings.
                 </p>
 
                 <h3 id="sec2-2">2.2 Deep Learning for Cham Epigraphy & Transliteration</h3>
@@ -865,7 +865,7 @@
 
                 <h3 id="sec2-3">2.3 Manuscript Preservation & Digitization Initiatives</h3>
                 <p>
-                    Beyond epigraphy on stone, historical Cham manuscripts on paper (<em>kertas</em>) and palm-leaf have been documented through preservation missions. Most notably, the <em>Historic Cham Manuscripts of Vietnam</em> collection—hosted and preserved by the Southeast Asia Digital Library (SEADL) at Northern Illinois University—preserves 977 digitized manuscripts comprising more than 57,800 high-resolution page scans. In addition, field surveys estimate approximately 3,000 physical manuscripts preserved within Cham community families and religious dignitaries across Vietnam and Cambodia. The French National Research Agency funded the CHAMDOC project (<em>Cham Documentation</em>, ANR-19-CE27-0018, 2019–2024) to support preservation, cataloging, and linguistic documentation.
+                    Beyond epigraphy on stone, historical Cham manuscripts on paper (<em>kertas</em>) and palm-leaf have been documented through preservation missions. The Historic Cham Manuscripts of Vietnam collection in the Southeast Asia Digital Library (SEADL) contains 977 digitized manuscripts comprising more than 57,800 page scans. In addition, field surveys estimate approximately 3,000 physical manuscripts preserved within Cham community families and religious dignitaries across Vietnam and Cambodia. The French National Research Agency funded the CHAMDOC project (<em>Cham Documentation</em>, ANR-19-CE27-0018, 2019–2024) to support preservation, cataloging, and linguistic documentation.
                 </p>
                 <p>
                     However, despite the preservation of tens of thousands of scanned folios, almost none of these archives possess open, machine-readable character-level or line-level bounding box ground-truth annotations suitable for training supervised deep learning models. The bottleneck for Cham OCR is therefore not an absolute absence of physical manuscripts, but rather the severe scarcity of annotated, verified training ground truth.
@@ -889,7 +889,7 @@
 
                 <h3 id="sec3-1">3.1 The Interlinear Diacritic Collision Problem</h3>
                 <p>
-                    Historical Cham manuscripts frequently exhibit tight interlinear spacing (observed in synthetic and normalized rendering spaces at standard font scale as tight as 3–12 pixels between lines). Because upper dependent vowels reach high above the core text band (ascender excursions extending up to \(+18	ext{px}\) at nominal 32–36px glyph heights) and subjoined medials plunge below (descender excursions down to \(-22	ext{px}\)), the ascenders of line \(n+1\) routinely touch or intertwine with the descenders of line \(n\) under dense formatting. Standard off-the-shelf text detection models (such as baseline DBNet trained on natural scene Latin or CJK text) frequently encounter two severe failure modes when applied to dense historical manuscripts:
+                    In digital modeling and synthetic text rendering at nominal glyph heights (32–36px), interlinear clearance often narrows to 3–12 pixels between adjacent lines (with ascender excursions extending up to \(+18\text{px}\) and descender excursions down to \(-22\text{px}\) in normalized coordinate space). Because upper dependent vowels reach high above the core text band and subjoined medials plunge below, the ascenders of line \(n+1\) routinely touch or intertwine with the descenders of line \(n\) under dense formatting. Standard off-the-shelf text detection models (such as baseline DBNet trained on natural scene Latin or CJK text) frequently encounter two severe failure modes when applied to dense historical manuscripts:
                 </p>
                 <ul>
                     <li>Merging adjacent lines into single monolithic boxes.</li>
@@ -1090,9 +1090,14 @@
                     Deep neural OCR networks require hundreds of thousands of diverse labeled textlines. However, surviving physical Cham manuscripts lack open character-level bounding box ground truth. To overcome this limitation, we engineered an algorithmic synthesizer (<code>scripts/generate_data.py</code>) capable of rendering 150,000 diverse textline crops from authentic literary corpora.
                 </p>
 
-                <h3 id="sec5-2">5.2 Calibrated Synthetic Dataset Partitioning</h3>
+                <h3 id="sec5-2">5.2 Calibrated Synthetic Dataset Partitioning & Dual-Generator Architecture</h3>
                 <p>
-                    The synthetic dataset generator (<code>scripts/generate_data.py</code>) partitions the target 150,000 textline synthesis corpus into <strong>15% evaluation/locked splits</strong> (22,500 lines: <code>val_clean_short</code> 3%, <code>val_clean_long</code> 3%, <code>val_noisy_short</code> 3%, <code>val_noisy_long</code> 3%, <code>locked_test</code> 3%) and <strong>85% training split</strong> (127,500 lines) structured across 6 calibrated sub-categories:
+                    To support both baseline reproducibility and ongoing experimental modeling, the repository provides two distinct synthetic dataset generation pipelines:
+                </p>
+
+                <h4 style="margin-top: 1.25rem; font-size: 1.05rem; color: var(--text-primary);">Pipeline A: Baseline General Generator (<code>scripts/generate_data.py</code>)</h4>
+                <p>
+                    The baseline generator produces a 150,000 textline corpus partitioned into <strong>15% evaluation/locked splits</strong> (22,500 lines: <code>val_clean_short</code> 3%, <code>val_clean_long</code> 3%, <code>val_noisy_short</code> 3%, <code>val_noisy_long</code> 3%, <code>locked_test</code> 3%) and an <strong>85% training split</strong> (127,500 lines) structured across 6 calibrated sub-categories:
                 </p>
                 <div class="table-responsive">
                     <table>
@@ -1152,6 +1157,18 @@
                     </table>
                 </div>
 
+                <h4 style="margin-top: 1.5rem; font-size: 1.05rem; color: var(--text-primary);">Pipeline B: Experimental V25 Generator (<code>scripts/generate_data_v25.py</code>)</h4>
+                <p>
+                    For the V25 experimental iteration, a dedicated multi-threaded generator (<code>generate_data_v25.py</code>) synthesizes 150,000 textline images (140,000 train + 10,000 validation) directly synchronized with <code>configs/v25_dataset_manifest.json</code> across 5 core pillars:
+                </p>
+                <ol style="margin-bottom: 1.5rem; line-height: 1.7; color: var(--text-secondary);">
+                    <li><strong>Canonical Cham Literature (43.3% / 65,000 lines)</strong>: Classical literary vocabulary (<em>Po Klong Garai</em>, 57-stanza verse).</li>
+                    <li><strong>Anti-Motion Blur Base (20.0% / 30,000 lines)</strong>: Clean base images paired with dynamic in-RAM directional motion blur (\(7\times 7\) to \(13\times 13\)) and defocus.</li>
+                    <li><strong>Inline Bilingual Code-Switching (16.7% / 25,000 lines)</strong>: Intra-line mixed phrases with Vietnamese diacritics and Latin annotations.</li>
+                    <li><strong>Stanza Numerals & Boundary Punctuation (12.0% / 18,000 lines)</strong>: Verse numbers \(1\)–\(99\) (<span class="cham-script">꩑꩞</span>..<span class="cham-script">꩙꩙꩞</span>) and boundary marks (<span class="cham-script">꩞</span>, <code>:</code>, <code>–</code>).</li>
+                    <li><strong>Adversarial Minimal Pairs (8.0% / 12,000 lines)</strong>: Micro-stroke pairs (<span class="cham-script">ꨲ</span> U+AA32 Vowel Sign UE vs <span class="cham-script">ꨶ</span> U+AA36 Medial WA, variable Double Danda <span class="cham-script">꩝꩝</span> spacing 2–8px, 3-tier diacritic stacks).</li>
+                </ol>
+
                 <h3 id="sec5-3">5.3 Digital Typefaces & TrueType cmap Tofu Glyph Safeguards</h3>
                 <p>
                     When rendering synthetic text, digital Cham typefaces must be verified for glyph coverage. The primary bundled and reproducible typeface distributed directly within this repository is <strong>Noto Sans Cham</strong> (<code>Regular</code>, <code>Bold</code>, <code>Black</code>; SIL Open Font License 1.1, located in <code>ocr-training/data/fonts/</code>). External reference typefaces evaluated during research—including <em>Cham Roman</em>, <em>EFEO Cham</em>, and community fonts—are not redistributed in this repository and must be acquired independently if researchers wish to reproduce specific historical font variations.
@@ -1554,9 +1571,9 @@ if not hasattr(np, 'typeDict'): np.typeDict = {}</code></pre>
                     While our pipeline demonstrates substantial gains on synthetic and semi-controlled documents, rigorous academic honesty requires acknowledging the following boundaries:
                 </p>
                 <ul>
-                    <li><strong>Synthetic-to-Real Domain Gap</strong>: The recognizer is trained predominantly on synthetic textlines rendered with available digital TrueType fonts (<em>Noto Sans Cham</em>, <em>Cham Roman</em>, <em>EFEO Cham</em>). Historical manuscripts feature idiosyncratic scribal hands, variable ink viscosity, and non-standard ligature variations that may exhibit lower recognition confidence in the field.</li>
+                    <li><strong>Synthetic-to-Real Domain Gap</strong>: The neural recognizer is trained predominantly on synthetic textlines rendered with bundled <em>Noto Sans Cham</em> (SIL OFL 1.1). Historical manuscripts feature idiosyncratic scribal hands, variable ink viscosity, and non-standard ligature variations that may exhibit lower recognition confidence in the field.</li>
                     <li><strong>Absence of Large-Scale Real Manuscript Ground Truth</strong>: Due to the acute scarcity of digitized historical Cham archives with line-level annotations, current benchmarks rely on controlled synthetic stress-tests and curated test slices. Establishing an open, expert-verified historical palm-leaf manuscript benchmark represents an urgent necessity requiring future collaborative scholarship with native Cham elders and linguists.</li>
-                    <li><strong>Severe Biological Degradation & Epigraphy</strong>: Inscription stone rubbings and severely mold-damaged palm leaves (where over 40% of character ink has flaked away) remain beyond the capabilities of pure vision-based CTC sequence models. Integrating masked language models (MLMs) trained on historical Cham corpora is an active subject of future study.</li>
+                    <li><strong>Severe Biological Degradation & Epigraphy</strong>: Inscription stone rubbings and severely mold-damaged palm leaves (where character ink has substantially eroded) remain beyond the capabilities of pure vision-based CTC sequence models. Integrating masked language models (MLMs) trained on historical Cham corpora is an active subject of future study.</li>
                     <li><strong>Extreme Interlinear Overlap (Level 5)</strong>: When severe paper wrinkling causes interlinear wave amplitude to exceed line spacing (\(\text{Amplitude} > \text{Gap}\)), 1D projection and horizontal bounding boxes collapse, indicating the need for 2D polygonal baseline tracking models in future iterations.</li>
                 </ul>
             </section>
@@ -1574,13 +1591,13 @@ if not hasattr(np, 'typeDict'): np.typeDict = {}</code></pre>
                         <span>BibTeX Citation</span>
                         <button class="copy-btn" onclick="navigator.clipboard.writeText(document.getElementById('bibtex-code').innerText)">Copy BibTeX</button>
                     </div>
-<pre id="bibtex-code"><code>@software{cham_ocr_studio,
-  author = {Nguyen, Phuc},
-  title = {Cham OCR Studio: Deep Learning Pipeline and Paleographic Transcription Workbench for Historical Cham Manuscripts},
+<pre id="bibtex-code"><code>@software{cham_ocr_pipeline,
+  author = {Nguyen, Phuc H.},
+  title = {Cham-OCR: A Deep-Learning OCR Pipeline for Unicode Eastern Cham, toward Historical Manuscript and Epigraphic Recognition},
   year = {2026},
   publisher = {GitHub},
   url = {https://github.com/phucsd/Cham-OCR},
-  note = {Official web service: https://ocr.cham.asia}
+  note = {Production web service: https://ocr.cham.asia}
 }</code></pre>
                 </div>
 

ocr-training/TRAINING_ROADMAP_V25.md

@@ -12,14 +12,14 @@ flowchart TD
         E1["Mờ rung tay (Motion Blur)<br/>CER vọt 38.74% (Gấp 2.7 lần)<br/>Sót 8.3% dòng chữ"]
         E2["Song ngữ nội dòng (Inline Mixed)<br/>CER vọt 27.84%<br/>Mô hình Chăm mù chữ Việt, Việt mù Chăm"]
         E3["Mất dấu ngắt câu ꩞, :, –<br/>Gây nối nhầm dòng đoạn văn<br/>Paragraph Flow F1 tụt còn 70.7%"]
-        E4["Nhầm cặp dấu phụ chân ꨲ (Au) vs ꨶ (O)<br/>Khoảng cách Double Danda ꩝꩝"]
+        E4["Nhầm cặp dấu phụ chân ꨲ (Vowel Sign UE) vs ꨶ (Medial WA)<br/>Khoảng cách Double Danda ꩝꩝"]
     end
 
     subgraph Solutions_V25["4 Trụ Cột Huấn Luyện Cho Mô Hình V25"]
         P1["Trụ cột 1: Data Augmentation Mờ Động Cực Hạn (On-the-Fly)<br/>(Directional Motion Blur 7x7 đến 13x13, Defocus trong RAM)"]
         P2["Trụ cột 2: Từ Điển Hợp Nhất Chăm - Việt Động Chuẩn Hóa NFC<br/>(build_dict_v25.py giữ thứ tự V24 + Preflight checks + Weight Surgery)"]
         P3["Trụ cột 3: Gói Dữ Liệu Đặc Trị Số Khổ & Ngắt Câu Biên<br/>(Dãy số 1-99 ꩑꩞..꩙꩙꩞, bảo tồn ꩞, :, –, padding biên)"]
-        P4["Trụ cột 4: Cặp Đối Kháng Hard-Examples Minimal Pairs<br/>(Phân biệt ꨲ vs ꨶ, bảo tồn ꨯ, ꨯꨱ, Double Danda ꩝꩝ 2-8px)"]
+        P4["Trụ cột 4: Cặp Đối Kháng Hard-Examples Minimal Pairs<br/>(Phân biệt ꨲ (Vowel Sign UE) vs ꨶ (Medial WA), bảo tồn ꨯ (Vowel Sign E), ꨯꨱ, Double Danda ꩝꩝ 2-8px)"]
     end
 
     E1 ==> P1
@@ -67,10 +67,10 @@ flowchart TD
 
 ### Trụ Cột 4: Cặp Đối Kháng Hard-Examples Cho Dấu Phụ Dễ Nhầm
 - **Thực trạng**:
-  - Dấu `ꨲ` (Au, U+AA32) và `ꨶ` (O, U+AA36) có cấu trúc vi mô rất tương đồng.
+  - Dấu `ꨲ` (Vowel Sign UE, U+AA32) và `ꨶ` (Medial WA, U+AA36) có cấu trúc vi mô rất tương đồng.
   - Dấu Double Danda `꩝꩝` hay bị gộp thành Single Danda `꩝`.
-  - Nguyên âm trước `ꨯ` (E) và `ꨯꨱ` (Au) dễ bị nuốt khi nét vẽ thanh mảnh.
-- **Giải pháp cho v25 (Gói 4: 20,000 dòng)**:
+  - Nguyên âm phụ thuộc `ꨯ` (Vowel Sign E, U+AA2F) và `ꨯꨱ` (Vowel Sign AU, U+AA2F U+AA31) dễ bị nuốt khi nét vẽ thanh mảnh.
+- **Giải pháp cho v25 (Gói 5: 12,000 dòng)**:
   - Sinh **mẫu cặp từ tối thiểu (Minimal Pairs)**:
     - `{Phụ âm} + ꨲ` đối sánh trực tiếp với `{Phụ âm} + ꨶ` (ví dụ `ꨀꨲꩆ` vs `ꨀꨶꩆ`, `ꨓꨆꨴꨲꨩ` vs `ꨓꨆꨴꨶꨩ`, `ꨚꨲ` vs `ꨚꨶ`).
     - Dấu Double Danda `꩝꩝` với khoảng cách thay đổi từ 2px đến 8px.
@@ -82,14 +82,15 @@ flowchart TD
 
 Theo tệp đặc tả chuẩn `configs/v25_dataset_manifest.json`:
 
-| Gói Dữ Liệu (Bucket) | Số Lượng Dòng (Train) | Tỷ Lệ | Mục Tiêu Kỹ Thuật | Đặc Điểm Augmentation |
+| Gói Dữ Liệu (Pillar / Package) | Số Lượng Dòng (Train) | Tỷ Lệ | Mục Tiêu Kỹ Thuật | Đặc Điểm Augmentation |
 | :--- | :---: | :---: | :--- | :--- |
-| **Gói 1: Ngữ liệu Chăm chuẩn & Cổ tích** | 70,000 | 50.0% | Nền tảng từ vựng Chăm chuẩn xác (Po Klong Garai, sử thi, thơ 57 khổ) | Nền giấy cổ, texture, tiêu chuẩn sạch |
-| **Gói 2: Song ngữ nội dòng (Chăm + Việt)** | 25,000 | 17.9% | Nhận diện mượt mà Chăm kẹp Việt/Latin | Font kết hợp NotoSansCham + NotoSans |
-| **Gói 3: Số thứ tự khổ thơ & Dấu câu biên** | 25,000 | 17.9% | Nhận diện số 1-99, dấu `꩞`, `:`, `–` | Padding biên biến thiên 0-12px |
-| **Gói 4: Cặp đối kháng Hard-Examples** | 20,000 | 14.3% | Khắc phục triệt để `ꨲ` vs `ꨶ`, `꩝꩝` vs `꩝` | Minimal pairs, zoom nét chân |
+| **Gói 1: Ngữ liệu Chăm chuẩn & Văn học kinh điển** | 65,000 | 43.3% | Nền tảng từ vựng Chăm chuẩn xác (Po Klong Garai, sử thi, thơ 57 khổ) | Nền giấy cổ, texture, tiêu chuẩn sạch |
+| **Gói 2: Ngữ liệu đặc trị Mờ Rung Tay (Anti-Blur Base)** | 30,000 | 20.0% | Tăng cường độ thích nghi mờ rung tay | Ảnh cơ sở sinh sạch, làm mờ on-the-fly trong RAM |
+| **Gói 3: Song ngữ nội dòng (Chăm + Việt kẹp dòng)** | 25,000 | 16.7% | Nhận diện mượt mà Chăm kẹp Việt/Latin | Font kết hợp NotoSansCham + NotoSans |
+| **Gói 4: Số thứ tự khổ thơ & Dấu câu biên** | 18,000 | 12.0% | Nhận diện số 1-99, dấu `꩞`, `:`, `–` | Padding biên biến thiên 0-12px |
+| **Gói 5: Cặp đối kháng Hard-Examples Minimal Pairs** | 12,000 | 8.0% | Khắc phục triệt để `ꨲ` (UE) vs `ꨶ` (WA), `꩝꩝` vs `꩝` | Minimal pairs, zoom nét chân, tổ hợp 3 tầng |
 | **TỔNG CỘNG TẬP TRAIN v25** | **140,000** | **100%** | **Toàn diện mọi điều kiện thực tế** | **Ảnh gốc sinh sạch trên đĩa** |
-| **TẬP KIỂM THỬ (VALIDATION)** | **10,000** | — | **Đánh giá khách quan, cân đối 4 nhóm** | **Cố định hạt giống ngẫu nhiên** |
+| **TẬP KIỂM THỬ (VALIDATION)** | **10,000** | — | **Đánh giá khách quan, cân đối 5 nhóm** | **Cố định hạt giống ngẫu nhiên** |
 | **TỔNG QUY MÔ DATASET** | **150,000** | — | **Đồng bộ duy nhất với Manifest** | **On-the-fly Blur: 35% qua RecAug** |
 
 > [!IMPORTANT]
@@ -175,9 +176,9 @@ flowchart LR
 ### Bước 4: Khởi Động Huấn Luyện 3 Chặng Trên Kaggle GPU T4x2
 - Sử dụng cấu hình [rec_cham_v25.yml](file:///e:/Phuc's%20Data/Github/Cham-OCR/ocr-training/configs/rec_cham_v25.yml).
 - Tự động nạp thông tin xác thực bảo mật qua [kaggle_auth.py](file:///e:/Phuc's%20Data/Github/Cham-OCR/ocr-training/scripts/kaggle_auth.py) và `.env`.
-- **Chặng 1** (Epoch 1 -> 14, ~7.8h): Khởi động với initial weights từ Bước 3. Xuất checkpoint `latest`.
-- **Chặng 2** (Epoch 15 -> 27, ~7.3h): Resume thông qua `Global.checkpoints` trỏ vào full checkpoint Chặng 1 (`.pdparams + .pdopt + .states`).
-- **Chặng 3** (Epoch 28 -> 40, ~7.3h): Resume từ full checkpoint Chặng 2, hoàn thành toàn bộ 40 epochs.
+- **Chặng 1** (Epoch 1 -> 12, ~7.5h): Khởi động với initial weights từ Bước 3. Ngắt chặng tại epoch 12 (`stage_end_epoch: 12`), xuất checkpoint `latest` & `best_accuracy`.
+- **Chặng 2** (Epoch 13 -> 22, ~6.5h): Resume thông qua `Global.checkpoints` trỏ vào full checkpoint Chặng 1 (`.pdparams + .pdopt + .states`). Ngắt chặng tại epoch 22 (`stage_end_epoch: 22`).
+- **Chặng 3** (Epoch 23 -> 40, ~11.0h): Resume từ full checkpoint Chặng 2, hoàn thành toàn bộ 40 epochs, xuất model `inference` v25.
 
 ### Bước 5: Đóng Gói Inference & Kiểm Thử Nghiệm Thu (Non-Regression Gate)
 - Xuất model inference:

ocr-training/configs/v25_dataset_manifest.json

@@ -0,0 +1,108 @@
+{
+  "$schema": "http://json-schema.org/draft-07/schema#",
+  "name": "Cham-OCR V25 Unified Dataset Manifest",
+  "version": "v25.0",
+  "created_at": "2026-09-11",
+  "description": "Source of truth thống nhất cho toàn bộ quy mô, phân bổ dữ liệu và chính sách huấn luyện V25",
+  "data_distribution": {
+    "total_samples": 150000,
+    "train_samples": 140000,
+    "val_samples": 10000,
+    "train_val_ratio": "93.3% / 6.7%",
+    "packages": [
+      {
+        "id": "pkg_01_cham_canonical",
+        "name": "Ngữ liệu Chăm chuẩn & Văn học kinh điển",
+        "count": 65000,
+        "percentage": "43.3%",
+        "description": "Nền tảng từ vựng Chăm chuẩn xác từ sử thi Po Klong Garai, thơ 57 khổ, truyền thuyết và truyện cổ tích."
+      },
+      {
+        "id": "pkg_02_anti_blur_base",
+        "name": "Ngữ liệu đặc trị Mờ Rung Tay (Anti-Blur Base)",
+        "count": 30000,
+        "percentage": "20.0%",
+        "description": "Ảnh cơ sở được sinh sạch, áp dụng Directional Motion Blur (7x7-13x13) và Defocus on-the-fly trong DataLoader."
+      },
+      {
+        "id": "pkg_03_inline_mixed_bilingual",
+        "name": "Song ngữ Chăm - Việt kẹp nội dòng (Inline Mixed)",
+        "count": 25000,
+        "percentage": "16.7%",
+        "description": "Các cấu trúc Chăm kèm chú thích dịch nghĩa tiếng Việt có dấu, ngày tháng năm và ký hiệu."
+      },
+      {
+        "id": "pkg_04_stanza_and_punctuation",
+        "name": "Số thứ tự khổ thơ Chăm 1-99 & Dấu ngắt câu biên",
+        "count": 18000,
+        "percentage": "12.0%",
+        "description": "Dãy số khổ thơ Chăm 1-99 (꩑꩞..꩙꩙꩞), bảo tồn dấu ngắt ꩞, dấu hai chấm :, gạch ngang – và padding biên."
+      },
+      {
+        "id": "pkg_05_minimal_pairs_hard",
+        "name": "Cặp đối kháng Hard-Examples Minimal Pairs",
+        "count": 12000,
+        "percentage": "8.0%",
+        "description": "Tập trung phân biệt cặp dấu chân ꨲ (Vowel Sign UE, U+AA32) vs ꨶ (Medial WA, U+AA36), khoảng cách Double Danda ꩝꩝ (2-8px), tổ hợp dấu 3 tầng."
+      }
+    ]
+  },
+  "training_policy": {
+    "image_shape": [3, 48, 480],
+    "image_shape_rationale": "Chiều cao 48px bảo vệ dấu phụ mỏng; chiều rộng 480px chống co bẹp nét chữ trên dòng song ngữ dài.",
+    "batch_size_per_card": 32,
+    "global_batch_size": 64,
+    "num_workers": 2,
+    "amp_enabled": true,
+    "amp_level": "O1",
+    "blur_augmentation": "on_the_fly",
+    "blur_rationale": "Tuyệt đối không sinh ảnh mờ tĩnh lưu đĩa để tránh lỗi double-blur; biến dạng mờ được sinh động ngẫu nhiên trên RAM.",
+    "max_text_length_policy": "P99_dynamic_scan",
+    "max_text_length_rationale": "Quét toàn bộ nhãn tập train/val và đặt max_text_length = P99 để không bị cắt cụt nhãn song ngữ.",
+    "kaggle_execution_plan": {
+      "timeout_limit_hours": 12.0,
+      "stages": [
+        {
+          "stage": 1,
+          "epochs": "1 -> 12",
+          "est_hours": 7.5,
+          "checkpoint_save": "save_epoch_step: 1",
+          "output": "ch_PP-OCRv4_rec_v25_stage1/latest"
+        },
+        {
+          "stage": 2,
+          "epochs": "13 -> 22",
+          "est_hours": 6.5,
+          "resume_from": "stage 1 latest (pdparams + pdopt + states)",
+          "output": "ch_PP-OCRv4_rec_v25_stage2/latest"
+        },
+        {
+          "stage": 3,
+          "epochs": "23 -> 40",
+          "est_hours": 11.0,
+          "resume_from": "stage 2 latest (pdparams + pdopt + states)",
+          "output": "ch_PP-OCRv4_rec_v25_final/latest"
+        }
+      ]
+    }
+  },
+  "acceptance_gates": {
+    "non_regression_clean_a4": {
+      "description": "Đảm bảo độ chính xác trên văn bản Chăm in chuẩn A4 không bị suy giảm khi tối ưu các miền dữ liệu mới.",
+      "metric": "CER",
+      "max_allowed_delta": "+0.50%"
+    },
+    "anti_blur_target": {
+      "metric": "CER on motion blur tests",
+      "target": "< 18.0%"
+    },
+    "mixed_bilingual_target": {
+      "metric": "CER on inline mixed Cham-Viet",
+      "target": "< 12.0%"
+    },
+    "paragraph_flow_f1": {
+      "metric": "Paragraph reconstruction F1",
+      "target": "> 88.0%"
+    }
+  }
+}

ocr-training/scripts/generate_data_v25.py

@@ -0,0 +1,437 @@
+#!/usr/bin/env python3
+# -*- coding: utf-8 -*-
+"""
+Master Synthetic Dataset Generator for Cham OCR Model V25.
+Optimized for 32-core Data Prep CPU Machine on Lightning AI Studios.
+
+Produces 140,000 Train + 10,000 Val (150,000 total) line images across 5 core pillars:
+1. Standard & Authentic Cham Literature (Po Klong Garai, 57-stanza poem, administration)
+2. Extreme Anti-Motion Blur & Defocus Augmentation (Kernels 7x7 to 13x13, downsampling)
+3. Intra-line Bilingual Code-Switching (Cham phrase + Vietnamese diacritics + Latin)
+4. Full Stanza Digits (1-99: ꩑꩞..꩙꩙꩞) & Boundary Punctuation (꩞, :, –, ?, !)
+5. Minimal Pairs Hard-Examples (Vowel Sign UE ꨲ U+AA32 vs Medial WA ꨶ U+AA36, ꨯꨮꨩ vs ꨯꨱ, ꩝꩝ variable gap 2-8px)
+"""
+
+import os
+import sys
+import math
+import random
+import time
+import json
+import unicodedata
+import argparse
+import cv2
+import numpy as np
+from multiprocessing import Pool, cpu_count
+from PIL import Image, ImageDraw, ImageFont, ImageFilter, ImageEnhance
+
+# UTF-8 stdout encoding
+if sys.stdout and hasattr(sys.stdout, 'reconfigure'):
+    sys.stdout.reconfigure(encoding='utf-8')
+if sys.stderr and hasattr(sys.stderr, 'reconfigure'):
+    sys.stderr.reconfigure(encoding='utf-8')
+
+# NumPy 2.x compatibility monkeypatch
+if not hasattr(np, 'sctypes'):
+    np.sctypes = {
+        'int': [np.int8, np.int16, np.int32, np.int64],
+        'uint': [np.uint8, np.uint16, np.uint32, np.uint64],
+        'float': [np.float16, np.float32, np.float64],
+        'complex': [np.complex64, np.complex128],
+        'others': [bool, object, bytes, str]
+    }
+if not hasattr(np, 'bool'): np.bool = bool
+if not hasattr(np, 'int'): np.int = int
+if not hasattr(np, 'float'): np.float = float
+if not hasattr(np, 'typeDict'): np.typeDict = {}
+
+SCRIPT_DIR = os.path.dirname(os.path.abspath(__file__))
+TRAINING_DIR = os.path.dirname(SCRIPT_DIR)
+PROJECT_ROOT = os.path.dirname(TRAINING_DIR)
+
+# Fonts
+FONT_CHAM_REG = os.path.join(TRAINING_DIR, "data", "fonts", "NotoSansCham-Regular.ttf")
+FONT_CHAM_BOLD = os.path.join(TRAINING_DIR, "data", "fonts", "NotoSansCham-Bold.ttf")
+
+# ==============================================================================
+# 1. Ký Tự & Ngữ Liệu Tiếng Chăm & Song Ngữ V25
+# ==============================================================================
+CHAM_CONSONANTS = list("ꨆꨇꨈꨉꨊꨋꨌꨍꨎꨏꨐꨑꨒꨓꨔꨕꨖꨗꨘꨙꨚꨛꨜꨝꨞꨟꨠꨡꨢꨣꨤꨥꨦꨧꨨꨀꨁꨂꨃꨄꨅ")
+CHAM_DIGITS_LIST = list("꩐꩑꩒꩓꩔꩕꩖꩗꩘꩙")
+
+def to_cham_num(n):
+    """Chuyển số nguyên n thành chuỗi chữ số Chăm"""
+    return "".join(CHAM_DIGITS_LIST[int(d)] for d in str(n))
+
+# Ngữ liệu từ bài thơ 57 khổ và truyền thuyết Po Klong Garai
+CHAM_VOCAB_POOL = [
+    "ꨛꨯꨮ", "ꨆꨵꨯꨱꩃ", "ꨈꨣꩈ", "ꨧꨯꨱꩃ", "ꨚꩆ", "ꨆꩀ", "ꨨꨤꨩ", "ꨟꨐꨪꩌ", "ꨕꨶꨩ", "ꨝꨯꨱꩍ",
+    "ꨝꨰ", "ꨙꨮ", "ꨎꨤꨭꩀ", "ꨟꨐꨭꩌ", "ꨀꨳꨩ", "ꨝꨵꨯꨱꩍ", "ꨝꨩ", "ꨧꨩ", "ꨝꨵꩍ", "ꨀꨇꩆ",
+    "ꨗꨯꨱ", "ꨝꨣꨭꨥ", "ꨟꩃ", "ꨓꨮꩊ", "ꨊꨯꨱꩀ", "ꨣꨈꨵꨰ", "ꨚꨢꨯꨱꩌ", "ꨘꩆ", "ꨤꩄ", "ꨆꨳꨮꩃ",
+    "ꨧꨤꨪꩍ", "ꨕꨨꨵꩀ", "ꨨꨕꨯꩌ", "ꨀꨣꩀ", "ꨗꨫ", "ꨢꨭꩅ", "ꨚꨰꩀ", "ꨝꨪ", "ꨝꩀ", "ꨆꨮꨭ",
+    "ꨝꨴꨬ", "ꨆꨩ", "ꨨꨲꨩ", "ꨡꨯꩍ", "ꨀꨦꨪꩅ", "ꨟꩀ", "ꨟꨰ", "ꨝꨭꩍ", "ꨕꨫ", "ꨕꨤꩌ",
+    "ꨅꨩ", "ꨨꨕꨳꨮꩇ", "ꨀꨗꩀ", "ꨓꨌꨯꨱꨥ", "ꨙꩃ", "ꨀꨝꨪꩍ", "ꨤꨝꨭꩀ", "ꨎꨥ", "ꨥꨮꩀ", "ꨢꨮ",
+    "ꨐꨭꨩ", "ꨓꨴꨩ", "ꨚꨤꨬ", "ꨂꩀ", "ꨀꨗꨯꨱꩃ", "ꨧꨝꨭꩃ", "ꨈꨮꩇ", "ꨀꨧꨪꩅ", "ꨢꨮꨭ", "ꨕꨘꩀ",
+    "ꨎꨮ", "ꨚꨶꨮꩄ", "ꨚꨆꨴꨲꨩ", "ꨟꨪꩆ", "ꨙꨩ", "ꨝꨳꩀ", "ꨚꨗꨶꨮꩄ", "ꨔꨮꨭ", "ꨕꨯꨱꩀ", "ꨓꨳꩆ",
+    "ꨀꨶꨮꩆ", "ꨓꨝꨶꨮꩆ", "ꨚꨎꨳꩀ", "ꨆꨵꨮꨭ", "ꨀꨮꩈ", "ꨧꨝꩀ", "ꨈꨗꨴꨮꩍ", "ꨚꩀ", "ꨨꨚꨭꩍ", "ꨤꨪꨎꩃ",
+    "ꨝꨵꨬ", "ꨣꨬ", "ꨈꨴꨮꩇ", "ꨧꩃ", "ꨠꨎꨰꩀ", "ꨚꨙꨳꩀ", "ꨤꨯꨩ", "ꨤꨮꩍ", "ꨚꨕꨬ", "ꨈꨵꨰꩍ",
+    "ꨀꨤꨩ", "ꨨꨤꨶꨬ", "ꨆꨭꨢꨮꨭ", "ꨤꨪꨔꨬ", "ꨨꨤꨬ", "ꨇꨪꩀ", "ꨙꨪꩍ", "ꨥꩉ", "ꨕꨴꨬ", "ꨁꨗꨩ",
+    "ꨈꨭꨣꩈ", "ꨤꨳꩍ", "ꨣꨭꩇ", "ꨤꨶꩀ", "ꨓꨟꨩ", "ꨓꨗꩍ", "ꨣꨪꨢꨩ", "ꨖꨪꩅ", "ꨀꨧꩉ", "ꨚꨓꨯꨱ",
+    "ꨇꨪꩆ", "ꨨꨶꩀ", "ꨃꨥ", "ꨧꨣꨥꨩ", "ꨚꨙꩉ", "ꨢꩍ", "ꨄꨰ", "ꨆꨯꨱꩍ", "ꨚꨓꨬ", "ꨓꨭꩍ",
+    "ꨓꨌꨬ", "ꨌꨪꩍ", "ꨈꩆ", "ꨆꨴꨲꩍ", "ꨓꨟꨯꨱꩃ", "ꨨꨭꨩ", "ꨗꩌ", "ꨈꩍ", "ꨆꨕꨯꨱꩍ", "ꨕꨨꨵꨮꨭ",
+    "ꨌꨙꨳꩀ", "ꨓꨆꨶꨯꩈ", "ꨀꨥꨰꩅ", "ꨗꨆꨓꨯꨮꨩ", "ꨀꨆꨢꨯꨮꩅ", "ꨓꨆꨴꨲꨩ", "ꨚꨟꨆꨴꨲꨩ", "ꨣꨳꨪꩌ", "ꨨꨈꨴꨮꩌ"
+]
+
+# Minimal pairs cho Vowel Sign UE ꨲ (U+AA32) vs Consonant Sign Medial WA ꨶ (U+AA36)
+MINIMAL_PAIRS_UE_WA = [
+    ("ꨀꨲꩆ", "ꨀꨶꩆ"),
+    ("ꨓꨆꨴꨲꨩ", "ꨓꨆꨴꨶꨩ"),
+    ("ꨚꨴꨲꨩ", "ꨚꨴꨶꨩ"),
+    ("ꨆꨲꩆ", "ꨆꨶꩆ"),
+    ("ꨟꨲꩆ", "ꨟꨶꩆ"),
+    ("ꨣꨲꨩ", "ꨣꨶꨩ"),
+    ("ꨚꨟꨆꨴꨲꨩ", "ꨚꨟꨆꨴꨶꨩ"),
+    ("ꨚꨲ", "ꨚꨶ"),
+    ("ꨆꨲ", "ꨆꨶ"),
+    ("ꨀꨲ", "ꨀꨶ")
+]
+
+# Cụm dấu 3 tầng & tổ hợp nguyên âm phức tạp
+COMPLEX_CLUSTERS = [
+    "ꨣꨳꨪꩌ", "ꨓꨳꨪꩌ", "ꨚꨳꨪꩌ", "ꨕꨨꨵꩀ", "ꨝꨪꨗꨳꨪ", 
+    "ꨨꨕꨳꨮꩇ", "ꨞꨯꨚꨓꨪꩍ", "ꨧꨪꩆꨝꨳꨪ", "ꨆꨵꨯꨱꩃ", "ꨚꨗꨴꨯꨱꩃ",
+    "ꨗꨆꨓꨯꨮꨩ", "ꨚꨓꨯꨮꨩ", "ꨤꨯꨮꨩ", "ꨨꨝꨳꨯꨮꩆ", "ꨨꨈꨴꨮꩌ", "ꨣꨪꨈꨮꩌ"
+]
+
+# Mẫu song ngữ nội dòng (Cham phrase + Viet translation/commentary)
+BILINGUAL_TEMPLATES = [
+    ("ꨛꨯꨮ ꨆꨵꨯꨱꩃ ꨈꨣꩈ", "Vua Pô Klông Gia-rai"),
+    ("ꨛꨯꨮ ꨚꩆ", "Pô Păt - Po Pat"),
+    ("ꨚꨰꩀ ꨨꨤꨩ", "đắp bờ đập ngăn nước"),
+    ("ꨕꨶꨩ ꨝꨯꨱꩍ ꨝꨰ", "dẫn nguồn nước vào ruộng"),
+    ("ꨞꩇ ꨂꨣꩃ ꨌꩌ", "xứ sở người Chăm Panduranga"),
+    ("yutꨢꨮ", "bờ đập nước sông lớn"),
+    ("ꨁꨗꨩ ꨈꨭꨣꩈ", "Bà Thầy truyền dạy giáo lý"),
+    ("ꨤꨪꨔꨬ ꨓꨝꨳꩀ", "cơm nắm gói trong lá chuối"),
+    ("ꨝꨣꨭꨥ ꨟꩃ ꨣꨈꨵꨰ", "sau cuộc thi tài kết thúc"),
+    ("ꨕꨨꨵꨀ ꨨꨕꨯꩌ", "nước chảy quanh năm trên đồng"),
+    ("ꨎꨤꨭꩀ ꨟꨐꨭꩌ", "mở rộng hệ thống mương máng"),
+    ("ꨣꨈꨵꨰ ꨚꨤꨬ ꨂꩀ", "thi tài đắp đập ngăn sông"),
+    ("ꨓꩀ ꨕꨫ ꨚꨰꩀ ꨨꨤꨩ", "đến lúc đắp bờ đập nước"),
+    ("ꨝꨰ ꨘꩆ ꨀꨦꨪꩅ", "ruộng lúa tốt tươi màu mỡ"),
+    ("ꨣꨈꨵꨰ ꨧꨝꨭꩃ", "thi tài đắp đập đá lớn")
+]
+
+# ==============================================================================
+# 2. Bộ Sinh Nội Dung Văn Bản (V25 Text Sampler)
+# ==============================================================================
+def sample_v25_text(pillar_type):
+    """
+    Sinh dòng văn bản theo từng trụ cột kỹ thuật:
+    - pillar 1: Cham chuẩn (chuẩn bị cho Anti-Motion Blur)
+    - pillar 2: Song ngữ nội dòng (Cham + Viet + Latin)
+    - pillar 3: Số khổ thơ 1-99 & Dấu ngắt câu (꩑꩞..꩔꩓꩞, :, –, ?)
+    - pillar 4: Cặp đối kháng Minimal Pairs (ꨲ vs ꨶ, ꩝꩝ variable gap, ꨯꨮꨩ)
+    """
+    if pillar_type == "bilingual":
+        # Trụ cột 2: Song ngữ nội dòng
+        pair = random.choice(BILINGUAL_TEMPLATES)
+        style = random.choice(["bracket", "colon", "hyphen", "en_dash"])
+        if style == "bracket":
+            return f"{pair[0]} ({pair[1]})"
+        elif style == "colon":
+            return f"{pair[0]}: {pair[1]}"
+        elif style == "en_dash":
+            return f"{pair[0]} – {pair[1]}"
+        else:
+            return f"{pair[0]} - {pair[1]}"
+
+    elif pillar_type == "stanza_punct":
+        # Trụ cột 3: Số khổ 1-99 & Dấu ngắt câu
+        num_val = random.randint(1, 99)
+        cham_num_str = to_cham_num(num_val)
+        sep = random.choice(["꩞ ", "꩝ "])
+        words = random.sample(CHAM_VOCAB_POOL, random.randint(2, 4))
+        content = " ".join(words)
+        end_mark = random.choice(["꩞", "꩝꩝", "꩝", ":", "?", "!"])
+        return f"{cham_num_str}{sep}{content}{end_mark}"
+
+    elif pillar_type == "minimal_pairs":
+        # Trụ cột 4: Cặp đối kháng
+        sub = random.choice(["ue_wa", "double_danda", "complex_clusters"])
+        if sub == "ue_wa":
+            pair = random.choice(MINIMAL_PAIRS_UE_WA)
+            target = pair[0] if random.random() < 0.5 else pair[1]
+            surrounding = random.sample(CHAM_VOCAB_POOL, random.randint(2, 3))
+            return f"{target} " + " ".join(surrounding) + random.choice(["꩝", "꩝꩝", "꩞", ""])
+        elif sub == "double_danda":
+            words = random.sample(CHAM_VOCAB_POOL, random.randint(3, 5))
+            return " ".join(words) + "꩝꩝"
+        else:
+            c = random.choice(COMPLEX_CLUSTERS)
+            words = random.sample(CHAM_VOCAB_POOL, random.randint(2, 3))
+            return f"{c} " + " ".join(words) + random.choice(["꩝", "꩞", ""])
+
+    else:
+        # Chuẩn (sẽ áp dụng Motion Blur nếu thuộc pillar anti_blur)
+        words = random.sample(CHAM_VOCAB_POOL, random.randint(3, 6))
+        text = " ".join(words)
+        if random.random() < 0.5:
+            text += random.choice(["꩝", "꩝꩝", "꩞", ""])
+        return text
+
+# ==============================================================================
+# 3. Kỹ Thuật Render Ảnh & Tinh Chỉnh Nét Chữ
+# ==============================================================================
+def render_line_v25(text, is_bold=False, font_size=26, pad_left=12, pad_right=12, danda_gap=3):
+    """
+    Render dòng chữ Chăm / Song ngữ.
+    Hỗ trợ biến thiên khoảng cách Double Danda '꩝꩝' từ 2px đến 8px.
+    """
+    font_path = FONT_CHAM_BOLD if is_bold else FONT_CHAM_REG
+    font = ImageFont.truetype(font_path, font_size)
+
+    # Nếu có ꩝꩝ ở cuối và cần biến thiên khoảng cách
+    if text.endswith("꩝꩝") and danda_gap != 3:
+        base_text = text[:-2]
+        bbox_base = font.getbbox(base_text) if base_text else (0, 0, 0, font_size)
+        w_base = (bbox_base[2] - bbox_base[0]) if base_text else 0
+        
+        bbox_danda = font.getbbox("꩝")
+        w_danda = bbox_danda[2] - bbox_danda[0]
+        h_line = max(bbox_base[3] - bbox_base[1], bbox_danda[3] - bbox_danda[1]) + 16
+        w_total = pad_left + w_base + (6 if base_text else 0) + w_danda + danda_gap + w_danda + pad_right
+
+        img = Image.new("RGB", (max(w_total, 60), max(h_line, 48)), (255, 255, 255))
+        draw = ImageDraw.Draw(img)
+
+        cur_x = pad_left
+        y = 8
+        if base_text:
+            draw.text((cur_x, y), base_text, fill=(20, 20, 25), font=font)
+            cur_x += w_base + 6
+        draw.text((cur_x, y), "꩝", fill=(20, 20, 25), font=font)
+        cur_x += w_danda + danda_gap
+        draw.text((cur_x, y), "꩝", fill=(20, 20, 25), font=font)
+        return img
+
+    # Render thông thường
+    bbox = font.getbbox(text)
+    tw = bbox[2] - bbox[0]
+    th = bbox[3] - bbox[1]
+    w = tw + pad_left + pad_right
+    h = max(th + 16, 48)
+
+    img = Image.new("RGB", (max(w, 60), h), (255, 255, 255))
+    draw = ImageDraw.Draw(img)
+    x = pad_left - bbox[0]
+    y = 8 - bbox[1]
+    draw.text((x, y), text, fill=(20, 20, 25), font=font)
+    return img
+
+# ==============================================================================
+# 4. Pipeline Biến Dạng Thực Địa (Augmentations) - Đặc Trị Motion Blur
+# ==============================================================================
+def apply_heavy_motion_blur(img_np, ksize=9, angle=45.0):
+    """Tạo ma trận directional motion blur kernel mô phỏng rung tay máy ảnh di động."""
+    kernel = np.zeros((ksize, ksize), dtype=np.float32)
+    rad = math.radians(angle)
+    dx = math.cos(rad)
+    dy = math.sin(rad)
+    cx, cy = ksize // 2, ksize // 2
+    for r in range(-ksize // 2, ksize // 2 + 1):
+        kx = int(round(cx + r * dx))
+        ky = int(round(cy + r * dy))
+        if 0 <= kx < ksize and 0 <= ky < ksize:
+            kernel[ky, kx] = 1.0
+    if kernel.sum() > 0:
+        kernel /= kernel.sum()
+    else:
+        kernel[cx, cy] = 1.0
+    return cv2.filter2D(img_np, -1, kernel)
+
+def augment_image_v25(img_pil, is_motion_blur_pillar=False, bake_blur=False):
+    """
+    Áp dụng các phép biến dạng hình ảnh thực tế:
+    - Nếu bake_blur=False (MẶC ĐỊNH CHO V25): Không nướng (bake) motion blur nặng vào ảnh đĩa.
+      Mọi thao tác làm mờ được thực hiện On-the-fly trong DataLoader qua RecAug để tránh double-blur.
+    - Giữ biến dạng nhẹ: nhiễu hạt giấy cổ, độ tương phản và chuẩn hóa chiều cao 48px.
+    """
+    img_bgr = cv2.cvtColor(np.array(img_pil), cv2.COLOR_RGB2BGR)
+    h, w = img_bgr.shape[:2]
+
+    if bake_blur and is_motion_blur_pillar:
+        # Chỉ áp dụng nếu người dùng cố ý bật bake_blur
+        ksize = random.choice([7, 9, 11, 13])
+        angle = random.uniform(0.0, 180.0)
+        img_bgr = apply_heavy_motion_blur(img_bgr, ksize=ksize, angle=angle)
+        if random.random() < 0.4:
+            scale = random.uniform(0.5, 0.75)
+            small = cv2.resize(img_bgr, (max(int(w * scale), 20), max(int(h * scale), 16)), interpolation=cv2.INTER_LINEAR)
+            img_bgr = cv2.resize(small, (w, h), interpolation=cv2.INTER_CUBIC)
+
+    # Thêm nhiễu nền giấy cổ nhẹ
+    if random.random() < 0.6:
+        noise = np.random.normal(0, random.uniform(1.0, 3.5), (h, w, 3)).astype(np.float32)
+        img_bgr = np.clip(img_bgr.astype(np.float32) + noise, 0, 255).astype(np.uint8)
+
+    # Thay đổi độ tương phản và độ sáng
+    if random.random() < 0.5:
+        alpha = random.uniform(0.85, 1.15)
+        beta = random.uniform(-10, 10)
+        img_bgr = np.clip(img_bgr.astype(np.float32) * alpha + beta, 0, 255).astype(np.uint8)
+
+    # Đảm bảo resize về chuẩn height 48px cho V25
+    target_h = 48
+    scale = target_h / float(h)
+    new_w = max(int(round(w * scale)), 32)
+    img_resized = cv2.resize(img_bgr, (new_w, target_h), interpolation=cv2.INTER_CUBIC)
+    return img_resized
+
+# ==============================================================================
+# 5. Hàm Worker Đa Tiến Trình (Multiprocessing Task)
+# ==============================================================================
+def generate_sample_worker(args):
+    """Worker tạo một mẫu ảnh và nhãn."""
+    sample_idx, pillar_type, out_img_path, bake_blur = args
+
+    # 1. Sinh văn bản và chuẩn hóa tuyệt đối Unicode NFC
+    raw_text = sample_v25_text(pillar_type)
+    text = unicodedata.normalize('NFC', raw_text)
+
+    # 2. Render ảnh
+    is_bold = (random.random() < 0.35)
+    pad_l = random.randint(2, 14)
+    pad_r = random.randint(2, 14)
+    danda_gap = random.randint(2, 8) if "꩝꩝" in text else 3
+    
+    img_pil = render_line_v25(text, is_bold=is_bold, pad_left=pad_l, pad_right=pad_r, danda_gap=danda_gap)
+
+    # 3. Biến dạng (Augmentation sạch cho lưu trữ đĩa)
+    is_blur_pillar = (pillar_type == "anti_blur")
+    img_final = augment_image_v25(img_pil, is_motion_blur_pillar=is_blur_pillar, bake_blur=bake_blur)
+
+    # 4. Ghi file ảnh
+    cv2.imwrite(out_img_path, img_final)
+
+    # 5. Trả về tên file tương đối và nhãn
+    rel_path = os.path.basename(os.path.dirname(out_img_path)) + "/" + os.path.basename(out_img_path)
+    return f"{rel_path}\t{text}\n"
+
+# ==============================================================================
+# 6. Điều Phối Huấn Luyện Toàn Cục (Master Generator Loop)
+# ==============================================================================
+def main():
+    default_manifest = os.path.join(TRAINING_DIR, "configs", "v25_dataset_manifest.json")
+    parser = argparse.ArgumentParser(description="Generate Cham OCR V25 Synthetic Dataset (Manifest-Aligned)")
+    parser.add_argument("--manifest", type=str, default=default_manifest, help="Đường dẫn file manifest JSON chuẩn")
+    parser.add_argument("--output_dir", type=str, default="./data/cham_synthetic_v25", help="Thư mục xuất dữ liệu")
+    parser.add_argument("--num_train", type=int, default=None, help="Số mẫu Train (mặc định lấy từ manifest: 140,000)")
+    parser.add_argument("--num_val", type=int, default=None, help="Số mẫu Val (mặc định lấy từ manifest: 10,000)")
+    parser.add_argument("--bake_blur", action="store_true", default=False, help="Bake mờ nặng vào ảnh đĩa (mặc định False: on-the-fly)")
+    parser.add_argument("--workers", type=int, default=None, help="Số tiến trình CPU song song (mặc định cpu_count)")
+    args = parser.parse_args()
+
+    # Nạp cấu hình từ manifest nếu có
+    num_train = args.num_train
+    num_val = args.num_val
+    pillars = ["standard", "anti_blur", "bilingual", "stanza_punct", "minimal_pairs"]
+    weights = [0.433, 0.200, 0.167, 0.120, 0.080] # Tương ứng 65k, 30k, 25k, 18k, 12k
+
+    if os.path.exists(args.manifest):
+        try:
+            with open(args.manifest, "r", encoding="utf-8") as f:
+                manifest_data = json.load(f)
+                dist = manifest_data.get("data_distribution", {})
+                if num_train is None:
+                    num_train = dist.get("train_samples", 140000)
+                if num_val is None:
+                    num_val = dist.get("val_samples", 10000)
+                print(f"📖 Đã nạp cấu hình phân bổ dữ liệu từ Manifest: {args.manifest}")
+        except Exception as e:
+            print(f"⚠️ Không đọc được manifest ({e}), dùng thông số mặc định.")
+
+    if num_train is None: num_train = 140000
+    if num_val is None: num_val = 10000
+
+    num_workers = args.workers or cpu_count()
+    print("=" * 75)
+    print("🚀 MASTER SYNTHETIC DATASET GENERATOR V25 (MANIFEST SYNCHRONIZED)")
+    print(f"   • Số CPU Workers song song : {num_workers} tiến trình")
+    print(f"   • Quy mô mẫu Huấn luyện     : {num_train:,} ảnh")
+    print(f"   • Quy mô mẫu Kiểm thử       : {num_val:,} ảnh")
+    print(f"   • Tổng quy mô dữ liệu       : {num_train + num_val:,} ảnh")
+    print(f"   • Chính sách Mờ Động Học   : {'Bake tĩnh trên đĩa' if args.bake_blur else 'On-The-Fly trong DataLoader (Khuyến nghị)'}")
+    print(f"   • Thư mục đầu ra           : {args.output_dir}")
+    print("=" * 75)
+
+    train_img_dir = os.path.join(args.output_dir, "train_images")
+    val_img_dir = os.path.join(args.output_dir, "val_images")
+    os.makedirs(train_img_dir, exist_ok=True)
+    os.makedirs(val_img_dir, exist_ok=True)
+
+    # Chuẩn bị danh sách tham số cho Train
+    print(f"📦 Chuẩn bị danh mục tác vụ cho {num_train:,} mẫu Train...")
+    train_tasks = []
+    for i in range(num_train):
+        p_type = random.choices(pillars, weights=weights)[0]
+        img_name = f"train_{i+1:07d}.png"
+        out_path = os.path.join(train_img_dir, img_name)
+        train_tasks.append((i, p_type, out_path, args.bake_blur))
+
+    # Chuẩn bị danh sách tham số cho Val
+    print(f"📦 Chuẩn bị danh mục tác vụ cho {num_val:,} mẫu Val...")
+    val_tasks = []
+    for i in range(num_val):
+        p_type = random.choices(pillars, weights=weights)[0]
+        img_name = f"val_{i+1:06d}.png"
+        out_path = os.path.join(val_img_dir, img_name)
+        val_tasks.append((i, p_type, out_path, args.bake_blur))
+
+    t_start = time.time()
+
+    # 1. Sinh tập Train với đa tiến trình
+    print(f"\n⚡ Đang sinh tập Train ({num_train:,} ảnh) trên {num_workers} CPU cores...")
+    train_label_path = os.path.join(args.output_dir, "train_label.txt")
+    with open(train_label_path, "w", encoding="utf-8") as f_train:
+        with Pool(processes=num_workers) as pool:
+            chunksize = 250
+            completed = 0
+            for label_line in pool.imap_unordered(generate_sample_worker, train_tasks, chunksize=chunksize):
+                f_train.write(label_line)
+                completed += 1
+                if completed % 25000 == 0 or completed == num_train:
+                    elapsed = time.time() - t_start
+                    fps = completed / max(elapsed, 1e-5)
+                    print(f"   [Train Progress] {completed:,} / {num_train:,} ảnh ({completed/num_train*100:.1f}%) - Tốc độ: {fps:.1f} ảnh/s")
+
+    t_train_done = time.time()
+    print(f"✅ Hoàn thành tập Train trong {t_train_done - t_start:.2f}s (~{(t_train_done - t_start)/60:.1f} phút)")
+
+    # 2. Sinh tập Val với đa tiến trình
+    if num_val > 0:
+        print(f"\n⚡ Đang sinh tập Val ({num_val:,} ảnh) trên {num_workers} CPU cores...")
+        val_label_path = os.path.join(args.output_dir, "val_label.txt")
+        with open(val_label_path, "w", encoding="utf-8") as f_val:
+            with Pool(processes=num_workers) as pool:
+                chunksize = 250
+                completed = 0
+                for label_line in pool.imap_unordered(generate_sample_worker, val_tasks, chunksize=chunksize):
+                    f_val.write(label_line)
+                    completed += 1
+                    if completed % 5000 == 0 or completed == num_val:
+                        print(f"   [Val Progress] {completed:,} / {num_val:,} ảnh ({completed/num_val*100:.1f}%)")
+    else:
+        val_label_path = os.path.join(args.output_dir, "val_label.txt")
+        print(f"\n❄️  Bỏ qua sinh tập Val (num_val = 0). Giữ nguyên tập Val đóng băng tại: {val_label_path}")
+
+    total_time = time.time() - t_start
+    print("=" * 75)
+    print(f"🎉 HOÀN TẤT SINH TOÀN BỘ BỘ DỮ LIỆU V25 ({num_train + num_val:,} ẢNH)!")
+    print(f"   • Tổng thời gian thực thi: {total_time:.2f}s (~{total_time/60:.2f} phút)")
+    print(f"   • Tệp nhãn Train          : {train_label_path}")
+    print(f"   • Tệp nhãn Val            : {val_label_path}")
+    print("=" * 75)
+
+if __name__ == "__main__":
+    main()