Release weights (HAND-Decoding v1.1.0)

README.md

@@ -0,0 +1,70 @@
+---
+license: cc-by-4.0
+library_name: pytorch
+pipeline_tag: image-to-text
+tags:
+- handwritten-text-recognition
+- document-layout-analysis
+- htr
+- ocr
+- historical-documents
+- read-2016
+---
+
+# hand-read2016-page
+
+Part of **HAND: Unified Text–Layout Decoding for Handwritten Document Recognition** (Hamdan, Rahiche, Cheriet).
+Code: [github.com/DocumentRecognitionModels/HAND-Decoding](https://github.com/DocumentRecognitionModels/HAND-Decoding) · Demo: [HAND-Decoding demo](https://huggingface.co/spaces/MHamdan/HAND-Decoding-demo)
+
+The READ 2016 single-page model reported in the paper (1,258,600 training samples).
+
+- **Architecture:** fully convolutional encoder and eight-layer transformer decoder; one output
+  stream of characters and layout tokens (page, page number, section, annotation, body).
+- **Parameters:** 7,033,700 plus 365,968 in the speculative draft heads (m = 5).
+- **Input:** an RGB document image, resampled to 150 dpi.
+- **Result reported in the paper:** READ 2016 single-page test (50 pages): CER 3.55 %, WER 13.31 %, LOER 0.0529, mAP-CER 0.9264.
+- **Scope:** Reads one page. On a double- or triple-page image it reads the first page and stops.
+
+Verified before release: decoding the stored example image with these weights reproduces the
+prediction recorded in the repository token for token; with the draft heads, speculative decoding reproduces greedy decoding.
+
+```python
+# git clone https://github.com/DocumentRecognitionModels/HAND-Decoding && cd HAND-Decoding && pip install -r requirements.txt
+import sys; sys.path.insert(0, "release")
+from hand_release.inference import HANDRecognizer
+
+model = HANDRecognizer.from_pretrained("MHamdan/hand-read2016-page", kv_cache=True, speculative=True)
+page = model.read("page.jpg", source_dpi=300)   # 300 for a raw scan, 150 if already resampled
+print(page.text)      # transcription; page.raw keeps the layout tokens inline
+```
+
+## Licence and attribution
+
+These weights are released under the Creative Commons Attribution 4.0 International licence
+(CC BY 4.0, https://creativecommons.org/licenses/by/4.0/). They are Adapted Material of:
+
+- **Pretrained Document Attention Network for Handwritten Text Recognition**, Denis Coquenet,
+  Zenodo, DOI 10.5281/zenodo.7244382, CC BY 4.0, file `fcn_read_2016_line_syn.pt`
+  (sha256 557d6c349131b113c1e316effdb028eff9b37bf39a48d0689125a3783b86a7c3). Its encoder weights
+  initialised this model, which was then trained further on READ 2016 page images and synthetic
+  pages. The model was trained on READ 2016 (Sánchez et al., Zenodo 10.5281/zenodo.1297399,
+  CC BY 4.0).
+
+The weights are provided as-is, without warranty. The code that loads them is MIT-licensed, with
+DAN-derived files under CeCILL-C; see `release/NOTICE.md` in
+https://github.com/DocumentRecognitionModels/HAND-Decoding.
+
+## Citation
+
+The paper is under review. Until it is published, please cite the preprint, which appeared under an earlier title:
+
+```bibtex
+@article{hamdan2024hand,
+  title   = {{HAND}: Hierarchical Attention Network for Multi-Scale Handwritten
+             Document Recognition and Layout Analysis},
+  author  = {Hamdan, Mohammed and Rahiche, Abderrahmane and Cheriet, Mohamed},
+  journal = {arXiv preprint arXiv:2412.18981},
+  year    = {2024},
+  url     = {https://arxiv.org/abs/2412.18981}
+}
+```

charset.json

@@ -0,0 +1,111 @@
+{
+  "charset": [
+    "\n",
+    " ",
+    "(",
+    ")",
+    "+",
+    ",",
+    "-",
+    ".",
+    "/",
+    "0",
+    "1",
+    "2",
+    "3",
+    "4",
+    "5",
+    "6",
+    "7",
+    "8",
+    "9",
+    ":",
+    "A",
+    "B",
+    "C",
+    "D",
+    "E",
+    "F",
+    "G",
+    "H",
+    "I",
+    "J",
+    "K",
+    "L",
+    "M",
+    "N",
+    "O",
+    "P",
+    "Q",
+    "R",
+    "S",
+    "T",
+    "U",
+    "V",
+    "W",
+    "Y",
+    "Z",
+    "[",
+    "]",
+    "a",
+    "b",
+    "c",
+    "d",
+    "e",
+    "f",
+    "g",
+    "h",
+    "i",
+    "j",
+    "k",
+    "l",
+    "m",
+    "n",
+    "o",
+    "p",
+    "q",
+    "r",
+    "s",
+    "t",
+    "u",
+    "v",
+    "w",
+    "x",
+    "y",
+    "z",
+    "¬",
+    "¾",
+    "Ö",
+    "ß",
+    "ä",
+    "ö",
+    "ü",
+    "ÿ",
+    "ā",
+    "ē",
+    "ō",
+    "ū",
+    "ȳ",
+    "̄",
+    "̈",
+    "—",
+    "Ⓐ",
+    "Ⓑ",
+    "Ⓝ",
+    "Ⓟ",
+    "Ⓢ",
+    "ⓐ",
+    "ⓑ",
+    "ⓝ",
+    "ⓟ",
+    "ⓢ"
+  ],
+  "vocab_size": 99,
+  "additional_tokens": 1,
+  "special_tokens": {
+    "end": 99,
+    "start": 100,
+    "pad": 101
+  },
+  "note": "index convention from hand/OCR/ocr_dataset_manager.py:74-99 with charset_mode='seq2seq'; the output layer covers 0..vocab_size, i.e. the charset plus <end>"
+}

config.json

@@ -0,0 +1,49 @@
+{
+  "model_type": "hand-dan-page",
+  "input_channels": 3,
+  "dropout": 0.5,
+  "enc_dim": 256,
+  "nb_layers": 5,
+  "pe_h_max": 500,
+  "pe_w_max": 1000,
+  "l_max": 15000,
+  "dec_num_layers": 8,
+  "dec_num_heads": 4,
+  "dec_res_dropout": 0.1,
+  "dec_pred_dropout": 0.1,
+  "dec_att_dropout": 0.1,
+  "dec_dim_feedforward": 256,
+  "attention_win": 100,
+  "use_tokens_from_all_lines": true,
+  "use_first_pass_tokens": true,
+  "two_step_pos_enc_mode": "cat",
+  "use_line_indices": false,
+  "vocab_size": 99,
+  "additional_tokens": 1,
+  "max_char_prediction": 3000,
+  "max_line_pred": 100,
+  "max_pred_per_line": 150,
+  "working_dpi": 150,
+  "image_mean": [
+    202.70203512453844,
+    190.85819291654516,
+    136.08993740340918
+  ],
+  "image_std": [
+    71.95683449787683,
+    72.00679752035164,
+    57.33656372156373
+  ],
+  "expected_parameters": 7033700,
+  "parameters_encoder": 1706240,
+  "parameters_decoder": 5327460,
+  "source_checkpoint": {
+    "path": "outputs/e14_budget_1p26M_s0/checkpoints/best_3580.pt",
+    "sha256": "d94af1419554a057ae671733af2fe4cf3f63d1f8b3652f9de17277b432632f5f",
+    "bytes": 84821823,
+    "epoch": 3580,
+    "step": 1253350,
+    "recorded_best_valid_cer": 0.0409,
+    "use_line_indices_source": "checkpoint carries no 'use_line_indices' key; inferred from additional_tokens == 1, i.e. (additional_tokens == 3) -> False"
+  }
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b842d1e1e1d9568825253b79d01b91702635a665d19227c9a2748de0ec49515b
+size 28169776

preprocessor_config.json

@@ -0,0 +1,20 @@
+{
+  "do_rgb": true,
+  "working_dpi": 150,
+  "resample": "PIL.Image.BILINEAR",
+  "image_mean": [
+    202.70203512453844,
+    190.85819291654516,
+    136.08993740340918
+  ],
+  "image_std": [
+    71.95683449787683,
+    72.00679752035164,
+    57.33656372156373
+  ],
+  "do_resize_to_fixed_size": false,
+  "do_pad": false,
+  "encoder_reduction_h": 32,
+  "encoder_reduction_w": 8,
+  "note": "Training-set channel statistics for READ_2016_page_sem_dan, read from outputs/e14_budget_1p26M_s0/results/params.txt:202-221. Scale is 0-255, NOT 0-1, and these are NOT ImageNet statistics. The page must reach the model at 150 dpi: raw READ 2016 scans are 300 dpi and must be halved (PIL BILINEAR); the pages under formatted/ already are 150 dpi and must NOT be resized again."
+}

spec_heads_m5.json

@@ -0,0 +1,13 @@
+{
+  "m": 5,
+  "hidden": 256,
+  "vocab_out": 100,
+  "parameters": 365968,
+  "d_model": 256,
+  "base_checkpoint_sha256": "d94af1419554a057ae671733af2fe4cf3f63d1f8b3652f9de17277b432632f5f",
+  "source": {
+    "path": "outputs/spec_heads_m5/heads.pt",
+    "sha256": "3598151fd92a50fd722b3ac0931310ad0ca2776b62dd84339ad51ade2dd71976",
+    "bytes": 1469127
+  }
+}

spec_heads_m5.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc73d191871da0241bcd82ea68d2844ffea2385635712c9951802ecfe9bbbf2a
+size 1465160