InviolusOCR 2026-10-31: conv CTC model, int8 keeps accuracy

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+inviolusocr.keras filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,135 @@
+---
+license: cc-by-nc-sa-4.0
+language:
+- tr
+library_name: keras
+pipeline_tag: image-to-text
+tags:
+- ocr
+- text-recognition
+- turkish
+- tinyml
+- tflite
+- int8
+- ctc
+- microcontroller
+---
+
+# InviolusOCR: tiny Turkish line OCR for microcontrollers
+
+A compact convolutional CTC recognizer for **one cropped line of printed
+Turkish text**: 560,954 parameters and a **598 KB full-int8 TensorFlow Lite**
+file that keeps almost all of the float model's accuracy.
+
+> **Türkçe özet:** Mikrodenetleyiciler için tek satır Türkçe metin okuyan,
+> 561 bin parametreli, int8 TFLite dosyası 598 KB olan küçük bir OCR modeli.
+> int8 hâli, ayrılmış test satırlarının %78,4'ünü ve gerçek sahne metni
+> (TS-TR) test kırpıntılarının %58,2'sini harfi harfine doğru okuyor. Ticari
+> olmayan kullanım içindir (CC BY-NC-SA 4.0).
+
+This replaces [inviolus/erdem-ocr-tr-tinyml](https://huggingface.co/inviolus/erdem-ocr-tr-tinyml)
+(2026-10-09), whose int8 file lost most of its accuracy.
+
+## Files
+
+| File | What it is |
+|---|---|
+| `inviolusocr.keras` | Float model, input `(batch, 32, 160, 1)`, output `(batch, 40, 90)` softmax |
+| `inviolusocr_int8.tflite` | Full-int8 export, input `[1, 32, 160, 1]` int8 (scale 0.007843, zero point -1), output `[1, 40, 90]` int8 (scale 0.00390625, zero point -128) |
+| `alphabet.json` | Character set used to decode the output |
+| `benchmark.json` | All scores below, for this model and the earlier versions |
+
+## Model
+
+- **Input:** one horizontal text line, grayscale, resized to 160×32, scaled
+  to `[-1, 1]` (`pixel / 127.5 - 1`).
+- **Architecture:** 4 conv + batch-norm + pool blocks, a dense projection,
+  then four residual dilated 1-D convolutions (dilation 1, 2, 4, 1) instead of
+  an LSTM, and a dense softmax. Only ops that quantize cleanly to int8.
+- **Output:** 40 timesteps × 90 classes. Class `0` is padding, `1..88` are the
+  characters of `alphabet.json` in order, `89` is the CTC blank.
+- **Character set:** digits, ASCII letters, `ç ğ ı İ ö ş ü Ç Ğ Ö Ş Ü`, space
+  and `.,:;!?-_/()'%`. Up to 24 characters per line.
+- **Not included:** text detection. The line must already be cropped.
+
+## Usage
+
+```python
+import json
+import numpy as np
+import tensorflow as tf
+
+alphabet = json.load(open("alphabet.json", encoding="utf-8"))["alphabet"]
+BLANK = len(alphabet) + 1
+
+def preprocess(path):
+    image = tf.io.decode_image(tf.io.read_file(path), channels=1, expand_animations=False)
+    image = tf.image.resize(image, [32, 160])
+    return (tf.cast(image, tf.float32) / 127.5 - 1.0).numpy()
+
+def decode(ids):
+    text, previous = [], None
+    for i in ids:
+        if i != previous and 0 < i < BLANK:
+            text.append(alphabet[i - 1])
+        previous = i
+    return "".join(text)
+
+# int8 TFLite (the file meant for devices)
+interpreter = tf.lite.Interpreter(model_path="inviolusocr_int8.tflite")
+interpreter.allocate_tensors()
+inp, out = interpreter.get_input_details()[0], interpreter.get_output_details()[0]
+scale, zero = inp["quantization"]
+x = np.clip(np.round(preprocess("line.png") / scale + zero), -128, 127).astype(np.int8)
+interpreter.set_tensor(inp["index"], x[np.newaxis])
+interpreter.invoke()
+print(decode(interpreter.get_tensor(out["index"])[0].argmax(-1)))
+```
+
+## Results
+
+Exact-line accuracy (the whole line correct), all models scored on the same
+rows: 12,093 held-out lines (synthetic, Zenodo, TS-TR, esengul3, STS-TR) and
+the 2,078-crop official TS-TR test split (real scene text, never trained on).
+
+![Exact-line accuracy on held-out lines, float and int8](benchmark-1031-heldout-light.svg)
+
+| Model | Params | int8 size | Held-out float | Held-out **int8** | TS-TR float | TS-TR **int8** |
+|---|---:|---:|---:|---:|---:|---:|
+| erdem-ocr 2026-10-09 (LSTM) | 631,226 | 665 KB | 58.8% | 32.9% | 45.3% | 33.5% |
+| unpublished 2026-10-25 (LSTM) | 631,226 | 665 KB | 80.0% | 32.1% | 65.1% | 48.7% |
+| **InviolusOCR 2026-10-31** | **560,954** | **598 KB** | 79.4% | **78.4%** | 60.0% | **58.2%** |
+
+Character error rate of the int8 file: 9.1% held-out, 18.1% TS-TR.
+
+The 2026-10-25 model is not published because it was trained with a dataset
+that states no license.
+
+## Limitations
+
+- Printed text only; no handwriting, curved text or full pages.
+- Real scene text is still the weak spot (58% exact).
+- **Licence plates are not read** (0% exact on 250 plate crops in a first,
+  unchecked test).
+- Text detection and line cropping must be done before this model runs.
+
+## Training data
+
+Trained for 200 minutes on CPU, starting from the 2026-10-09 weights, on
+about 66,000 crops:
+
+- Synthetic Turkish lines rendered from a 50k-word Turkish word list.
+- [Turkish OCR Text Image Dataset](https://zenodo.org/records/21923181), Zenodo, CC BY 4.0.
+- [Turkish Scene Text Recognition (TS-TR)](https://www.kaggle.com/datasets/serdaryildiz/turkish-scene-text-recognition-dataset), Serdar Yıldız, CC BY-NC 4.0.
+- [Synthetic Turkish Scene Text (STS-TR)](https://www.kaggle.com/datasets/serdaryildiz/synthetic-turkish-scene-text-recognition-dataset), Serdar Yıldız, CC BY-NC 4.0.
+- [Turkish Word OCR](https://huggingface.co/datasets/esengul3/turkish-word-ocr), esengul3, CC BY-SA 4.0.
+
+## License
+
+Because of the non-commercial (TS-TR, STS-TR) and share-alike (esengul3)
+terms of the training data, this model is released under
+**[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/)**:
+non-commercial use only, credit this model and the datasets above, and share
+adaptations under the same license.
+
+Training code (MIT): [ErdemWilkinson/InviolusOCR](https://github.com/ErdemWilkinson/InviolusOCR).

alphabet.json

@@ -0,0 +1,4 @@
+{
+  "alphabet": "0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ çğıİöşüÇĞÖŞÜ.,:;!?-_/()'%",
+  "blank_id": 89
+}

benchmark.json

@@ -0,0 +1,222 @@
+{
+  "old_2026-10-09": {
+    "params": 631226,
+    "tflite_bytes": 664648,
+    "tstr_test/float": {
+      "samples": 2078,
+      "exact": 0.45332050048123196,
+      "cer": 0.24171940494780966
+    },
+    "tstr_test/int8": {
+      "samples": 2078,
+      "exact": 0.33493743984600577,
+      "cer": 0.333771677488288
+    },
+    "heldout/float": {
+      "samples": 12093,
+      "exact": 0.5876953609526172,
+      "cer": 0.13569795363093237
+    },
+    "heldout/int8": {
+      "samples": 12093,
+      "exact": 0.3289506325973704,
+      "cer": 0.20805435130018604
+    },
+    "plates/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9095901985635826
+    },
+    "plates_nospace/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.911134903640257
+    },
+    "plates/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.926489226869455
+    },
+    "plates_nospace/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9443254817987152
+    }
+  },
+  "old_2026-10-10": {
+    "params": 631226,
+    "tflite_bytes": 664648,
+    "tstr_test/float": {
+      "samples": 2078,
+      "exact": 0.595283926852743,
+      "cer": 0.17489931782690885
+    },
+    "tstr_test/int8": {
+      "samples": 2078,
+      "exact": 0.4273339749759384,
+      "cer": 0.2717185830525191
+    },
+    "heldout/float": {
+      "samples": 12093,
+      "exact": 0.7716034069296287,
+      "cer": 0.09050901074405225
+    },
+    "heldout/int8": {
+      "samples": 12093,
+      "exact": 0.39518729843711237,
+      "cer": 0.1772891551454258
+    },
+    "plates/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8939585973806506
+    },
+    "plates_nospace/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8950749464668094
+    },
+    "plates/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9142374313476975
+    },
+    "plates_nospace/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.917558886509636
+    }
+  },
+  "old_2026-10-25": {
+    "params": 631226,
+    "tflite_bytes": 664648,
+    "tstr_test/float": {
+      "samples": 2078,
+      "exact": 0.651106833493744,
+      "cer": 0.14843428947152132
+    },
+    "tstr_test/int8": {
+      "samples": 2078,
+      "exact": 0.4865255052935515,
+      "cer": 0.24623982904577957
+    },
+    "heldout/float": {
+      "samples": 12093,
+      "exact": 0.8003803853468949,
+      "cer": 0.07878971652420486
+    },
+    "heldout/int8": {
+      "samples": 12093,
+      "exact": 0.320929463325891,
+      "cer": 0.18703337950024995
+    },
+    "plates/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8821292775665399
+    },
+    "plates_nospace/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9250535331905781
+    },
+    "plates/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8901563160118293
+    },
+    "plates_nospace/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9518201284796574
+    }
+  },
+  "A_lstm_clean": {
+    "params": 631226,
+    "tflite_bytes": 664648,
+    "tstr_test/float": {
+      "samples": 2078,
+      "exact": 0.5871029836381135,
+      "cer": 0.17818689898906878
+    },
+    "tstr_test/int8": {
+      "samples": 2078,
+      "exact": 0.4157844080846968,
+      "cer": 0.2721295306977891
+    },
+    "heldout/float": {
+      "samples": 12093,
+      "exact": 0.7687091705945588,
+      "cer": 0.08910761262405652
+    },
+    "heldout/int8": {
+      "samples": 12093,
+      "exact": 0.4822624658893575,
+      "cer": 0.1620950492128404
+    },
+    "plates/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9032530629488804
+    },
+    "plates_nospace/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9089935760171306
+    },
+    "plates/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.904098014364174
+    },
+    "plates_nospace/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.9298715203426124
+    }
+  },
+  "B_conv": {
+    "params": 560954,
+    "tflite_bytes": 597816,
+    "tstr_test/float": {
+      "samples": 2078,
+      "exact": 0.5996150144369586,
+      "cer": 0.166762554450563
+    },
+    "tstr_test/int8": {
+      "samples": 2078,
+      "exact": 0.5822906641000962,
+      "cer": 0.18089915344785074
+    },
+    "heldout/float": {
+      "samples": 12093,
+      "exact": 0.7935169106094435,
+      "cer": 0.0860343711328378
+    },
+    "heldout/int8": {
+      "samples": 12093,
+      "exact": 0.7839245844703547,
+      "cer": 0.09139410429352325
+    },
+    "plates/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.866497676383608
+    },
+    "plates_nospace/float": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8490364025695931
+    },
+    "plates/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.8601605407689058
+    },
+    "plates_nospace/int8": {
+      "samples": 250,
+      "exact": 0.0,
+      "cer": 0.841541755888651
+    }
+  }
+}

inviolusocr.keras

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fc1e15a32a277ac49a7397fbc7a21e1415636c34f993eaa78a7121de92203271
+size 2365508

inviolusocr_int8.tflite

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:693a342760ec0d44aa2ea0d367657d5f1e01dda69eb256b3bdaff1fab561721b
+size 597816