File size: 17,615 Bytes
f9bc0c1
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
# Indra OCR β€” Engineering Investigation (v3.1.1, package 0.3.2)

Investigator: OCR_INDRA workspace, 2026-10-17.
Method: every claim below is backed by (a) model artifacts inspected bit-for-bit
(`models/indra-ocr-v3.1.1/`, Hub revision `v3.1.1`) or (b) source at a cited
file:line in the cloned repo (`indra/`, commit `3d773ae`, tag v0.3.2).
Verification scripts live in `tools/indra_eval/` and are re-runnable.

Status legend: **[V]** verified with evidence here, **[B]** blocked (needs the
remote machine `khalti-t4`), **[X]** closed as refuted/inaccurate vs the
earlier report `INDRA_OCR_NEPALI_FEASIBILITY.md`.

---

## 0. Verdict

**The recognizer is trainable in principle and Nepali-ready in vocabulary, but
fine-tuning is a reimplementation project: the repo ships no training code and
no training weights for anything but the recognizer.**

- Zero-shot evaluation (Experiment A) is *ready today* β€” the adapter
  (`tools/indra_eval/eval_indra.py`) runs Indra end-to-end on CPU and scores
  CER; it needs only the frozen Nepali eval sets from the remote machine [B].
- The CTC recognizer (`rec.onnx`) covers 100% of Devanagari (all 128 assigned
  codepoints U+0900–U+097F verified in the vocab) β€” no head surgery needed [V].
- In a same-crop head-to-head under our environment, the ~1.24 M-param
  Indra recognizer scored **CER 3.49%** on a synthetic Nepali page vs the
  deployed PP-OCRv5-Mobile Devanagari recognizer's published behavior on
  similar synthetic material (to be re-measured on identical crops) [V smoke / B].
- Fine-tuning only the recognizer with `model.safetensors` as init is the
  *only* supported adaptation path: the artifacts ship trainable weights for
  `rec/` exclusively; det/order/orient/math exist only as ONNX graphs with no
  accompanying state dict [V].

---

## 1. What exactly is the ~1M-param recognizer? [V]

**Answer: a MobileNetV3-style 1-D CNN trunk with padding-masked SE blocks and
a flat CTC head β€” 1,236,366 parameters exactly, no RNN, no attention.**

Evidence: `tools/indra_eval/inspect_rec_nodes.py` (201-node dump of
`rec.onnx`), `tools/indra_eval/inspect_rec_mask.py` (intermediate-tensor run),
`tools/indra_eval/inspect_models.py` (safetensors ↔ ONNX initializer
cross-check: 108 tensors, 1,236,366 F32 params, byte-identical set).

### 1.1 Tensor flow (all verified from the graph)

```
input  x      [B,1,32,W]  f32 ink map   (0=paper, 1=ink; see Β§1.3)
input  widths [B]         int64         (true crop widths BEFORE padding)

0  Conv(k=32x4, stride=32x4, 1β†’128, pad 0)   "collapse height, stride time 4"
2  Squeeze axis 2 β†’ [B,128,T]  with T = W/4
1  BN(eps 1e-5, stem-only BatchNormalization) β†’ HardSwish
43 Mul Γ— Cast_4  (padding mask, see Β§1.2)  β†’ trunk input

10 Γ— { dw Conv [128,1,9] g=128 pad 4/4
       expand Conv 1Γ—1 128β†’256 β†’ HardSwish
       SE: Mul(mask) β†’ ReduceSum(t)  ←/β†’ ReduceSum(mask) = valid count
           Div (mean over VALID timesteps, Clip min 1)
           fc1 256β†’64 β†’ Relu β†’ fc2 64β†’256 β†’ Sigmoid β†’ gate Mul
       project Conv 1Γ—1 256β†’128
       Add residual β†’ Mul(mask)  }          ← mask re-applied per block

188 Conv 1Γ—1 128β†’1678 (head) β†’ Transpose [B,T,1678]
191 ArgMax β†’ ids [B,T]
190/193..200 softmax-max (logsumexp-free: ReduceMax, Exp, ReduceSum, Recip)
           β†’ conf [B,T]
27  Clip(lengths) β†’ lengths [B]
```

Op census (from the node dump): 52 Conv, 11 HardSwish, 10 Relu, 10 Sigmoid
(SE gates), 32 Mul, 12 ReduceSum, 1 BatchNormalization, 1 ArgMax β€” plus
glue. The parameters: 10 identical residual blocks (dw 128Γ—1Γ—9 + bias,
expand 256Γ—128, project 128Γ—256, SE 64Γ—256 + 256Γ—64) β‰ˆ 118 k/block + stem
128Γ—32Γ—4 + BN 4Γ—128 + head 1678Γ—128 β‰ˆ 1,236,366 total [V].

### 1.2 The padding mask `Cast_4` β€” semantics now fully pinned [V, closes a [U]]

`Cast_4 = (arange(W/4) < lengths).float()` of shape `[B,1,W']`, built in-graph
from:
- nodes 3–27: `lengths = floor((widths+3)/4) = ceil(widths/4)`, clipped to
  `W/4` (Clip max = `Shape(x)[2] / 4`... clipped to the feature length);
  numeric table: `widths 1,2,3,4 β†’ 1; 5 β†’ 2; 9 β†’ 3; 65 β†’ 17; 129 β†’ 32 (= W/4,
  clipped); 200 β†’ 32`.
- nodes 31–40: `Range(0, W/4) < lengths` (broadcast `[1,W'] < [B,1]`) β†’
  `Cast` f32 twice (PyTorch export artifact of `unsqueeze().to(f32)`).

The mask is consumed **three ways** (this closes the earlier "[U] what does
Cast_4 do?"):
1. **post-stem zeroing** (node 43) and **per-block output re-zeroing**
   (nodes 43, 61, 75, …, 187): the trunk never carries energy into padded
   timesteps β€” proven numerically: with ink drawn across the whole 128-column
   input but `widths=64`, final-block features at t β‰₯ 16 are *exactly 0.0*
   (max |f| = 0.0) while the same pixels unmasked (widths=128) reach |f|=15.6;
2. **padding-aware SE pooling** (each block: `ReduceSum(HW Γ— mask) /
   max(ReduceSum(mask), 1)` β†’ channel means over VALID timesteps only; the
   mask sum is the divisor, so padded columns cannot dilute the squeeze stats);
3. the **head reads masked features** (node 187β†’188), so at padded timesteps
   the graph emits whatever the head computes on a zero vector β€” measured:
   argmax = 0 (blank) with conf β‰ˆ 0.78. **`ids/conf` at t β‰₯ lengths are NOT
   special-cased; callers must truncate by `lengths`.** `pipeline.py:297`
   does exactly this (`n = int(lengths[k])`).

### 1.3 Time stride is 4, not 8 [X β€” closes a wrong claim]

`rec_preproc.py:24` says `WIDTH_MULTIPLE = 8  # widths are padded to a
multiple of this (timestep stride)`. The graph contradicts the parenthetical:
the stem Conv strides are `[32,4]`, giving **one timestep per 4 input
columns** (`T = W/4`, `lengths = ceil(widths/4)`), and padding to a multiple
of 8 is about the W dimension of the input only (it makes every 2 timesteps
aligned, a vectorization nicety). Verified numerically: `T = 32` for
`W = 128`, and `lengths` equals `ceil(widths/4)` at every measured value.
The comment is wrong as written; the code is right.

### 1.4 Output contract [V β€” closes a [U]]

Graph outputs are exactly `ids [B,T] int64`, `conf [B,T] f32`,
`lengths [B] int64` (argmax + softmax max prob + valid count computed
in-graph). `pipeline.py:294` consumes them positionally as
`best, pmax, lengths`, then:
- truncates to `n = int(lengths[k])` (:297),
- CTC-greedy collapse: keep β‰  blank(0) and skip repeats (:299),
- confidence = mean `pmax` over kept tokens (:315),
- **drops the whole line if conf < `MIN_CONF = 0.5`** (:60, :316–319).

Any retrained recognizer exported to ONNX must reproduce this contract
(argmax + softmax-max + lengths on the same output names/order), or
`pipeline.py:294` and the x-positions step (which maps CTC peak timesteps
back to page x via the crop scale, :295–306) must be updated.

### 1.5 Recognizer input preprocessing (the shard that fine-tuning must clone)

`rec_preproc.py` β€” `crop_line` (:27): expand box by `PAD_Y=0.2 Γ— h` vertically
and `PAD_X=0.3 Γ— h` horizontally (:22–23), restrict to page, resize so
height = `LINE_HEIGHT = 32` keeping aspect (`out_w = max(WIDTH_MULTIPLE,
min(max_width=4096, round(cw*32/ch)))`), producing an ink map where paper = 0.
`pad_batch` (:56) right-pads W to a multiple of 8 in the batch; `width_batches`
(:69) groups crops so W-dim batch padding is bounded. The TRAINER uses the
same crop per its docstring (:5–10 "expand … resize … ink map") β€” the docstring
states train/infer share this shard, which is why zero-shot works at all and
why fine-tuning must reimplement `crop_line` + `pad_batch` bit-exactly.

## 2. What handles Devanagari? [V]

**Only `rec.onnx` reads Devanagari β€” and its vocabulary already includes
every assigned Devanagari codepoint.**

- `vocab.py:170` `Vocab.multi()` β†’ 1,677 tokens + blank class 0 (:3, :207),
  head = 1,678 classes. The `meta.json` vocab string is exactly this vocab
  (`tools/indra_eval/inspect_vocab.py`).
- Coverage: **128/128 assigned codepoints in U+0900–U+097F** β€” 80 Lo letters,
  11 Mc vowel signs, 23 Mn marks incl. ΰ€Ό (U+093C nukta) and the conjunct
  virama ΰ€•ΰ₯, plus ΰ₯‰ (U+0950), ΰ€ ΰ₯€ (U+0964), ΰ₯₯ (U+0965), digits ΰ₯¦β€“ΰ₯―
  (U+0966–096F), β‚Ή (U+20B9). Bengali block also present (96 codepoints).
- The recognizer is routed by two *fixed* guards, not the network:
  - `script.onnx` (hanzi binary classifier, sigmoid scalar; `pipeline.py:274–289`)
    sends hanzi lines to `rec_cjk.onnx` β€” irrelevant for Nepali pages
    (all-Devanagari pages never trigger it; the classifier is line-level and
    trained on hanzi).
  - `vocab.canonical` (:42) NFC-normalizes; `Vocab.encode` (:212) drops
    out-of-vocab codepoints β€” fine-tuning data must be pre-normalized the
    same way or the loss silently degrades (the trainer's data pipeline is
    closed; safest is to clone `canonical`/`encode`).
- Confirmed counter-evidence collected in the smoke run: ΰ€Ό (nukta) is often
  *dropped contextually* at 32 px height (ΰ€°ΰ€Ό and ΰ€° differ only by a 1–2 px
  dot); β‚Ή was read as ΰ€°. Both are training-distribution issues, not capability
  gaps β€” exactly the kind of thing fine-tuning would fix (Experiment B).

## 3. Component map: trainable / fixed-but-replaceable / fixed [V]

| Component | Where weights come from | Trainable via shipped artifacts? | Replaceability |
|---|---|---|---|
| `rec.onnx` recognizer | `model.safetensors` (108 tensors, all `rec/`) | **YES** β€” the only shipped state dict; init for a torch CTC retrain (Experiment B) | head/vocab identical β†’ drop-in for +1,678 classes |
| `det.onnx` line detector | ONNX only | NO state dict shipped | could retrain in torch (targets spec exists: `det_post.encode_targets`, `det_post.py:48` β€” `core logit + top/bottom distances` per line, `CORE_FRAC=0.4`, `CORE_X_SHRINK=0.15`) β€” but training code is closed β†’ high effort |
| `order.onnx` reading order/layout | ONNX only | NO | same: `order_post.label_order`/`decode_*` reveal supervision (10 block classes) but no weights |
| `orient.onnx` page rotation | ONNX only | NO | trivial to replace with a torch/tlclassifier or heuristic |
| `mask.onnx` CenterMask | NOT SHIPPED in v3.1.1 (`pipeline.py:201–205` optional) | β€” | absent here; `crop_lines(gray, boxes, 32, None)` runs w/o it (verified) |
| `script.onnx` + `rec_cjk.onnx` | ONNX only | NO | optional; would be excluded from a Devanagari build |
| `math*.onnx` (4 files) | NOT shipped unless `math=True` (`models.py:22–44`) | β€” | out of scope |
| geometry/deskew/layout postproc | pure Python (`pipeline.py:37–149`, `layout_post.py`) | fixed code, no training needed | replaceable with own code paths |
| text-layer (PDF) naive path | `textlayer.py` | fixed code | irrelevant to scans/Nepali notes |

**The decisive asymmetry:** Google ships a *recognizer-only* training starting
point. Fine-tuning anything else means either (a) training from scratch (det
targets spec'd, but no reference weights/recipe) or (b) leaving det/order
frozen ONNX and only exercising the rec head β€” that's Experiment B; both are
documented in `INDRA_EXPERIMENTS.md`.

## 4. Integration architecture (how a page becomes text) [V]

`pipeline.py:456–470` (`OrtOCR.__call__`):
1. `upright()`: optional `orient.onnx` turn above `ORIENT_MIN = 0.95` (:54) +
   skew estimate/deskew up to 5Β° (:62–63).
2. `detect()`: `det.onnx` on a page resized long-side to detector input,
   threshold β†’ line boxes (uses `det_post.page_to_input` / `decode`;
   `ADAPTIVE_MIN_H=9px` re-detect pass at a larger scale, :37–40).
3. `recognize()` (:264–320): CenterMask if present (not shipped); per-line
   `crop_line` β†’ `pad_batch` β†’ rec session, CTC collapse & conf gating
   (Β§1.2/Β§1.4).
4. line splitting at wide gaps (`SPLIT_GAP=3.0`, `CELL_GAP=1.8`,
   `GUTTER_GAP=1.6`, :44–47) and gutter detection (:342).
5. `reading_order()` on `order.onnx` (`feats [1,N,16]`, outputs pairwise salience
   `a`, block class `cls` [10 classes], same-block `same`, table t-heads `tab`)
   then layout (`layout_post.py: page_blocks`, `page_markdown`).

MIN_CONF=0.5 means **a busy page with low-quality Devanagari can silently
emit empty lines** in Experiment A β€” compare-on-GT-boxes (rec-lines mode)
sidesteps this entirely, which is why Experiment A does both full-pipeline
and rec-lines scoring.

## 5. Licensing [V]

- **Model weights** (`models/indra-ocr-v3.1.1/LICENSE`): **AI PUBS OPEN RAIL-M
  (MODIFIED), v0.1, 2023-03-02**. Section II ΒΆ36 states verbatim: *"Use may
  include creating any content with, fine-tuning, updating, running, training,
  evaluating and/or re-parametrizing the Model"* β€” **fine-tuning is explicitly
  licensed**. ΒΆ5 attaches Attachment A use-based restrictions (standard
  OpenRAIL misuse list β€” weapons/illegal/defamation; OCR and document
  processing is untouched) and requires passing the same restrictions to
  distributees; ΒΆ4 permits hosting/redistribution with notice. Derivative
  weights (fine-tuned checkpoints, distillations) fall under the same license.
- **Code** (`indra/LICENSE`): **Apache-2.0** β€” reuse, modification,
  redistribution of the inference package are standard.
- Practical consequence: our fine-tuned recognizer (Experiment B) is legally a
  "Derivative of the Model" under OpenRAIL-M β€” internal deployment and
  evaluation are unrestricted; redistribution requires keeping the license +
  restriction notice. Full matrix incl. Nepali datasets: `Β§12` of
  `INDRA_EXPERIMENTS.md` (dataset side **[B]** β€” needs `docs/DATA_LICENSING.md`
  on the remote machine).

## 6. Closing the prior report's [U] items

| Prior unknown | Status | Evidence |
|---|---|---|
| exact recognizer arch (RNN? CNN?) | **[V] + [X]**: pure CNN+CTC, NO RNN/attention | Β§1.1, node dump |
| timestep stride | **[X]**: 4, not 8 | Β§1.3 (graph constants + inc.timings) |
| output contract (best/pmax docstrings vs solo head) | **[V]**: `ids`,`conf`,`lengths` names; truncation mandatory | Β§1.4 |
| safetensors ↔ ONNX mapping (esp. depthwise `onnx::Conv_369…397`) | **[V]**: 108 tensors = 108 initializers, 1,236,366 params both ways; dw weights are `rec/onnx::Conv_369` [128,1,9] + `…370` [128] bias pairs | `inspect_models.py` |
| Devanagari coverage claim | **[V]**: 128/128 codepoints in vocab | `inspect_vocab.py` |
| What re-export must reproduce | **[V]**: mask + per-block re-zeroing + SE-valid-norm + decode tail; `lengths = ceil(widths/4)` clip W/4 | Β§1.2, `inspect_rec_mask.py` |
| Confidence semantics of padded steps | **[X]**: head-on-zeros β‰ˆ blank conf 0.78; caller truncates | Β§1.2(3) |
| vocab[token i+1] ↔ blank 0 | **[V]** | `vocab.py:3,207,241–242` |
| rec input normalization | **[V]**: ink map only (0 paper, 1 ink) per `crop_line`, no mean/std | `rec_preproc.py:1–10` |
| enables fine-tune? | **[V] yes for rec only** (weights exist; no training code; CTC contract reconstructed) | Β§3, `Β§B` doc |
| full pipeline runs on Nepali test image | **[V] smoke**: synthetic shaped page, 10/10 lines, CER 2.6–3.5% | `results/smoke/` |
| what differs in real (non-synth) Nepali | **[B]** frozen remote sets + baseline | Experiment A |
| trained det/order weights for fine-tune | **[V] not shipped** (ONNX only) | Β§3 |

## 7. Open items (not closed by this investigation)

1. **[B]** Zero-shot numbers on the frozen Nepali eval sets (needs `khalti-t4`).
2. **[B]** Head-to-head vs PP-OCRv5 Mobile Devanagari *on the same crops* β€” the
   remote machine holds the paddle recognizer + eval harness. The local
   smoke CER of 2.6–3.5% is on a synthetic page with a 32 px font and is
   *not* a benchmark.
3. **[B]** Licensing matrix final values need `docs/DATA_LICENSING.md` and
   `docs/DEVANAGARI_RESOURCES_INVENTORY.md` (remote).
4. FX-detector quality on Devanagari (qubee endpoints, aggressively slanted
   conjuncts) is unknowable without the det model being exercised on real
   scans β€” that's part of Experiment A full-pipeline.
5. The *root* training repo (referenced by name in the closed code) is not
   accessible; Archived recipe unknowns: batch sizes, LR schedule, data mix.

## 8. Reproduce everything

```bash
cd /mnt/c/Users/Dell/Desktop/OCR_INDRA            # or wherever the repo is
PY=/tmp/opencode/indra-lib                        # pip --target deps
export PYTHONPATH="$PWD/indra:$PY"

python3 tools/indra_eval/inspect_vocab.py          # vocab + Devanagari coverage
python3 tools/indra_eval/inspect_models.py         # graph I/O + param count
python3 tools/indra_eval/inspect_rec_nodes.py      # full node dump + smoke run
python3 tools/indra_eval/inspect_rec_mask.py       # mask semantics + lengths table

python3 tools/indra_eval/make_synth_page.py results/smoke   # synth page + GT
python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \
        --boxes results/smoke/synth_nepali_gt.jsonl --jsonl results/smoke/rec_only.jsonl
python3 tools/indra_eval/eval_indra.py predict results/smoke/synth_nepali_page.png \
        --jsonl results/smoke/full_run.jsonl --print-text
python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \
        results/smoke/rec_only.jsonl --detail results/smoke/cer_rec_only.json --rec-only
python3 tools/indra_eval/eval_indra.py cer results/smoke/synth_nepali_record.jsonl \
        results/smoke/full_run.jsonl --detail results/smoke/cer_full.json
```

Observed (this machine, WSL, CPU, 8 cores, 3 GB RAM): rec-only CER 3.49% /
WER 15.79%; full-pipeline CER 2.62% / WER 13.16%; 10/10 lines found;
mean_conf 0.962–0.963. Timings on this box are meaningless (det β‰ˆ 18.6 min
β€” machine-starved; bench on the T4).