snxtyle commited on
Commit
23a9602
·
verified ·
1 Parent(s): 27c6a4a

Publish the Marathi-then-Gujarati OCR LoRA

Browse files
Files changed (3) hide show
  1. README.md +91 -0
  2. adapter_config.json +51 -0
  3. adapter_model.safetensors +3 -0
README.md ADDED
@@ -0,0 +1,91 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: bodhan-ai/indic-ocr
3
+ library_name: peft
4
+ pipeline_tag: image-text-to-text
5
+ tags:
6
+ - lora
7
+ - ocr
8
+ - marathi
9
+ - gujarati
10
+ - indicocr
11
+ language:
12
+ - mr
13
+ - gu
14
+ ---
15
+
16
+ # IndicOCR MarGuj
17
+
18
+ LoRA for Marathi and Gujarati line OCR on the recognizer inside [bodhan-ai/indic-ocr](https://huggingface.co/bodhan-ai/indic-ocr) (`weights/ocr`, Qwen3.5-0.8B).
19
+
20
+ The same adapter was trained in two steps. It was first trained on the Marathi real-10k set, then continued for one epoch on 9,200 Gujarati gazette lines. The Gujarati run did not start a new LoRA. Vision weights stay frozen. Base weights are not in this repo.
21
+
22
+ ## Training
23
+
24
+ | | Marathi step | Gujarati step |
25
+ |---|---|---|
26
+ | Starting weights | `bodhan-ai/indic-ocr` `weights/ocr` | this adapter, after the Marathi step |
27
+ | Lines | 9,200 real Marathi lines | 9,200 real Gujarati lines |
28
+ | Optimizer steps | 1,150 | 1,150 |
29
+ | Batch | 1, gradient accumulation 8 | 1, gradient accumulation 8 |
30
+ | Learning rate | 5e-5 | 1e-4 |
31
+ | LoRA | r=16, alpha=32, language-model projections only | same adapter, continued |
32
+
33
+ Prompt: `Transcribe the text in this image. Write any mathematical expressions in LaTeX, using $...$ for inline math and $$...$$ for display equations.`
34
+
35
+ Images are area-clamped between 256 and 1536 pixels on a side. Generation uses tokenizer EOS id 262146.
36
+
37
+ ## Gujarati frozen 100
38
+
39
+ Same 100 long lines (`special_chars_expansion.jsonl`) and the same Gujarati metric definitions. PARSeq runs are piecewise. Chitrapathak and IndicOCR runs are whole-line.
40
+
41
+ | Metric | PARSeq base | PARSeq trained | Chitrapathak base | Chitrapathak MarGuj | IndicOCR base | IndicOCR MarGuj |
42
+ |---|---:|---:|---:|---:|---:|---:|
43
+ | CER ↓ | 31.57% | 27.45% | 80.35% | 8.81% | 1.76% | **1.23%** |
44
+ | Strict WER ↓ | 97.30% | 99.44% | 93.10% | 32.25% | 8.57% | **7.36%** |
45
+ | Punctuation F1 ↑ | 0.00% | 36.66% | 90.44% | 97.30% | 97.30% | **98.60%** |
46
+ | Matra ↑ | 84.01% | 91.87% | 0.00% | 92.10% | **98.53%** | **98.53%** |
47
+ | Conjunct ↑ | 78.70% | 89.51% | 0.00% | 64.20% | 97.22% | **97.53%** |
48
+ | Numeral ↑ | 54.60% | 82.18% | 0.00% | 92.53% | **96.55%** | 95.98% |
49
+ | Character accuracy ↑ | 68.43% | 72.55% | 19.65% | 91.19% | 98.24% | **98.77%** |
50
+ | Exact match ↑ | 0/100 | 0/100 | 0/100 | 5/100 | 43/100 | **46/100** |
51
+
52
+ ## Gujarati 800-line holdout
53
+
54
+ Short gazette lines, at most 10 words. This is the split held out from Gujarati training. IndicOCR MarGuj: CER 0.014%, exact match 797/800, vowel signs 100%, conjuncts 100%, numerals 100%, punctuation F1 99.98%.
55
+
56
+ ## Use
57
+
58
+ ```python
59
+ import torch
60
+ from peft import PeftModel
61
+ from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
62
+
63
+ base_id = "bodhan-ai/indic-ocr"
64
+ adapter_id = "snxtyle/IndicOCR-MarGuj"
65
+ device = "cuda" if torch.cuda.is_available() else "mps"
66
+ dtype = torch.bfloat16
67
+
68
+ model = Qwen3_5ForConditionalGeneration.from_pretrained(
69
+ base_id, subfolder="weights/ocr", torch_dtype=dtype, attn_implementation="sdpa"
70
+ )
71
+ model = PeftModel.from_pretrained(model, adapter_id).to(device).eval()
72
+ processor = AutoProcessor.from_pretrained(base_id, subfolder="weights/ocr")
73
+
74
+ prompt = (
75
+ "Transcribe the text in this image. Write any mathematical expressions in LaTeX, "
76
+ "using $...$ for inline math and $$...$$ for display equations."
77
+ )
78
+ messages = [{
79
+ "role": "user",
80
+ "content": [
81
+ {"type": "image", "image": image},
82
+ {"type": "text", "text": prompt},
83
+ ],
84
+ }]
85
+ text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
86
+ inputs = processor(text=[text], images=[image], padding=True, return_tensors="pt").to(device)
87
+ eos = 262146
88
+ with torch.no_grad():
89
+ out = model.generate(**inputs, max_new_tokens=512, do_sample=False, eos_token_id=eos)
90
+ print(processor.batch_decode(out[:, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)[0].strip())
91
+ ```
adapter_config.json ADDED
@@ -0,0 +1,51 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "bodhan-ai/indic-ocr",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "kasa_config": null,
16
+ "layer_replication": null,
17
+ "layers_pattern": null,
18
+ "layers_to_transform": null,
19
+ "loftq_config": {},
20
+ "lora_alpha": 32,
21
+ "lora_bias": false,
22
+ "lora_dropout": 0.05,
23
+ "lora_ga_config": null,
24
+ "megatron_config": null,
25
+ "megatron_core": "megatron.core",
26
+ "modules_to_save": null,
27
+ "monteclora_config": null,
28
+ "peft_type": "LORA",
29
+ "peft_version": "0.21.0",
30
+ "qalora_group_size": 16,
31
+ "r": 16,
32
+ "rank_pattern": {},
33
+ "revision": null,
34
+ "target_modules": [
35
+ "v_proj",
36
+ "o_proj",
37
+ "q_proj",
38
+ "up_proj",
39
+ "down_proj",
40
+ "gate_proj",
41
+ "k_proj"
42
+ ],
43
+ "target_parameters": null,
44
+ "task_type": "CAUSAL_LM",
45
+ "trainable_token_indices": null,
46
+ "use_bdlora": null,
47
+ "use_dora": false,
48
+ "use_qalora": false,
49
+ "use_rslora": false,
50
+ "velora_config": null
51
+ }
adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7145a7ad819e8d181c977204439cea1a2e6b445191d0c34057bd11071a503c26
3
+ size 25587104