Instructions to use snxtyle/IndicOCR-MarGuj with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use snxtyle/IndicOCR-MarGuj with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("bodhan-ai/indic-ocr") model = PeftModel.from_pretrained(base_model, "snxtyle/IndicOCR-MarGuj") - Notebooks
- Google Colab
- Kaggle
Publish the Marathi-then-Gujarati OCR LoRA
Browse files- README.md +91 -0
- adapter_config.json +51 -0
- adapter_model.safetensors +3 -0
README.md
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
base_model: bodhan-ai/indic-ocr
|
| 3 |
+
library_name: peft
|
| 4 |
+
pipeline_tag: image-text-to-text
|
| 5 |
+
tags:
|
| 6 |
+
- lora
|
| 7 |
+
- ocr
|
| 8 |
+
- marathi
|
| 9 |
+
- gujarati
|
| 10 |
+
- indicocr
|
| 11 |
+
language:
|
| 12 |
+
- mr
|
| 13 |
+
- gu
|
| 14 |
+
---
|
| 15 |
+
|
| 16 |
+
# IndicOCR MarGuj
|
| 17 |
+
|
| 18 |
+
LoRA for Marathi and Gujarati line OCR on the recognizer inside [bodhan-ai/indic-ocr](https://huggingface.co/bodhan-ai/indic-ocr) (`weights/ocr`, Qwen3.5-0.8B).
|
| 19 |
+
|
| 20 |
+
The same adapter was trained in two steps. It was first trained on the Marathi real-10k set, then continued for one epoch on 9,200 Gujarati gazette lines. The Gujarati run did not start a new LoRA. Vision weights stay frozen. Base weights are not in this repo.
|
| 21 |
+
|
| 22 |
+
## Training
|
| 23 |
+
|
| 24 |
+
| | Marathi step | Gujarati step |
|
| 25 |
+
|---|---|---|
|
| 26 |
+
| Starting weights | `bodhan-ai/indic-ocr` `weights/ocr` | this adapter, after the Marathi step |
|
| 27 |
+
| Lines | 9,200 real Marathi lines | 9,200 real Gujarati lines |
|
| 28 |
+
| Optimizer steps | 1,150 | 1,150 |
|
| 29 |
+
| Batch | 1, gradient accumulation 8 | 1, gradient accumulation 8 |
|
| 30 |
+
| Learning rate | 5e-5 | 1e-4 |
|
| 31 |
+
| LoRA | r=16, alpha=32, language-model projections only | same adapter, continued |
|
| 32 |
+
|
| 33 |
+
Prompt: `Transcribe the text in this image. Write any mathematical expressions in LaTeX, using $...$ for inline math and $$...$$ for display equations.`
|
| 34 |
+
|
| 35 |
+
Images are area-clamped between 256 and 1536 pixels on a side. Generation uses tokenizer EOS id 262146.
|
| 36 |
+
|
| 37 |
+
## Gujarati frozen 100
|
| 38 |
+
|
| 39 |
+
Same 100 long lines (`special_chars_expansion.jsonl`) and the same Gujarati metric definitions. PARSeq runs are piecewise. Chitrapathak and IndicOCR runs are whole-line.
|
| 40 |
+
|
| 41 |
+
| Metric | PARSeq base | PARSeq trained | Chitrapathak base | Chitrapathak MarGuj | IndicOCR base | IndicOCR MarGuj |
|
| 42 |
+
|---|---:|---:|---:|---:|---:|---:|
|
| 43 |
+
| CER ↓ | 31.57% | 27.45% | 80.35% | 8.81% | 1.76% | **1.23%** |
|
| 44 |
+
| Strict WER ↓ | 97.30% | 99.44% | 93.10% | 32.25% | 8.57% | **7.36%** |
|
| 45 |
+
| Punctuation F1 ↑ | 0.00% | 36.66% | 90.44% | 97.30% | 97.30% | **98.60%** |
|
| 46 |
+
| Matra ↑ | 84.01% | 91.87% | 0.00% | 92.10% | **98.53%** | **98.53%** |
|
| 47 |
+
| Conjunct ↑ | 78.70% | 89.51% | 0.00% | 64.20% | 97.22% | **97.53%** |
|
| 48 |
+
| Numeral ↑ | 54.60% | 82.18% | 0.00% | 92.53% | **96.55%** | 95.98% |
|
| 49 |
+
| Character accuracy ↑ | 68.43% | 72.55% | 19.65% | 91.19% | 98.24% | **98.77%** |
|
| 50 |
+
| Exact match ↑ | 0/100 | 0/100 | 0/100 | 5/100 | 43/100 | **46/100** |
|
| 51 |
+
|
| 52 |
+
## Gujarati 800-line holdout
|
| 53 |
+
|
| 54 |
+
Short gazette lines, at most 10 words. This is the split held out from Gujarati training. IndicOCR MarGuj: CER 0.014%, exact match 797/800, vowel signs 100%, conjuncts 100%, numerals 100%, punctuation F1 99.98%.
|
| 55 |
+
|
| 56 |
+
## Use
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
import torch
|
| 60 |
+
from peft import PeftModel
|
| 61 |
+
from transformers import AutoProcessor, Qwen3_5ForConditionalGeneration
|
| 62 |
+
|
| 63 |
+
base_id = "bodhan-ai/indic-ocr"
|
| 64 |
+
adapter_id = "snxtyle/IndicOCR-MarGuj"
|
| 65 |
+
device = "cuda" if torch.cuda.is_available() else "mps"
|
| 66 |
+
dtype = torch.bfloat16
|
| 67 |
+
|
| 68 |
+
model = Qwen3_5ForConditionalGeneration.from_pretrained(
|
| 69 |
+
base_id, subfolder="weights/ocr", torch_dtype=dtype, attn_implementation="sdpa"
|
| 70 |
+
)
|
| 71 |
+
model = PeftModel.from_pretrained(model, adapter_id).to(device).eval()
|
| 72 |
+
processor = AutoProcessor.from_pretrained(base_id, subfolder="weights/ocr")
|
| 73 |
+
|
| 74 |
+
prompt = (
|
| 75 |
+
"Transcribe the text in this image. Write any mathematical expressions in LaTeX, "
|
| 76 |
+
"using $...$ for inline math and $$...$$ for display equations."
|
| 77 |
+
)
|
| 78 |
+
messages = [{
|
| 79 |
+
"role": "user",
|
| 80 |
+
"content": [
|
| 81 |
+
{"type": "image", "image": image},
|
| 82 |
+
{"type": "text", "text": prompt},
|
| 83 |
+
],
|
| 84 |
+
}]
|
| 85 |
+
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
| 86 |
+
inputs = processor(text=[text], images=[image], padding=True, return_tensors="pt").to(device)
|
| 87 |
+
eos = 262146
|
| 88 |
+
with torch.no_grad():
|
| 89 |
+
out = model.generate(**inputs, max_new_tokens=512, do_sample=False, eos_token_id=eos)
|
| 90 |
+
print(processor.batch_decode(out[:, inputs["input_ids"].shape[-1]:], skip_special_tokens=True)[0].strip())
|
| 91 |
+
```
|
adapter_config.json
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alora_invocation_tokens": null,
|
| 3 |
+
"alpha_pattern": {},
|
| 4 |
+
"arrow_config": null,
|
| 5 |
+
"auto_mapping": null,
|
| 6 |
+
"base_model_name_or_path": "bodhan-ai/indic-ocr",
|
| 7 |
+
"bias": "none",
|
| 8 |
+
"corda_config": null,
|
| 9 |
+
"ensure_weight_tying": false,
|
| 10 |
+
"eva_config": null,
|
| 11 |
+
"exclude_modules": null,
|
| 12 |
+
"fan_in_fan_out": false,
|
| 13 |
+
"inference_mode": true,
|
| 14 |
+
"init_lora_weights": true,
|
| 15 |
+
"kasa_config": null,
|
| 16 |
+
"layer_replication": null,
|
| 17 |
+
"layers_pattern": null,
|
| 18 |
+
"layers_to_transform": null,
|
| 19 |
+
"loftq_config": {},
|
| 20 |
+
"lora_alpha": 32,
|
| 21 |
+
"lora_bias": false,
|
| 22 |
+
"lora_dropout": 0.05,
|
| 23 |
+
"lora_ga_config": null,
|
| 24 |
+
"megatron_config": null,
|
| 25 |
+
"megatron_core": "megatron.core",
|
| 26 |
+
"modules_to_save": null,
|
| 27 |
+
"monteclora_config": null,
|
| 28 |
+
"peft_type": "LORA",
|
| 29 |
+
"peft_version": "0.21.0",
|
| 30 |
+
"qalora_group_size": 16,
|
| 31 |
+
"r": 16,
|
| 32 |
+
"rank_pattern": {},
|
| 33 |
+
"revision": null,
|
| 34 |
+
"target_modules": [
|
| 35 |
+
"v_proj",
|
| 36 |
+
"o_proj",
|
| 37 |
+
"q_proj",
|
| 38 |
+
"up_proj",
|
| 39 |
+
"down_proj",
|
| 40 |
+
"gate_proj",
|
| 41 |
+
"k_proj"
|
| 42 |
+
],
|
| 43 |
+
"target_parameters": null,
|
| 44 |
+
"task_type": "CAUSAL_LM",
|
| 45 |
+
"trainable_token_indices": null,
|
| 46 |
+
"use_bdlora": null,
|
| 47 |
+
"use_dora": false,
|
| 48 |
+
"use_qalora": false,
|
| 49 |
+
"use_rslora": false,
|
| 50 |
+
"velora_config": null
|
| 51 |
+
}
|
adapter_model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7145a7ad819e8d181c977204439cea1a2e6b445191d0c34057bd11071a503c26
|
| 3 |
+
size 25587104
|