Feature Extraction
Transformers
Safetensors
English
Hindi
multilingual
address-parsing
geocoding
entity-resolution
india
lgd
pincode
open-government-data
Instructions to use judicialmind/BharatAddr with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use judicialmind/BharatAddr with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="judicialmind/BharatAddr")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("judicialmind/BharatAddr", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from judicialmind/BharatAddr: direct link, hf CLI and curl.
- Browser
- Download file 13.1 kB
-
https://huggingface.co/judicialmind/BharatAddr/resolve/main/README.md
- Command line
-
hf download hf://judicialmind/BharatAddr/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/judicialmind/BharatAddr/resolve/main/README.md
13.1 kB
| license: cc-by-nc-sa-4.0 | |
| pretty_name: BharatAddr | |
| language: | |
| - en | |
| - hi | |
| - multilingual | |
| library_name: transformers | |
| pipeline_tag: feature-extraction | |
| base_model: | |
| - Qwen/Qwen3-Embedding-0.6B | |
| datasets: | |
| - judicialmind/BharatAddr-Gazetteer | |
| tags: | |
| - address-parsing | |
| - geocoding | |
| - entity-resolution | |
| - india | |
| - lgd | |
| - pincode | |
| - open-government-data | |
| extra_gated_prompt: 'This model is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 licence (CC | |
| BY-NC-SA 4.0): research, academic, personal and other non-commercial use is permitted with attribution; adaptations must | |
| be shared under the same licence; commercial use is not permitted. The original public source data remains under the Government | |
| Open Data License - India (GODL-India). Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai.' | |
| extra_gated_fields: | |
| Name: text | |
| Email: text | |
| Organization: text | |
| I accept the CC BY-NC-SA licence terms (research and non-commercial use only): checkbox | |
| extra_gated_button_content: Accept and request access | |
| # BharatAddr | |
| <!-- jm:top --> | |
| Licence: CC BY-NC-SA 4.0 — research and non-commercial use only; derivatives must be shared under the same licence. Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai | |
| Access is approved automatically when the email you enter matches your Hugging Face account email and belongs to the organisation you name; other requests are reviewed manually. | |
| <!-- /jm:top --> | |
| Messy Indian address text in, official place out: village or town, sub-district, district and state with their | |
| Local Government Directory (LGD) codes, plus a PIN code. BharatAddr reads English, Hindi (Devanagari), other Indic | |
| scripts, Roman transliterations, abbreviations (Vill., PO, Teh., Tq., Mandal, Dist./Distt., Zila, UP/M.P.), typos, | |
| missing levels and wrong PINs. On 63 districts it never saw during training it picks the exact village **95.74%** of | |
| the time, against 86.37% for a strong non-neural matcher, and gets the PIN right 98.08% of the time. | |
| **Held-out districts (`test_district`, 63 districts never seen in training), exact match %, n = 10,000** | |
| | System | Village / town | Sub-district | District | State | PIN | | |
| |---|---|---|---|---|---| | |
| | **BharatAddr** | **95.74** | **98.24** | **99.28** | **99.82** | **98.08** | | |
| | Qwen3-Embedding-0.6B zero-shot + hierarchy reranking | 89.95 | 95.22 | 97.99 | 99.06 | 95.32 | | |
| | Qwen3-Embedding-0.6B zero-shot, retrieval only | 67.47 | 78.11 | 85.36 | 90.97 | 76.53 | | |
| | Non-neural baseline (normalise + fuzzy + hierarchy + PIN) | 86.37 | 92.56 | 96.06 | 97.83 | 92.73 | | |
| Hindi-script addresses: 95.58% village accuracy (baseline 89.69%). | |
| ## Quickstart | |
| The repository is the runtime: a small `bharataddr` package, the gazetteer, the fine-tuned encoder and a | |
| pre-computed index of all 680,293 gazetteer records. Download it, install the requirements and run from the folder. | |
| ```bash | |
| huggingface-cli download judicialmind/BharatAddr --local-dir BharatAddr | |
| cd BharatAddr | |
| pip install -r requirements.txt | |
| ``` | |
| Python: | |
| ```python | |
| from bharataddr import BharatAddr | |
| ba = BharatAddr.load(".") # neural resolver; uses a GPU when available, else CPU | |
| for r in ba.resolve(["H.No 12, Vill Ram Nagar, Teh. Milak, Distt Rampur, UP 244701", | |
| "ग्राम राम नगर, तहसील मिलक, जिला रामपुर"]): | |
| print(r["name"], r["code"], r["subdistrict"], r["district"], r["state"], r["pin"]) | |
| fast = BharatAddr.load(".", neural=False) # baseline only: no model, CPU | |
| print(fast.resolve(["Vill Ram Nagar, Teh. Milak, Distt Rampur, UP 244701"])[0]) | |
| ``` | |
| Run it from inside the downloaded folder. Both addresses resolve to **Ram Nagar** (LGD village 117381), sub-district | |
| Milak, district Rampur, Uttar Pradesh, PIN 244701, and the baseline returns the same village. Loading builds the | |
| gazetteer index in about a minute on CPU; after that an address takes tens of milliseconds on CPU. | |
| Command line (one JSON object per input line; also `--file addresses.txt` or stdin): | |
| ```bash | |
| python -m bharataddr.resolve "H.No 12, Vill Ram Nagar, Teh. Milak, Distt Rampur, UP 244701" | |
| python -m bharataddr.resolve --no-neural "ग्राम राम नगर, तहसील मिलक, जिला रामपुर" | |
| ``` | |
| Input: free-text address strings (any order of components, any of the scripts above). Output, per address: | |
| | Field | Meaning | | |
| |---|---| | |
| | `input` | the address as given (CLI only) | | |
| | `eid` | gazetteer id: `v<LGD village code>` or `u<LGD local-body code>` (towns) | | |
| | `kind` | `village` or `town` | | |
| | `code`, `name` | LGD code and official English name of the village / town | | |
| | `sd_code`, `subdistrict` | LGD sub-district code and name (empty for towns without one) | | |
| | `d_code`, `district` | LGD district code and name | | |
| | `s_code`, `state` | LGD state / UT code and name | | |
| | `pin` | PIN code: the PIN written in the address when it belongs to the chosen place, else the place's LGD PIN | | |
| | `score` | encoder cosine similarity of the chosen record (neural) or hierarchy score (baseline) | | |
| When the text has no usable village evidence the baseline returns `eid`/`code` as null and still fills the best | |
| consistent district, state and sub-district. | |
| ## How it works | |
| 1. **Gazetteer** (`gazetteer/`, also published as | |
| [BharatAddr-Gazetteer](https://huggingface.co/datasets/judicialmind/BharatAddr-Gazetteer)): 675,247 LGD villages | |
| and 5,046 towns with hierarchy, PINs, local-script and Devanagari names, aliases and old names. | |
| 2. **Encoder** (`model/`): Qwen3-Embedding-0.6B fine-tuned to map a messy address close to its canonical record | |
| "village (Devanagari), sub-district, district, state PIN". Queries are prefixed with the instruction stored in | |
| `model/config_sentence_transformers.json`; embeddings use last-token pooling and L2 normalisation. The records are | |
| pre-embedded in `index/doc_emb.f16.npy` (rows aligned with `index/docs.parquet`). | |
| 3. **Resolver** (`bharataddr/resolve.py`): the encoder's top-50 records plus the baseline's top-20 candidates are | |
| ranked by encoder similarity, with the baseline's hierarchy score (village, sub-district, district, state and PIN | |
| evidence found in the text) breaking ties. | |
| 4. **Baseline** (`bharataddr/baseline.py`, `--no-neural`): script-aware romanisation, phonetic and consonant-skeleton | |
| keys, exact + fuzzy n-gram matching at every level, level-marker hints (Dist., Teh., Vill., PO), PIN lookup and a | |
| hierarchy-consistent score per candidate. About 10 ms per address per CPU core. | |
| ## Files | |
| | Path | What | | |
| |---|---| | |
| | `bharataddr/` | resolver package: `resolve.py` (API + CLI), `baseline.py`, `neural.py`, `text.py`, `evaluate.py` | | |
| | `gazetteer/` | `entries.parquet`, `units.parquet`, `pins.parquet`, `stats.json` (same as BharatAddr-Gazetteer) | | |
| | `model/` | fine-tuned encoder (safetensors, tokenizer, sentence-transformers config) | | |
| | `index/` | `docs.parquet` (record id + canonical text) and `doc_emb.f16.npy` (680,293 × 1,024 fp16 embeddings) | | |
| | `metrics.json` | all reported results, with breakdowns by script, PIN condition, place kind and district | | |
| | `eval/` | BharatAddr predictions for `test_district` and `test_random` | | |
| | `requirements.txt` | pinned runtime dependencies | | |
| ## Results | |
| Benchmark: the synthetic messy-address splits of | |
| [BharatAddr-Gazetteer](https://huggingface.co/datasets/judicialmind/BharatAddr-Gazetteer) (`synthetic/`). Exact match | |
| of the LGD code at each level; PIN counts as correct when it is one of the PINs LGD lists for the gold village. | |
| **Random held-out villages and towns (`test_random`, districts seen in training), exact match %, n = 10,000** | |
| | System | Village / town | Sub-district | District | State | PIN | | |
| |---|---|---|---|---|---| | |
| | **BharatAddr** | **95.27** | **97.73** | **99.17** | **99.66** | **97.57** | | |
| | Qwen3-Embedding-0.6B zero-shot + hierarchy reranking | 90.25 | 95.39 | 98.03 | 99.07 | 95.49 | | |
| | Qwen3-Embedding-0.6B zero-shot, retrieval only | 65.65 | 75.97 | 83.35 | 89.30 | 74.68 | | |
| | Non-neural baseline | 87.15 | 93.10 | 96.40 | 98.10 | 93.07 | | |
| **Village accuracy on held-out districts by input script (`test_district`)** | |
| | Script of the address | Addresses | BharatAddr | Baseline | | |
| |---|---|---|---| | |
| | English / Roman | 5,537 | 96.24 | 85.19 | | |
| | Hindi (Devanagari) | 2,240 | 95.58 | 89.69 | | |
| | Mixed Roman + Devanagari | 1,230 | 95.12 | 86.34 | | |
| | Romanised from Devanagari / local script | 981 | 93.99 | 85.42 | | |
| | Other Indic script | 12 | 100.00 | 91.67 | | |
| **Village accuracy on held-out districts by PIN condition (`test_district`)** | |
| | PIN in the address | Addresses | BharatAddr | Baseline | | |
| |---|---|---|---| | |
| | Correct PIN present | 5,497 | 97.18 | 92.36 | | |
| | No PIN | 3,884 | 93.98 | 79.84 | | |
| | Wrong PIN | 619 | 94.02 | 74.15 | | |
| **Encoder retrieval recall on held-out districts (`test_district`), %** | |
| | Encoder | R@1 | R@10 | R@50 | | |
| |---|---|---|---| | |
| | **BharatAddr** | **95.74** | **99.77** | **99.93** | | |
| | Qwen3-Embedding-0.6B zero-shot | 67.47 | 87.85 | 93.46 | | |
| Macro village accuracy over the 63 held-out districts: 96.36 (baseline 86.79). | |
| Reproduce the tables from the shipped predictions and the benchmark files: | |
| ```bash | |
| huggingface-cli download judicialmind/BharatAddr-Gazetteer --repo-type dataset --local-dir BharatAddr-Gazetteer | |
| python -m bharataddr.evaluate --gold BharatAddr-Gazetteer/synthetic/test_district.parquet \ | |
| --pred eval/pred_bharataddr_test_district.parquet | |
| python -m bharataddr.evaluate --gold BharatAddr-Gazetteer/synthetic/test_district.parquet --run --limit 500 | |
| ``` | |
| The first command scores the shipped predictions (add `--json` for the breakdowns in `metrics.json`); the second | |
| runs the resolver end to end on the first 500 addresses (add `--no-neural` for the baseline). To score your own | |
| system, write a parquet with columns `qid`, `eid`, `sd_code`, `d_code`, `s_code`, `pin` and pass it as `--pred`. | |
| ## Coverage | |
| All of India: 35 states and union territories, 793 districts, 7,295 | |
| sub-districts, 675,247 villages and 5,046 towns, 17,995 distinct PINs. Places are resolved to the village / | |
| town level; street, house and landmark details are ignored. Benchmark addresses are synthetic and built from the same | |
| official directories; no personal data is used anywhere. | |
| ## Source data | |
| Contains data from the Local Government Directory (LGD), Ministry of Panchayati Raj, and the Locality based Pincode directory, Department of Posts, Government of India, published on data.gov.in under the Government Open Data License - India (GODL-India), https://data.gov.in/government-open-data-license-india . | |
| | Source (data.gov.in resource id) | Publisher | Used for | | |
| |---|---|---| | |
| | LGD - Villages (`c967fe8f-69c4-42df-8afc-8a2c98057437`) | Ministry of Panchayati Raj | village / sub-district / district / state codes, English + local-script names | | |
| | LGD - Villages with PIN Codes (`f17a1608-5f10-4610-bb50-a63c80d83974`) | Ministry of Panchayati Raj | village -> PIN | | |
| | LGD - Local Bodies (`1a6c26ed-d67c-40ea-aa20-d38d35f341a5`) | Ministry of Panchayati Raj | gram-panchayat names, urban local bodies (towns) | | |
| | Locality based Pincode, 15 Feb 2016 (`7eca2fa3-d6f5-444e-b3d6-faa441e35294`) | Department of Posts | post-office names per PIN, DoP village spellings (locality_detail columns not used) | | |
| | District crosswalk (derived from the portals above) | - | observed district spellings | | |
| Model weights derive from [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B). | |
| <!-- jm:source --> | |
| Original data: published by Ministry of Panchayati Raj (Local Government Directory) and Department of Posts, Ministry of Communications (Locality based Pincode directory) on the Open Government Data Platform India ([data.gov.in](https://data.gov.in)) and, where marked, NITI Aayog's National Data and Analytics Platform ([NDAP](https://ndap.niti.gov.in)), under the [Government Open Data License - India](https://data.gov.in/government-open-data-license-india) (GODL-India). The original public data remains under GODL-India; the CC BY-NC-SA 4.0 licence of this repository covers JudicialMind's compilation, derived fields, annotations, benchmarks and models. | |
| <!-- /jm:source --> | |
| ## Citation | |
| ```bibtex | |
| @misc{judicialmind2026bharataddr, | |
| title={BharatAddr: Resolving Messy Indian Addresses to LGD Villages, Districts and PIN Codes}, | |
| author={JudicialMind}, | |
| year={2026}, | |
| publisher={Hugging Face}, | |
| howpublished={\url{https://huggingface.co/judicialmind/BharatAddr}} | |
| } | |
| ``` | |
| <!-- jm:license --> | |
| ## License | |
| [CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) (see `LICENSE`): research, academic, personal and other non-commercial use, with attribution to JudicialMind and to the original data publishers; adaptations must be shared under the same licence. Commercial use requires a separate licence. | |
| Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai. | |
| Base model(s): `Qwen/Qwen3-Embedding-0.6B` (apache-2.0). Their licence terms and notices continue to apply to the upstream components; see `THIRD_PARTY_NOTICES.md`. | |
| <!-- /jm:license --> | |