NIC-Coder-0.6B / README.md
sakasurya's picture
Access note: verified organisation email
0a62d5a verified
|
Raw History Blame Contribute Delete
13 kB
---
license: cc-by-nc-sa-4.0
base_model: Qwen/Qwen3-Embedding-0.6B
language:
- en
tags:
- text-classification
- nic-2008
- india
- msme
- udyam
- industry-classification
library_name: transformers
pipeline_tag: text-classification
extra_gated_prompt: 'This model is released under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 licence (CC
BY-NC-SA 4.0): research, academic, personal and other non-commercial use is permitted with attribution; adaptations must
be shared under the same licence; commercial use is not permitted. The original public source data remains under the Government
Open Data License - India (GODL-India). Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai.'
extra_gated_fields:
Name: text
Email: text
Organization: text
I accept the CC BY-NC-SA licence terms (research and non-commercial use only): checkbox
extra_gated_button_content: Accept and request access
---
# NIC-Coder-0.6B
<!-- jm:top -->
Licence: CC BY-NC-SA 4.0 — research and non-commercial use only; derivatives must be shared under the same licence. Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai
Access is approved automatically when the email you enter matches your Hugging Face account email and belongs to the organisation you name; other requests are reviewed manually.
<!-- /jm:top -->
NIC-Coder-0.6B assigns an Indian business name, optionally with a short description of what the business does, to a **NIC-2008** (National Industrial Classification 2008) code. It returns ranked 5-digit sub-classes with their official titles, plus 4-digit class, 3-digit group and 2-digit division predictions that are always consistent with the 5-digit ranking. It covers 1,177 NIC-2008 sub-classes in 82 divisions and was fine-tuned from `Qwen/Qwen3-Embedding-0.6B` on UDYAM MSME registrations.
From the name alone, on business names it never saw, it gets the exact 5-digit code right **16.4%** of the time, against **7.2%** for a TF-IDF + linear classifier trained on the same data and **5.7%** for zero-shot matching of names to NIC titles with the base embedding model. In three states held out of training altogether (Kerala, Odisha, Uttarakhand) the gap holds: **13.8%** vs 5.7% and 5.1%.
**Name only, unseen names: accuracy (%)**
| Test set | NIC-Coder-0.6B | TF-IDF + linear | Zero-shot embedding |
|---|---|---|---|
| Random states, NIC5 top-1 | **16.4** | 7.2 | 5.7 |
| Random states, NIC5 top-5 | **38.1** | 17.1 | 14.5 |
| Random states, NIC2 top-1 | **29.5** | 18.6 | 15.4 |
| Held-out states, NIC5 top-1 | **13.8** | 5.7 | 5.1 |
| Held-out states, NIC5 top-5 | **33.4** | 13.8 | 12.9 |
| Held-out states, NIC2 top-1 | **27.2** | 16.1 | 14.6 |
## Quickstart
```bash
pip install -r requirements.txt # or: torch transformers safetensors huggingface_hub
```
```python
from huggingface_hub import snapshot_download
import sys
path = snapshot_download("judicialmind/NIC-Coder-0.6B")
sys.path.insert(0, path)
from model import NICCoder
m = NICCoder.load(path) # GPU if available, else CPU
res = m.predict(
["Kaveri Rice Mill", "Sai Krupa Medical Stores", "Kavya Tailoring Works"],
activities=[None, None, "custom tailoring"],
k=3,
)
for r in res:
print(r["nic5"][0], r["nic2"][0])
```
**Top prediction for the three made-up names above (CPU run)**
| Input | NIC5 | Title | Probability | NIC2 | Probability |
|---|---|---|---|---|---|
| Kaveri Rice Mill | 10612 | Rice milling | 0.545 | 10 | 0.749 |
| Sai Krupa Medical Stores | 47721 | Retail sale of pharmaceuticals, medical and orthopaedic goods and toilet articles | 0.434 | 47 | 0.479 |
| Kavya Tailoring Works + "custom tailoring" | 14105 | Custom tailoring | 0.996 | 14 | 0.996 |
`predict(names, activities=None, k=5, adjust=True)` takes a list of business names and an optional list of activity descriptions (same length, `None` where there is none). It returns one dict per input:
| Key | Content |
|---|---|
| `nic5` | top-k `(code, NIC-2008 title, probability)` for 5-digit sub-classes |
| `nic4`, `nic3`, `nic2` | top-k `(code, probability)` for the 4-digit class, 3-digit group and 2-digit division |
Command line (one JSON line per input; reads `name` or `name<TAB>activity` lines from stdin when no name is given):
```bash
python predict.py "Kaveri Rice Mill" -k 3
printf 'Kavya Tailoring Works\tcustom tailoring\n' | python predict.py -k 3
```
### Input format
The model sees `business: <normalised name>` or `business: <normalised name> | activity: <activity text>`. Normalisation is done for you (`text.normalize_name`): lower-casing, `&` to `and`, punctuation to spaces, a leading `M/s` or `Messrs` removed, and trailing legal suffixes (Pvt, Private, Ltd, Limited, LLP, OPC) removed. Activity text is lower-cased. Inputs are truncated at 64 tokens.
### Prior adjustment
`adjust=True` (default) adds a log-prior correction to the scores so predictions follow how often each code occurs among UDYAM registrations. This maximises accuracy on registrations drawn the way UDYAM is. `adjust=False` (CLI `--balanced`) gives class-balanced scores: lower overall accuracy, less pull toward the most common codes, useful when you rank rare industries or apply your own prior.
### Label hierarchy
NIC-2008 codes are nested by prefix: `10501` (sub-class) is in class `1050`, group `105` and division `10` (Manufacture of food products). The 4/3/2-digit probabilities are sums of the 5-digit probabilities under each prefix. `label_map.json` holds the 1,177 sub-class codes and their official titles.
## Files
| File | Content |
|---|---|
| `backbone/` | fine-tuned Qwen3-Embedding-0.6B weights and tokenizer (Transformers format) |
| `head.safetensors` | classifier weights over the 1,177 NIC5 sub-classes, logit scale and prior correction |
| `label_map.json` | NIC5 codes (head order), NIC-2008 titles, maximum input length |
| `model.py` | `NICCoder.load(path_or_repo_id)` and `NICCoder.predict(...)` |
| `text.py` | input normalisation and model-input formatting |
| `predict.py` | command-line interface |
| `metrics.json` | all evaluation results (NIC-Coder, zero-shot embedding, baselines; per state) |
| `label_stats.json` | aggregate registration counts by NIC5 code, division and state |
| `requirements.txt` | pinned runtime dependencies |
## Results
Test names are held out: no normalised business name in the test sets appears in training. Each test set keeps the natural mix of codes. *Random states*: 50,000 registrations from the states used in training. *Held-out states*: 46,712 registrations from Kerala, Odisha and Uttarakhand, none of which were used in training. Accuracy is top-1 / top-5 at each NIC level. Macro-F1 is NIC5 top-1 averaged over the codes present in the test set. *Any-of* counts a top-1 NIC5 that matches any of the activities the enterprise listed. For scale, the single most common sub-class (96098, household maintenance services) accounts for 4.0% of registrations.
**Random states, unseen names, name only (%)**
| Method | NIC5 top-1 | NIC5 top-5 | NIC4 top-1 | NIC2 top-1 | NIC2 top-5 | NIC5 macro-F1 | NIC5 any-of top-1 |
|---|---|---|---|---|---|---|---|
| **NIC-Coder-0.6B** (default) | 16.4 | 38.1 | 20.0 | 29.5 | 65.0 | 7.3 | 18.4 |
| NIC-Coder-0.6B, `adjust=False` | 9.7 | 25.1 | 13.8 | 23.6 | 55.3 | 6.6 | 11.4 |
| TF-IDF (char + word) + linear | 7.2 | 17.1 | 10.6 | 18.6 | 45.0 | 4.6 | 8.5 |
| Nearest NIC title, TF-IDF | 5.1 | 11.9 | 7.3 | 12.4 | 30.8 | 2.9 | 5.9 |
| Nearest NIC title, Qwen3-Embedding-0.6B zero-shot | 5.7 | 14.5 | 8.3 | 15.4 | 39.4 | 3.5 | 6.6 |
**Held-out states, unseen names, name only (%)**
| Method | NIC5 top-1 | NIC5 top-5 | NIC4 top-1 | NIC2 top-1 | NIC2 top-5 | NIC5 macro-F1 | NIC5 any-of top-1 |
|---|---|---|---|---|---|---|---|
| **NIC-Coder-0.6B** (default) | 13.8 | 33.4 | 16.7 | 27.2 | 62.0 | 5.8 | 15.3 |
| NIC-Coder-0.6B, `adjust=False` | 8.0 | 20.1 | 11.3 | 21.5 | 50.0 | 5.1 | 9.1 |
| TF-IDF (char + word) + linear | 5.7 | 13.8 | 8.5 | 16.1 | 40.2 | 3.5 | 6.5 |
| Nearest NIC title, TF-IDF | 4.5 | 10.2 | 6.5 | 11.8 | 28.8 | 2.5 | 5.2 |
| Nearest NIC title, Qwen3-Embedding-0.6B zero-shot | 5.1 | 12.9 | 7.5 | 14.6 | 37.6 | 3.5 | 5.7 |
**Held-out states, name only, by state (%)**
| State | Test registrations | NIC5 top-1 | NIC2 top-1 |
|---|---|---|---|
| Kerala | 17,203 | 15.2 | 30.9 |
| Odisha | 22,645 | 11.5 | 22.7 |
| Uttarakhand | 6,864 | 18.3 | 32.5 |
### Name + activity text
In these tests the activity text was a lower-cased span of 40–100% of the words of the correct code's official NIC-2008 title, so they measure how well a model maps a near-verbatim NIC description to its code, not free-form business descriptions.
**Random states, unseen names, name + activity text (%)**
| Method | NIC5 top-1 | NIC5 top-5 | NIC4 top-1 | NIC2 top-1 | NIC2 top-5 | NIC5 macro-F1 | NIC5 any-of top-1 |
|---|---|---|---|---|---|---|---|
| **NIC-Coder-0.6B** (default) | 98.4 | 99.8 | 98.8 | 99.2 | 99.9 | 96.6 | 98.5 |
| NIC-Coder-0.6B, `adjust=False` | 97.8 | 99.5 | 98.4 | 98.8 | 99.8 | 95.4 | 97.8 |
| TF-IDF (char + word) + linear | 96.8 | 99.3 | 98.0 | 98.6 | 99.6 | 94.5 | 96.8 |
| Nearest NIC title, TF-IDF | 96.4 | 99.1 | 97.0 | 98.0 | 99.5 | 92.9 | 96.4 |
| Nearest NIC title, Qwen3-Embedding-0.6B zero-shot | 72.9 | 88.5 | 76.0 | 81.4 | 94.7 | 70.6 | 73.3 |
**Held-out states, unseen names, name + activity text (%)**
| Method | NIC5 top-1 | NIC5 top-5 | NIC4 top-1 | NIC2 top-1 | NIC2 top-5 | NIC5 macro-F1 | NIC5 any-of top-1 |
|---|---|---|---|---|---|---|---|
| **NIC-Coder-0.6B** (default) | 98.2 | 99.7 | 98.6 | 99.1 | 99.8 | 95.7 | 98.2 |
| NIC-Coder-0.6B, `adjust=False` | 97.4 | 99.4 | 98.1 | 98.7 | 99.7 | 94.3 | 97.4 |
| TF-IDF (char + word) + linear | 96.6 | 99.2 | 97.8 | 98.4 | 99.6 | 93.9 | 96.6 |
| Nearest NIC title, TF-IDF | 96.1 | 99.0 | 96.8 | 97.9 | 99.5 | 92.4 | 96.2 |
| Nearest NIC title, Qwen3-Embedding-0.6B zero-shot | 73.9 | 88.7 | 76.8 | 82.3 | 94.8 | 69.5 | 74.2 |
With a NIC-title-like description every trained method is near the ceiling; NIC-Coder stays ahead at every level and lifts NIC5 macro-F1 to 96.6 (random states) and 95.7 (held-out states). The name-only setting is where the model's learned knowledge of Indian business names matters: many names ("Shree Ganesh Enterprises") carry no industry signal at all, while others ("... Dairy Farm", "... Auto Spares") do.
## Coverage
- **Source:** 39,648,419 UDYAM MSME registrations from 35 states and union territories (data.gov.in snapshot, May 2026); 21,199,026 distinct normalised business names. Only aggregate counts are included in this repository (`label_stats.json`); there are no registration rows, enterprise names or addresses.
- **Labels:** 1,177 NIC-2008 5-digit sub-classes, 379 classes, 221 groups and 82 divisions. The label is the first activity listed on the registration.
- **Held-out states:** Kerala, Odisha and Uttarakhand were excluded from training and are used only for testing.
## Source data
- Ministry of Micro, Small and Medium Enterprises. "List of MSME Registered Units under UDYAM". data.gov.in resource 8b68ae56-84cf-4728-a0a6-1be11028dea7, https://api.data.gov.in/resource/8b68ae56-84cf-4728-a0a6-1be11028dea7. Snapshot: May 2026.
- National Industrial Classification 2008 (NIC-2008) codes and titles: Central Statistics Office (CSO), Ministry of Statistics and Programme Implementation (MoSPI), Government of India.
<!-- jm:source -->
Original data: published by Ministry of Micro, Small and Medium Enterprises (UDYAM registration) on the Open Government Data Platform India ([data.gov.in](https://data.gov.in)) and, where marked, NITI Aayog's National Data and Analytics Platform ([NDAP](https://ndap.niti.gov.in)), under the [Government Open Data License - India](https://data.gov.in/government-open-data-license-india) (GODL-India). The original public data remains under GODL-India; the CC BY-NC-SA 4.0 licence of this repository covers JudicialMind's compilation, derived fields, annotations, benchmarks and models.
<!-- /jm:source -->
## Citation
```bibtex
@misc{judicialmind2026niccoder,
title={NIC-Coder-0.6B: NIC-2008 industry codes from Indian business names},
author={JudicialMind},
year={2026},
publisher={Hugging Face},
howpublished={\url{https://huggingface.co/judicialmind/NIC-Coder-0.6B}}
}
```
<!-- jm:license -->
## License
[CC BY-NC-SA 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) (see `LICENSE`): research, academic, personal and other non-commercial use, with attribution to JudicialMind and to the original data publishers; adaptations must be shared under the same licence. Commercial use requires a separate licence.
Commercial use: contact surya@judicialmind.ai or hello@judicialmind.ai.
Base model(s): `Qwen/Qwen3-Embedding-0.6B` (apache-2.0). Their licence terms and notices continue to apply to the upstream components; see `THIRD_PARTY_NOTICES.md` and `NOTICE`.
<!-- /jm:license -->