Gopya (गोप्य)

Nepali PII detection that runs in the browser, by Fiste AI. Text in; names, places, phone numbers and identity numbers marked, or taken out. गोप्य means confidential.

• Small. 35 MB. It loads in a web page and reads a sentence in about 30 ms there, so the text can stay on the device. Try it in your browser.
• Exact. Nepali writes case markers against a name, as in सीतालाई. Gopya ends each span inside the word, so it takes out the name and leaves the grammar.
• Made for Nepal. Citizenship, national identity, PAN, passport, licence and voter numbers, bank accounts, number plates, phone numbers, and dates in both calendars and both scripts.

Benchmark

Names of people, places and organisations in 215 sentences of Nepali news that no system had seen. Higher is better.

Model	Weights	Exact	Whole words	Characters found
Gopya 0.1	34M	0.81	0.83	93%
XLM-R base NER HRL (Davlan)	277M	0.70	0.81	89%
EU PII anonymisation (bards.ai)	278M	0.66	0.83	97%
WikiNEural multilingual NER (Babelscape)	177M	0.45	0.56	65%
mBERT Nepali NER (Saugatkafley)	177M	0.34	0.68	77%
Nepali NER RoBERTa (meutsabdahal)	124M	0.32	0.78	83%

Exact: start, end and label all right. Whole words: the same after every span is widened to whole words, which forgives a case marker left on a name. Characters found: the share of the characters of names inside any span found. Over all four labels of this set, dates included, Gopya scores 0.81 exact and finds 94% of the characters.

The labels were written with the help of an AI assistant before any system was run, and are waiting for review by a native speaker, so the scores may move. The second test set, every system and the method are in the full benchmark.

Use it

# pip install onnxruntime tokenizers numpy huggingface_hub
import sys
from huggingface_hub import snapshot_download

folder = snapshot_download("fiste-ai/gopya")
sys.path.insert(0, folder)
from gopya import Gopya

pii = Gopya(folder)
text = "मेरो नाम सीता थापा हो, फोन ९८४१२३४५६७।"
pii.find(text)     # [(9, 18, 'PERSON'), (27, 37, 'PHONE')]
pii.redact(text)   # मेरो नाम [PERSON_1] हो, फोन [PHONE_1]।

In a web page, web/gopya.js runs the same model with ONNX Runtime Web. The demo page is the example.

It finds PERSON, LOCATION, ORGANIZATION, DATE, PHONE, EMAIL, CITIZENSHIP_NO, NATIONAL_ID, PAN, PASSPORT_NO, LICENSE_NO, VOTER_ID, BANK_ACCOUNT and VEHICLE_PLATE.

Limits

• It is a helper and not a guarantee. On news it misses about 6 in 100 characters of personal data. Read the result before you rely on it.
• It learned from news and from made-up forms and messages. On text copied from web pages, where table cells run together, it is weaker: 0.66 exact on 75 snippets from government sites, and for names alone three larger models do better there.
• The identity numbers are tested on made-up text, apart from a few real PAN and bank account numbers. Public pages do not print the others.
• Larger models find a little more. The best of them finds 97% of the characters of names in the news set, against 93% for Gopya, with spans that are less exact and 8 times the weights.

Weights CC BY-NC 4.0, code MIT; see LICENSE.md and NOTICE.md. The model learned from the EverestNER corpus, which is for non-commercial use, so there is no commercial licence for this version. Questions: contact us.