Rakshya server (रक्षा)

The large model of Rakshya: personal data found in Nepali and English text, for servers. By Fiste AI. It has the rules, the 21 labels and the code of the small model, with 278 million weights in place of 40 million. Use the small one in a web page or on a phone, and this one where 1.1 GB and a few tens of milliseconds a sentence are no trouble.

Benchmark

882 real texts in seven test sets that no system had seen: news, web pages, government pages and comments. The number is the F1 of exact spans over every kind of personal data in a set. Higher is better.

System Weights News (ne) Web (ne) Government (ne) Comments (ne, Latin) News (en) Web (en) News (hi) Mean
Rakshya 0.2 40M 0.878 0.802 0.731 0.529 0.803 0.703 0.804 0.750
Rakshya 0.2 server 278M 0.870 0.815 0.740 0.520 0.813 0.740 0.800 0.757
Gopya 0.1 34M 0.855 0.683 0.636 0.233 0.729 0.563 0.725 0.632
EU PII anonymisation (bards.ai) 278M 0.647 0.601 0.409 0.253 0.787 0.530 0.727 0.565
XLM-R base NER HRL (Davlan) 277M 0.660 0.617 0.267 0.302 0.772 0.498 0.712 0.547
XLM-R large, CoNLL-2003 (FacebookAI) 559M 0.651 0.574 0.200 0.265 0.783 0.481 0.760 0.531
EU PII anonymisation v3 (bards.ai) 308M 0.144 0.704 0.328 0.162 0.605 0.709 0.528 0.454
Nym PII multilingual (Wismut) 308M 0.427 0.420 0.301 0.252 0.655 0.478 0.595 0.447

None of the 25 published systems we ran scores higher than Rakshya 0.2 server on any of the seven sets.

All 25 systems, the scores by kind, the false alarms and the method are in the full benchmark.

Use it

# pip install git+https://github.com/fiste-ai/rakshya
from rakshya import Rakshya

pii = Rakshya(model="server")     # fetches this model once, 1.1 GB
pii.redact("मेरो नाम सीता थापा हो, फोन ९८४१२३४५६७।")

From a shell: rakshya --server "some text". On a processor it reads a sentence of about 21 tokens in 40 ms on one thread and 15 ms on four (a laptop processor, Intel Core i5-13420H). With onnxruntime-gpu installed, Rakshya(model="server", providers=["CUDAExecutionProvider"]) runs it on a graphics card.

The repository also loads as a plain token classifier in transformers. Read that way the model gives its tags alone: the rules and the cut at the case marker are in the package.

The scores above were measured with these weights on a graphics card. The ONNX file here holds the same weights and gives the same tag on 10,259 of 10,259 tokens of a check.

Limits

The limits of the small model hold here too: it is a helper and not a guarantee, Nepali in Latin letters is the hardest, Hindi is experimental, and identity numbers are tested mostly on made-up text. See the card of the small model.

Weights CC BY-NC 4.0, code MIT; see LICENSE.md and NOTICE.md. Part of the training data is for non-commercial use, so there is no commercial licence for this version. Questions: contact us.

Downloads last month
24
Safetensors
Model size
0.3B params
Tensor type
F32
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for fiste-ai/rakshya-server

Quantized
(54)
this model