Temperate PII

This is the model of Temperate PII, a service that finds personal data in European text. The model finds person names and street addresses. Rules per country find the structured values, such as IBANs, phone numbers and national ID numbers, and check their checksums.

The model is a 6-layer student, distilled from a larger teacher and built on Multilingual MiniLM. It has 107 million parameters and runs as an int8 ONNX model on a CPU. It was trained on English, Dutch, German, French, Spanish and Italian.

Results

On the benchmark in the GitHub repository, 2,041 texts in the six languages:

Type	Values	Recall	Precision
Person names	205	98.0%	96.2%
Street addresses	275	100.0%	94.2%

On 26 business texts that we wrote by hand, such as emails, invoices and contracts, it found 76 of the 83 names and addresses. We built the benchmark ourselves, so read these numbers with care. The repository explains how we measured, and has the results for every type and a comparison with Azure AI Language.

Use

Run the service from the repository. fetch-model.sh downloads these files and checks each one against its digest:

git clone https://github.com/temperate-works/pii
cd pii
./fetch-model.sh

File	What it holds
model.onnx	The model, int8 with static quantization
tokenizer.json	The tokenizer of FacebookAI/xlm-roberta-base
labels.json	The BIO labels. The service uses only PERSON and STREET_ADDRESS; rules find the other types
calibration.json	The score threshold per type

The service reads texts in windows of 256 tokens that overlap by 32.

A surname on its own is left alone on purpose, when no title, greeting, label or earlier full name points to a person. Masking every word that could be a surname would also mask many company and place names.

Licence

Apache-2.0. The model is built on microsoft/Multilingual-MiniLM-L12-H384 (MIT) and the tokenizer of FacebookAI/xlm-roberta-base (MIT). It was trained on ai4privacy/pii-masking-openpii-1.5m by ai4privacy (CC BY 4.0) and Wismut/nym-pii-multilingual-data (MIT).

Made by Temperate.