DARE / README.md
arlindkadra's picture
Update README.md
af61198 verified
|
Raw History Blame Contribute Delete
8.38 kB
---
license: cc-by-nc-4.0
library_name: transformers
pipeline_tag: token-classification
base_model: jhu-clsp/mmBERT-base
extra_gated_heading: "Request access to DARE"
extra_gated_prompt: >-
DARE is released under the CC BY-NC 4.0 licence. By requesting access, you
agree to use the model for non-commercial purposes only, to give appropriate
credit, and not to use it for surveillance, profiling or re-identification
of individuals. For commercial licensing, please contact support@dataanonymiser.com.
extra_gated_fields:
Affiliation: text
Intended use:
type: select
options:
- Research
- Education
- Personal non-commercial project
- label: Other
value: other
I agree to use DARE for non-commercial purposes only, under the CC BY-NC licence: checkbox
I will not use DARE for surveillance, profiling or re-identification of individuals: checkbox
extra_gated_button_content: "Agree and request access"
language:
- bn
- de
- en
- es
- fa
- fr
- hi
- it
- nl
- pt
- sv
- uk
- zh
tags:
- pii
- pii-detection
- pii-redaction
- pii-masking
- personal-data
- sensitive-data
- privacy
- data-privacy
- data-protection
- anonymization
- anonymisation
- de-identification
- redaction
- data-masking
- gdpr
- ner
- named-entity-recognition
- token-classification
- multilingual
- modernbert
- mmbert
- encoder
---
# DARE: Multilingual PII Detection
DARE is a 307.6M-parameter bidirectional encoder for detecting personally identifiable information (PII) in text.
It employs an [mmBERT-base](https://huggingface.co/jhu-clsp/mmBERT-base) backbone with a 117-label BIO token-classification head that covers 58 PII types.
| | |
|---|---|
| Architecture | `ModernBertForTokenClassification` (22 layers, hidden size 768) |
| Parameters | 307,620,213 |
| Labels | 117 (B-/I- tags for 58 types, plus an outside tag) |
| Context | 512-token windows with 64-token stride |
## Usage
Access is gated: accept the terms on this page, then log in with `hf auth login` before running the snippet.
The snippet below needs `transformers>=5.0`, because the tokenizer is saved as `TokenizersBackend`.
It implements the full decoder: a PII threshold of 0.175 on `1 - p(O)`, 512-token windows with a 64-token stride, BIO merging and ASCII word completion at span boundaries.
```python
import torch
from transformers import AutoTokenizer, AutoModelForTokenClassification
repo = "DataAnonymiser/DARE"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForTokenClassification.from_pretrained(repo).eval()
O = model.config.label2id["O"]
def detect(text, tau=0.175):
"""Spans whose PII score 1 - p(O) >= tau, over 512-token windows overlapping by 64 tokens."""
enc = tok(text, max_length=512, stride=64, truncation=True, padding=True,
return_overflowing_tokens=True, return_offsets_mapping=True, return_tensors="pt")
offsets = enc.pop("offset_mapping")
enc.pop("overflow_to_sample_mapping", None)
with torch.no_grad():
probs = model(**enc).logits.softmax(-1)
labels = {} # token offsets -> label; a token is PII if any window flags it
for w in range(probs.shape[0]):
for (s, e), p, keep in zip(offsets[w].tolist(), probs[w], enc["attention_mask"][w].tolist()):
if not keep or s == e or labels.get((s, e), "O") != "O":
continue
q = p.clone(); q[O] = 0
labels[(s, e)] = model.config.id2label[int(q.argmax())] if 1 - p[O] >= tau else "O"
spans, cur = [], None
for (s, e) in sorted(labels):
lab = labels[(s, e)]
if lab == "O":
cur = None
continue
prefix, typ = lab.split("-", 1)
if prefix == "B" or cur is None or cur["type"] != typ:
cur = {"type": typ, "start": s, "end": e}
spans.append(cur)
else:
cur["end"] = e
word = lambda c: c.isascii() and c.isalnum()
out = []
for sp in spans:
s, e = sp["start"], sp["end"]
while s < e and text[s].isspace(): s += 1 # tokens carry their leading space
while e > s and text[e - 1].isspace(): e -= 1
while s > 0 and s < e and word(text[s]) and word(text[s - 1]): s -= 1 # complete ASCII words
while e < len(text) and s < e and word(text[e - 1]) and word(text[e]): e += 1
if s < e:
out.append({"type": sp["type"], "start": s, "end": e, "text": text[s:e]})
return out
print(detect("Dear Ms. Johanna Becker, your account DE89 3704 0044 0532 0130 00 was "
"charged on 12 March 2024. Call +49 30 1234567 or write to j.becker@example.org."))
# [{'type': 'first_name', 'start': 5, 'end': 16, 'text': 'Ms. Johanna'},
# {'type': 'last_name', 'start': 17, 'end': 23, 'text': 'Becker'},
# {'type': 'account_number', 'start': 38, 'end': 65, 'text': 'DE89 3704 0044 0532 0130 00'},
# {'type': 'date', 'start': 81, 'end': 94, 'text': '12 March 2024'},
# {'type': 'phone_number', 'start': 101, 'end': 115, 'text': '+49 30 1234567'},
# {'type': 'email', 'start': 128, 'end': 148, 'text': 'j.becker@example.org'}]
```
The output above was produced on CPU with transformers 5.16.1 and torch 2.13.0.
## PII types
account_number, age, api_key, bank_routing_number, biometric_identifier, blood_type, certificate_license_number, city, company_name, coordinate, country, county, credit_debit_card, customer_id, cvv, date, date_of_birth, date_time, device_identifier, education_level, email, employee_id, employment_status, fax_number, first_name, gender, health_info, health_plan_beneficiary_number, http_cookie, ipv4, ipv6, language, last_name, license_plate, location, mac_address, medical_record_number, national_id, occupation, password, person, phone_number, pin, political_view, postcode, race_ethnicity, religious_belief, sexuality, ssn, state, street_address, swift_bic, tax_id, time, unique_id, url, user_name, vehicle_identifier
## Results
The table reports character-level entity-agnostic detection. Brackets are 95% bootstrap confidence intervals over documents.
| System | REDACT F2 | ai4privacy F2 |
|---|---:|---:|
| Presidio | 48.19 [47.11, 49.27] | 60.53 [60.17, 60.90] |
| GLiNER-PII | 59.54 [58.34, 60.73] | 79.43 [79.07, 79.79] |
| Qwen3.5 2B (prompted) | 43.97 [41.95, 45.99] | 72.52 [72.04, 73.00] |
| Qwen3.5 9B (prompted) | 69.35 [67.80, 70.87] | 91.43 [91.17, 91.69] |
| Gemma 4 E2B-it (prompted) | 63.15 [61.68, 64.60] | 79.90 [79.45, 80.33] |
| OpenAI Privacy Filter | 66.71 [65.43, 67.96] | 82.62 [82.23, 83.01] |
| **DARE** | **75.46** [74.39, 76.48] | **94.12** [93.97, 94.27] |
The test sets are the [REDACT benchmark](https://github.com/guneeshvats/REDACT-PII-Benchmark) (the benchmark authors' language-stratified 1,000-document sample, 25 languages) and a language-stratified 10,000-document sample of the [ai4privacy/pii-masking-openpii-1.5m](https://huggingface.co/datasets/ai4privacy/pii-masking-openpii-1.5m) validation split (30 languages).
The prompted language models ran with greedy decoding, bfloat16 and a 512-token output budget.
## Languages
Bengali, Chinese, Dutch, English, French, German, Hindi, Italian, Persian, Portuguese, Spanish, Swedish and Ukrainian.
The model also transfers to other languages, and the ai4privacy evaluation includes some of them, but no guarantee is made for languages outside this list.
## Intended use and limitations
- **Intended use.** Detecting and masking PII before text is stored, shared or processed, as a first pass with human or rule-based review where the stakes are high.
- **No guarantee of anonymisation.** DARE misses about one in five gold PII characters on REDACT. Masking its detections does not by itself make data anonymous in a legal sense.
- **Recall-oriented.** The default threshold favours recall, so some non-PII text is flagged. Raise `tau` for higher precision.
- **Out of scope.** Surveillance, profiling or re-identification of individuals.
## License
The model weights are released under [CC BY-NC 4.0](https://creativecommons.org/licenses/by-nc/4.0/) and may be used for non-commercial purposes with attribution.
For commercial licensing, please contact us at [support@dataanonymiser.com](mailto:support@dataanonymiser.com).
The base model is under its own licence; see `NOTICE.md`.
## Citation
Citation information will be added after the review period.