Deerpark/pii-tagger-hipaa

PII Tagger HIPAA

A 45M-parameter GLiNER span tagger for the identifiers HIPAA's Safe Harbor method lists, in clinical text: names, employers, towns and counties, addresses, phone numbers, emails, dates, ages over 89, unique characteristics (an unusual job, a first or a record, a tattoo's words), record, plan, account and licence numbers, device and vehicle ids, and case, claim, authorisation and accession numbers. It gives their exact positions and writes no text. It runs on a CPU, and its 16-bit ONNX copy (91 MB) runs in a browser.

This is E9: Deerpark/pii-tagger-edge (round 7) trained on clinical text for HIPAA mode only (E7), then taught three more labels: unique characteristic, date and age over 89. The previous version, E7, stays at revision dbd5081f1197b25afbe40c43c83a7868be2ed9b1. For general text use pii-tagger-edge, which this model was not chosen for.

Use

from gliner import GLiNER

model = GLiNER.from_pretrained("Deerpark/pii-tagger-hipaa")

LABELS = ["name", "street address", "postcode", "email address", "phone number", "ssn", "account number",
          "credit card number", "iban", "swift code", "date of birth", "customer id", "employee id", "username",
          "passport number", "driver license number", "tax id",
          "employer", "town or county", "other identifying number",
          "date", "age over 89", "unique characteristic"]
model.predict_entities(text, LABELS, threshold=0.25)

• Ask with these label names. The last six are this model's own (veil_labels in gliner_config.json); it is asked for town or county instead of city. Medical record and health plan numbers come back as customer id.
• 0.25 is the threshold chosen on dev and tested.
• It often marks only part of a unique characteristic (the town in "the first woman elected sheriff of ..."). As tested, each unique characteristic span is widened to its sentence, out to the nearest . ; ! ? | or line break on each side; veil's veil.tagged.widened does this.
• ZIP codes, web and IP addresses aren't asked of it: rules find those by their shape, as they also find most dates and ages.
• Give it up to about 200 words at a time, split on whole lines.

In veil, TaggedRedactor(profile=HIPAA) runs it beside the HIPAA profile's rules, at 0.25 and widened.

Results

Scored once, after the model was frozen, with veil's HIPAA rules beside it, against E7 at 0.35 with the same rules, read with the 16-bit ONNX copy (found: every word of a detail hidden; review-adjusted: after blind reviewers sorted each miss into real, boundary, not an identifier, or debatable; other words hidden: of the words no mark covers; 95% intervals from 2,000 resamples of documents):

Sealed test	System	Found	Found, review-adjusted	Other words hidden
clinical6: 960 documents, 10,061 identifiers	rules alone	87.29%		2.63%
	rules + E7	97.31%	98.66%	3.99%
	rules + this model	98.55%	99.56%	4.14%
clinical6u: 320 documents dense in unique characteristics (194)	rules + E7	12.9% of unique characteristics	36.1%	3.81%
	rules + this model	92.8%	96.4%	7.57%

On clinical6, against E7, found rose 1.24 points (+0.98 to +1.51) and other words hidden 0.15 point (+0.01 to +0.29); documents with a real or debatable miss left fell from 11.7% to 4.5%. Its remaining real misses are mostly employers and names. On text dense in unique characteristics, widening hides whole sentences, and other words hidden nearly doubles.

In veil's browser extension (onnxruntime-web, one thread, an Intel i5 MacBook), a 200-word message takes about 3.1 s against E7's 2.85 s (9% more, for three more labels).

The test documents are agent-written, as the training documents are; how this carries to real notes hasn't been measured.

Training

From E7, ten 500-step segments with E7's recipe (batch 8, learning rates 1e-5 and 2e-5, focal loss 0.75 and 2, bf16); the tenth was chosen. The mix is E7's, plus:

• 320 more agent-written clinical documents, their writers told to put unique characteristics in most (unusual jobs and titles, firsts and records, civic roles, events that made the news, tattoo words, named boats and horses) and as often look-alikes that don't identify anyone (ordinary jobs, common hobbies, public figures); 223 unique characteristics marked. All invented.
• Two earlier test sets of agent-written documents, spent, as audited.
• Every training chunk marking a unique characteristic, three times.

E7's mix: agent-written clinical documents (discharge summaries, emergency and EMS notes, clinic notes, letters, lab and device reports, nursing and hospice notes, billing and claims, patient messages, in English and some Spanish, every identifier marked, all invented), the same in capitals or OCR-misread, Nemotron-PII's healthcare and pharmaceutical documents (train split, © NVIDIA, CC BY 4.0), case reports from the PubMed Central open-access subset under CC BY or CC0 (each article under its own licence; the PMCIDs and licences are in veil's repository, benchmarks/train/clinical/pmc_raw/reports.jsonl), and a replay of round 7's training data.

Each example was offered only the labels its source marks throughout. No training document shares more than a fifth of its 8-word runs with any development or test document. The protocol, every judge's and reviewer's verdict and the integrity log are in veil's repository (benchmarks/judged/PROTOCOL.md, "E9").