| --- |
| license: cc-by-sa-4.0 |
| datasets: |
| - nvidia/Nemotron-PII |
| language: |
| - en |
| tags: [pii, redaction, token-classification, logs] |
| --- |
| |
| # cynch-piitag |
|
|
| ## Use it in 30 seconds |
|
|
| ```sh |
| curl -sSfL https://huggingface.co/scottmas/piitag/resolve/main/install.sh | sh |
| cat app.log | piitag redact > app.redacted.log |
| ``` |
|
|
| `install.sh` picks the binary for your OS and CPU (Linux x86_64 or aarch64, static; macOS arm64 or |
| x86_64), checks its sha256 against `SHA256SUMS` from the same pinned commit of this repo, and puts |
| `piitag` in `~/.local/bin` (or `$PIITAG_INSTALL_DIR`). The binary is about 7 MB with the model |
| inside: no runtime, no network, nothing else to download. On Windows, download |
| `piitag-<version>-x86_64-pc-windows-gnu.exe` from the repo's files and check it against |
| `SHA256SUMS`. |
|
|
| ```text |
| $ printf 'login ok user=jane.doe@example.com ip=203.0.113.42\n2026-10-10T12:00:01Z WARN request 77 took 412 ms\n' | piitag redact |
| login ok user=[EMAIL] ip=[IPV4] |
| 2026-10-10T12:00:01Z WARN request 77 took 412 ms |
| ``` |
|
|
| - **Files:** `piitag redact a.log b.log` reads them in order, as `cat a.log b.log` would. |
| `piitag redact -i app.log` rewrites the file in place (a temp file renamed over it, so a failed |
| run leaves it untouched); `-i.bak` keeps the original as `app.log.bak`. |
| - **What counts as PII:** `--labels identifiers` (the default: names, contact details, ids, card and |
| account numbers, keys and passwords; it keeps timestamps, places and companies), `--labels all` |
| (also dates, places, organizations and demographics), or `contact`, `financial`, `secrets`, |
| `government_ids`, or type names, comma-separated (`--labels email,ipv4`). |
| - **The mask:** `[KIND]` by default; `--mask stars` keeps the length (one `*` per character); |
| `--mask hash` writes `[KIND:1a2b3c4d]`, the same value giving the same tag within one run (a key |
| random per run, so tags cannot be looked up across runs). |
| - **`--json`:** where the hits are instead of the text, one line per input line: |
| `{"line":1,"hits":[{"start":14,"end":34,"kind":"email"},{"start":38,"end":50,"kind":"ipv4"}]}`, |
| offsets in bytes within the line. |
| - **Streaming:** it reads 64 KB blocks cut at a line end, detects them on a pool of threads |
| (`--threads N`; by default one per physical core, the performance cores on Apple silicon), and |
| writes them in order. A line is written as soon as it is detected: `tail -f app.log | piitag |
| redact` works. |
|
|
| **How good, honestly.** Averaged over PIIMB's six tasks (a local run of its harness, below), the |
| model catches 90.6% of the PII characters and redacts 1.65% of the non-PII characters. `piitag |
| redact` on 100 MB of synthetic logs (`piitag-train logcorpus --mb 100 --seed 0`, as text) runs at 38 |
| MB/s on one M3 Pro performance core and 172 MB/s on its 5, and at 15 MB/s on one vCPU of an |
| EPYC-Rome VM and 48 MB/s on its 4. That counts the exact rules, the decode and the I/O; the model |
| alone runs faster ("Speed" below). It is strongest in English; its training data also covers |
| OpenPII-1M's 23 languages (Russian PII F1 0.619). It misses some PII and flags some non-PII: it lowers the risk of a leak, and it is **not a |
| compliance guarantee**. Check its output on your own data before you rely on it. |
|
|
| piitag finds PII and credentials in bytes (logs, traces, JSON, command lines and prose), and returns |
| where each one is and what kind it is. It runs on one CPU core at 26–107 MB/s and has no runtime |
| dependencies. One model is embedded at build time (published as |
| [`scottmas/piitag`](https://huggingface.co/scottmas/piitag)), and the caller says which of its labels |
| are PII (`Labels`): a hit of any other label is not returned. |
|
|
| - **`Labels::preset(&m, "identifiers")`** redacts identifiers and secrets: names, contact details, |
| ids, card and account numbers, keys and passwords. It leaves dates, places, organizations and |
| demographics alone. Use it for logs, where a timestamp or a city is data you keep. |
| - **`Labels::all(&m)`** also redacts context PII: dates and times, URLs, companies, occupations, |
| places and demographics, which is every Nemotron-PII label. Use it for text you share, where those |
| details identify a person too. |
| - Other presets (`contact`, `financial`, `secrets`, `government_ids`) and any set of type names |
| (`Labels::of`) work the same way. |
|
|
| **The model is C2** (`c2-2026-10-10`, run C2-s1; `train/recipe.json`: the training mix chosen by |
| regression, `train/README.md` §6, with 55 types scored as 8 groups, 528,484 parameters, `pii_q` −316, |
| `gate_q` −611), chosen on validation and scored once on the test sets below. It is `piitag.piitag` in |
| [`scottmas/piitag`](https://huggingface.co/scottmas/piitag), the file `weights.lock` embeds and the |
| release binaries carry. It replaces the narrow model (34 identifier types, `pii_q` −1160) and the full |
| model (55 types, one cutoff, `pii_q` −273). The tables show it beside them, each scored as its own label scope, and beside the |
| previous candidate, anneal20-s1. |
|
|
| On PIIMB's nemotron-pii task, the narrow model masks 99.4% of the identifier characters and 1.2% of |
| the context characters. The full model masks 99.3% and 92.3%. |
|
|
| | | piitag narrow | piitag full | release candidate, `all` | release candidate, `identifiers` | Horizon small | Horizon base | OpenAI privacy-filter | OpenMed SuperClinical-Small | gravitee bert-small | previous candidate (anneal20-s1), `all` | previous candidate, `identifiers` | |
| |---|---|---|---|---|---|---|---|---|---|---|---| |
| | TAB test (ECHR), in-domain for piitag | **0.899** | 0.714 | 0.740 | 0.890 | 0.700 | 0.804 | 0.544 | 0.554 | 0.607 | 0.725 | 0.898 | |
| | RedactionBench | **0.748** | 0.601 | 0.627 | 0.747 | 0.351 | 0.306 | 0.186 | 0.200 | 0.120 | 0.611 | 0.727 | |
| | Privacy-Bench (Tonic) | 0.845 | 0.777 | 0.793 | 0.804 | 0.846 | **0.870** | 0.865 | 0.844 | 0.812 | 0.813 | 0.839 | |
| | Russian PII (redmadrobot) | 0.381 | 0.625 | 0.619 | 0.572 | **0.816** | 0.797 | 0.709 | 0.756 | 0.582 | 0.634 | 0.604 | |
| | parameters | 2.1M | 529k | 528k | 528k | 141M | 308M | 1.4B (50M active) | 141M (named "44M") | 28.5M | 528k | 528k | |
| | weights | 4.1 MB | 5.7 MB (5.1 MB of it the prior) | 5.7 MB, one file for both columns | | 563 MB (268 MB int8 ONNX) | | 2.8 GB | | 28.7 MB int8 ONNX | 5.7 MB | | |
| | throughput | 30.1 MB/s, one x86_64 core | 25.9 MB/s, one x86_64 core | 42.4 MB/s, one x86_64 core (EPYC-Milan) | 49.0 MB/s, one x86_64 core (EPYC-Milan) | about 0.009 MB/s, all M3 cores | | needs a GPU (per its evaluators) | | | 43.2 MB/s, one x86_64 core (EPYC-Milan) | 48.4 MB/s, one x86_64 core (EPYC-Milan) | |
|
|
| The release candidate is one file run with two label sets (`Labels`). Under `all` it scores above |
| the full file on TAB, RedactionBench and Privacy-Bench and 0.006 below it on Russian PII. Under |
| `identifiers` it is within 0.009 of the narrow file on TAB and RedactionBench, 0.041 below it on |
| Privacy-Bench, and 0.191 above it on Russian PII. Against the previous candidate it gains 0.015 on TAB |
| and 0.016–0.020 on RedactionBench, and loses 0.020–0.035 on Privacy-Bench and 0.015–0.032 on Russian |
| PII. |
|
|
| TAB's train split is in the training data, so TAB test is in-domain for piitag; no benchmark's dev or |
| test text was trained on, and no benchmark result chose either model. Every choice (data mix, model |
| size, epochs, seed, averaging window, operating point, gate) was made on validation data: |
| Nemotron-PII's val split, Gretel's validation split, the hard-case and JSON val sets, an |
| out-of-domain sample and TAB dev. The test sets and benchmarks were scored once, after the choice. |
|
|
| Each cell is character-level redaction F1 with the agent-io-guards metric: recall over the |
| must-redact characters, precision over must ∪ may. piitag's columns are the library's output (model ∪ |
| rules) on the bench's pinned benchmarks (`piitag-train bench run`), scored once. The other columns |
| are published on the |
| [Horizon-Labs/pii-redactor-small model card](https://huggingface.co/Horizon-Labs/pii-redactor-small) |
| (September 2026) with the same metric and splits. Our own run of Horizon small reproduces its card |
| column to within 0.001 on all four benchmarks, so the columns are comparable. Horizon small's |
| throughput is our measurement, not a published one: it took 395 s over the four benchmarks' 3.64 MB |
| on the M3 Pro's CPU, with PyTorch's default threads. |
|
|
| - **These four benchmarks are identifier benchmarks.** Their gold marks names, contact details and |
| ids, so the full model's dates, places and organizations count against its precision there (TAB |
| test: precision 0.595 against the narrow model's 0.941, recall 0.895 against 0.860). PIIMB below |
| labels context PII, and there the full model leads. Russian PII is the one benchmark here in a |
| language the full model trained on and the narrow model did not (0.625 against 0.381). |
| - **RedactionBench is long documents, and the card's harness truncates them.** It tokenizes with a |
| 2,048-token limit, so the other columns score only the first 2,048 tokens of each document: about |
| 30% of RedactionBench's text, and 84 of its 108 long documents get no redaction after token 2,046. |
| On the text Horizon small sees, F1 is 0.784 for piitag (the previous narrow export) and 0.751 for |
| Horizon small, and on that text's real (not synthetic) documents 0.817 and 0.889 (our runs). Read |
| the 0.748 against 0.351 as mostly a length effect, not a quality gap. |
| - **RedactionBench real-document slice:** the narrow model scores 0.790 and Horizon small 0.783 (our |
| runs only, through the same truncating harness for Horizon). |
| - **Other metrics:** RedactionBench's own leaderboard (R-Score, a different character metric) has |
| claude-opus-4-6 at 0.714 and gpt-5.4 at 0.659. Tonic's Privacy-Bench card (span-overlap F1) has |
| Tonic Textual at 91.7, Claude Opus 5 at 88.3 and Presidio at 82.8. Neither is on piitag's scale. |
|
|
| ```text |
| cargo test -p cynch-piitag |
| ``` |
|
|
| ## PIIMB |
|
|
| [PIIMB](https://huggingface.co/datasets/piimb/pii-masking-benchmark) scores the characters a model |
| masks against every labelled span, whatever its label, and ranks by F2. piitag's rows come from |
| PIIMB's own harness (`piimb evaluate`, piimb 0.3.0, the `sentences` subset at dataset revision |
| `5f47fff8`, the published results' revision), through a piitag pipeline |
| (`./piitag-train piimb-official`). The other rows are PIIMB's published results for the |
| most-downloaded models on the Hugging Face Hub, plus the best English mean, fetched 2026-10-05. |
| English F2 per task, and the mean over all six tasks (the published snapshot holds the English |
| tasks only): |
|
|
| | model | parameters | ai4privacy-en | nemotron-pii | gretel | privy | English mean | six-task mean | |
| |---|---|---|---|---|---|---|---| |
| | piitag release candidate, `all` | 528k | 0.980 | 0.953 | 0.974 | 0.858 | **0.941** | 0.890 | |
| | piitag previous candidate (anneal20-s1), `all` | 528k | 0.979 | 0.959 | 0.968 | 0.833 | 0.935 | 0.882 | |
| | piitag full | 529k | 0.980 | 0.955 | 0.968 | 0.828 | 0.933 | 0.879 | |
| | piitag narrow | 2.1M | 0.734 | 0.552 | 0.858 | 0.507 | 0.663 | 0.567 | |
| | fastino/gliner2-base-v1 | 208M | 0.750 | 0.739 | 0.891 | 0.499 | 0.720 | – | |
| | gravitee-io/bert-small-pii-detection | 28.5M | 0.900 | 0.693 | 0.818 | 0.987 | 0.849 | – | |
| | openai/privacy-filter | 1.4B | 0.843 | 0.607 | 0.869 | 0.536 | 0.714 | – | |
| | OpenMed/OpenMed-PII-SuperClinical-Small-44M-v1 | 141M | 0.949 | 0.972 | 0.953 | 0.659 | 0.883 | – | |
| | OpenMed/privacy-filter-multilingual | 1.4B | 0.942 | 0.766 | 0.864 | 0.468 | 0.760 | – | |
| | fastino/gliner2-privacy-filter-PII-multi | 307M | 0.914 | 0.854 | 0.936 | 0.646 | 0.837 | – | |
| | fastino/gliner2-multi-v1 | 307M | 0.903 | 0.829 | 0.870 | 0.661 | 0.816 | – | |
| | iiiorg/piiranha-v1-detect-personal-information | 278M | 0.803 | 0.360 | 0.607 | 0.369 | 0.535 | – | |
| | OpenMed/privacy-filter-multilingual-v2 | 1.4B | 0.975 | 0.907 | 0.973 | 0.818 | 0.918 | – | |
|
|
| - **The release candidate's rows are a local run of PIIMB's harness,** not a submission: nothing |
| was uploaded to the board. Its six-task mean is 0.890 (previous candidate 0.882, full 0.879; |
| mapa-eur-lex 0.594 against 0.574 and 0.567). Averaged over the six tasks it catches 90.6% of the PII |
| characters and redacts 1.65% of the rest. Run with `--labels identifiers`, its narrow view's English |
| mean is 0.950, against 0.935 for the previous candidate, 0.862 for the narrow file and 0.927 for the |
| full file. |
| - **The narrow model's PIIMB score is its scope, not its recall.** PIIMB counts a missed date or city |
| as a leak. On nemotron-pii, half the gold characters are context labels the narrow model leaves |
| alone by design, and on the other half it masks 99.4%. |
| - **The narrow view** is the same metric with the context labels excused: masking them is neither |
| credit nor charge. `piimbscore.NARROW_OF` maps each English task's labels to "must" or "may". |
| It reads, as F2, narrow and then full: nemotron-pii 0.992 and 0.974, gretel 0.904 and 0.967, |
| ai4privacy-en 0.939 and 0.977, privy 0.614 and 0.791. |
| - **privy's precision is mostly its filler.** privy is synthetic API traffic (XML, JSON, SQL) |
| in which every parameter that is not PII holds a random string (`"callback": "WsXwADdOH"`), and |
| only 6.8% of its characters are gold. piitag masks many of those strings as it would an unknown |
| token in a log; 18% of the narrow model's false-positive characters sit under credential names |
| (`oauth_token`, `access_token`, `key`), which a log redactor should mask anyway. |
| - **Multilingual tasks:** ai4privacy-multi F2 0.621 (narrow) and 0.977 (full), mapa-eur-lex 0.129 |
| and 0.567. The full model trains on OpenPII-1M's 23 languages and MAPA's EU legal text; the narrow |
| model and both priors are English-only. PIIMB's MAPA gold excludes places and organizations, which |
| the full scope masks, so part of the full model's MAPA gap is taxonomy, not misses. |
| - **Contamination:** none of PIIMB's 4,871 nemotron documents are in piitag's training or |
| validation data; all are in Nemotron-PII's held-out test split, which piitag never trained on. 93 |
| of 5,000 long sentences recur verbatim in the training split. They are Nemotron's own boilerplate |
| (closing lines, markdown table borders) and carry 2 of 42,626 entities. The full model trains on |
| the train splits of OpenPII-1M (ai4privacy), privy, MAPA and Gretel; PIIMB's tasks are their test |
| splits, which are refused, and `piitag-train contamination` drops every training document that |
| holds a PIIMB test sentence verbatim. |
|
|
| ## How it works |
|
|
| The output is the union of two parts, and the rules win on their bytes where the two overlap. |
|
|
| - **The model**: a token-level tagger. |
| - It cuts the buffer into tokens and classifies each token once. |
| - The inputs are hashed features of the token and of 4 neighbours on each side, plus a one-byte |
| general-English prior per token. |
| - A single windowed-attention layer mixes the neighbours, and everything runs on integer kernels. |
| - No feature holds a token's identity, so the model generalizes from shape and context rather than |
| from memorized values. |
| - **The exact rules**: deterministic matchers, the same for both scopes. |
| - Formats with a checksum or a fixed grammar. |
| - A secret after the key that names it (`--password=…`, `DB_PASSWORD=…`, `"api_key": "…"`, |
| `mysql -p…`). |
| |
| ## What it detects |
|
|
| | source | kinds | |
| |---|---| |
| | model, `identifiers` (34 types) | account_number, api_key, bank_routing_number, biometric_identifier, certificate_license_number, coordinate, credit_debit_card, customer_id, cvv, date_of_birth, device_identifier, email, employee_id, fax_number, health_plan_beneficiary_number, http_cookie, ipv4, ipv6, license_plate, mac_address, medical_record_number, national_id, password, person_name, phone_number, pin, postcode, ssn, street_address, swift_bic, tax_id, unique_id, user_name, vehicle_identifier; `pii` for a span with no confident type | |
| | model, `all` only (21 more types) | age, blood_type, city, company_name, country, county, date, date_time, education_level, employment_status, gender, honorific, language, occupation, political_view, race_ethnicity, religious_belief, sexuality, state, time, url | |
| | rules (`rules::KINDS`) | private_key, jwt, account_number (IBAN, length table + mod-97), credit_debit_card (Luhn), email, coordinate, vehicle_identifier (ISO 3779 check digit, or a cue), swift_bic, mac_address, ipv6, ipv4, api_key, secret (the value after a credential key: `password`, `token`, `api_key`, `client_secret`, …), prefixed_id, uuid, ulid, ksuid, hex_id, b64_id | |
| |
| A JWT or a known key format after a credential key keeps its own kind; any other value there is |
| `secret`. Credentials in traffic are `secret` (an AWS key id `api_key`): a URL's password |
| (`postgres://app:S3cr3t!x@db`), an `Authorization: Bearer …`/`Basic …` value, a signed URL's |
| signature (`X-Amz-Signature=`, `sig=`), an OAuth `code=` beside `state=` or `redirect_uri=`, and a |
| cookie whose name says session (`session`, `sid`, `PHPSESSID`, `connect.sid`, …). A cookie with any |
| other name is the model's call. |
|
|
| - **A value the model tags once is tagged at every bounded occurrence in the buffer**, so a name |
| that has its cue (`Dear Kerttu Lehtinen,`) is masked where it has none (`Kerttu Lehtinen,200,OK`). |
| A single word spreads only to as many occurrences as the model tagged itself: one wrong tag on |
| a common word does not cover the document. |
| - **A chunk boundary does not hide a cue.** Each chunk is scored with up to 64 bytes of its |
| neighbours' text on each side. |
| - **Infrastructure ids are not PII for the full model.** It learned, from synthetic logs and |
| headers, that an id under `trace_id`, `request_id` or `commit` is not a person's and one under |
| `user_id` or `customer_id` is. Its file says `"id_rules": "defer"`: the uuid, ulid, ksuid, hex and |
| base64 id rules let the model decide those bytes. On the synthetic held-out set it masks 0.5% of |
| infrastructure ids and 98.6% of identifying ones. The narrow model has no such key, and the id |
| rules mask every id, as before. |
|
|
| ## API |
|
|
| ```rust |
| use cynch_piitag::{Labels, Model, Tagger}; |
| |
| let m = Model::embedded()?; // or Model::load(path) / Model::from_bytes(bytes) |
| let labels = Labels::preset(&m, "identifiers").map_err(cynch_piitag::Error::Format)?; // or Labels::all(&m) |
| let tagger = Tagger::new(m, labels); |
| let x = br#"{"user":"Jane Doe","mail":"jane.doe@example.com"}"#; |
| let segments = [9..17, 27..47]; // JSON value literals; &[] means all of x |
| for hit in tagger.detect(x, &segments) { |
| // hit.range: byte range in x; hit.kind: "person_name", "email", ...; hit.source: Rule | Model |
| println!("{:?} {} {}", &x[hit.range.clone()], hit.kind, hit.source.name()); |
| } |
| # Ok::<(), cynch_piitag::Error>(()) |
| ``` |
|
|
| - **Hit shape**: hits are sorted and disjoint, and each one lies inside one segment. |
| - **Labels**: a model span of a type outside the set is dropped before a value is spread to its |
| other occurrences, and a rule hit counts as a type (`labels::RULE_TYPE`: `uuid` is `unique_id`, |
| `jwt` is `api_key`, ...). A `Labels` is built from the `Model` it runs with; `Tagger::new` refuses |
| one built for another model. |
| - **Segments restrict where hits land, not what the model reads**: the model reads all of `x`, its |
| context crosses segment edges, and its spans are clipped to the segments. The rules run inside each |
| segment. |
| - **Char boundaries**: on valid UTF-8 input whose segment edges are char boundaries, every hit edge is |
| one too (`tests/robust.rs`). |
| - **Threading**: `Tagger` is `Sync`. Scratch buffers are thread-local, so one tagger serves every |
| thread without a lock. |
| - **Kernels**: `Tagger::new` picks the CPU's SIMD kernel: AVX2 on x86_64, NEON with `dotprod` on |
| aarch64 (Apple M1 and later, Graviton2 and later, Neoverse N1 and later). `Tagger::with_kernel(m, |
| labels, Kernel::Scalar)` forces the portable kernels. Every kernel's output is bit-identical. |
| - **Token cache**: each thread keeps a 4 MiB cache of recently seen tokens (keyed by the token's |
| bytes and the separators on both sides), so a repeated token skips its features, embedding and |
| gate. It never changes output (`tests/token_cache.rs`). |
|
|
| The `piitag` binary runs the same path over a records file: |
| `piitag detect --in RECORDS.bin [--model FILE.piitag] [--labels identifiers|all|TYPE,..]` (or |
| `--rules-only`) prints one JSON line of hits per record; `--labels` defaults to `all`. |
|
|
| ### Using it from dstl |
|
|
| `dstl`'s `Distiller` holds one `Tagger` for the process and hands it each leaf as `key: value`, with |
| the value as the one segment. The key is context the model reads, and every hit lands in the value. |
|
|
| ## Speed |
|
|
| One thread over the bench corpus: Nemotron-PII's test split, 95.9 MB. The narrow model finds |
| 469,925 spans and the full model 848,037 spans. `Tagger`'s path runs end to end: chunk, featurize, |
| forward, decode. Instructions/byte is the stable measure; the VM's wall clock moves ±15% between |
| identical runs. The x86_64 columns were measured on 2026-10-08 with the 4 MiB token cache. The |
| aarch64 columns were measured before the x86 speed work, with the earlier 1 MiB cache; an ARM pass is |
| in progress. The M3 retires about twice as many instructions per cycle as the EPYC. Output is |
| bit-identical in every row. |
| |
| | | x86_64 AVX2 (AMD EPYC-Rome VM), narrow | x86_64, full | aarch64 NEON (Apple M3 Pro), narrow | aarch64, full | |
| |---|---|---|---|---| |
| | instructions/byte, token cache on (the default) | **190.9** | **241.8** | **275.9** | **345.3** | |
| | instructions/byte, token cache off | 342.5 | 393.5 | 374.0 | 445.8 | |
| | instructions/byte, cache emptied before every record | 330.5 | 381.4 | 407.6 | 478.0 | |
| | ns/byte (MB/s), token cache on | 33.2 (30.1) | 38.6 (25.9) | 16.5 (60.7) | 19.6 (51.1) | |
| | ns/byte (MB/s), token cache off | 40.4 (24.8) | 46.0 (21.7) | 21.4 (46.7) | 24.9 (40.2) | |
| | ns/byte (MB/s), portable scalar kernels | 144.4 (6.9) | 201.1 (5.0) | 49.6 (20.2) | 65.5 (15.3) | |
| |
| The rules alone, either scope: 10.0 instructions/byte and 1.9 to 2.3 ns/byte (435 to 525 MB/s) on |
| the EPYC across two runs; 9.6 instructions/byte and 0.76 ns/byte (1,325 MB/s) on the M3, measured |
| earlier. |
| |
| - **The token cache** hits 79.6% of the corpus's tokens (x86 run). A hit skips the token's |
| features, embedding and gate, which are most of the instructions. Attention and everything after |
| it run for every kept token. |
| - **The pessimistic row** empties the cache before each record. Each bench record is a single chunk, |
| and the cache is probed for a whole chunk before it is filled, so that row never hits a previous |
| record's token. A token repeated inside one chunk is still computed once, which is why the x86 |
| row now runs below the cache-off row. |
| - **The cascade gate** skips 50.4% of tokens for the narrow model and 15.8% for the full model, |
| which keeps more tokens because more of the text is PII to it. |
| - **The full model is 529k parameters, a quarter of the narrow model's**, but not faster: its |
| feature table is a quarter the size, and its gate keeps 84% of tokens where the previous 2.1M |
| full model's kept 71%, so attention runs on more of them. |
| |
| **The release candidate, per preset** (token cache on, the same corpus, one thread pinned to one core; |
| in parentheses, MB/s). Its gate is per group, so a selective label set skips more of the net. Measured |
| on 2026-10-10 from one build, in two interleaved rounds per machine, beside the previous candidate and |
| C2's neighbour-gated file. The x86_64 rows ran on an AMD EPYC-Milan VM with dedicated vCPUs (Hetzner |
| ccx43), not the EPYC-Rome VM of the table above. The M3 rows include the speed work since the table |
| above's aarch64 pass. Under `all` C2 costs what the previous candidate costs (254.8 against 255.1 |
| instructions/byte on x86_64, 199.1 against 199.6 on the M3); under `secrets` it costs 12% more on x86_64 |
| (224.7 against 201.3). |
|
|
| | | `all` | `identifiers` | `contact` | `financial` | `secrets` | `government_ids` | |
| |---|---|---|---|---|---|---| |
| | release candidate, x86_64 instructions/byte | 254.8 | 221.8 | 222.8 | 154.1 | 224.7 | 172.7 | |
| | release candidate, x86_64 ns/byte | 23.6 (42.4) | 20.4 (49.0) | 21.4 (46.7) | 14.5 (69.1) | 20.5 (48.7) | 16.1 (62.1) | |
| | release candidate, M3 instructions/byte | 199.1 | 173.4 | 173.5 | 124.2 | 176.0 | 138.1 | |
| | release candidate, M3 ns/byte | 14.2 (70.2) | 12.5 (79.7) | 12.4 (80.5) | 9.3 (107.3) | 12.6 (79.2) | 10.2 (98.4) | |
| | previous candidate (anneal20-s1), x86_64 ns/byte | 23.1 (43.2) | 20.7 (48.4) | 20.3 (49.2) | 13.9 (71.8) | 18.5 (53.9) | 15.5 (64.4) | |
| | previous candidate, M3 ns/byte | 14.5 (69.2) | 12.6 (79.2) | 12.4 (80.5) | 9.0 (111.7) | 11.6 (85.9) | 9.9 (101.3) | |
| | neighbour-gated C2, x86_64 ns/byte | 20.2 (49.6) | 19.4 (51.4) | 17.4 (57.3) | 17.3 (57.7) | 18.5 (54.1) | 14.8 (67.5) | |
| | neighbour-gated C2, M3 ns/byte | 14.0 (71.7) | 13.4 (74.7) | 11.8 (84.4) | 12.2 (82.1) | 13.0 (76.9) | 10.8 (92.3) | |
|
|
| The neighbour gate (see "The model file") is faster than the cascade on x86_64 under five presets and |
| slower under `financial`; on the M3 it is slower under four. On log-shaped text it keeps 78% of tokens |
| under `identifiers` (23% on the bench corpus) and runs 0.88× the cascade's speed (`scripts/ab.sh`, |
| cycles, below). |
| |
| **Many threads.** A `Tagger` is `Sync`: share one across threads, each with its own buffers and its own |
| 4 MiB token cache. `piitag-bench --threads N` splits the corpus into N contiguous shards of equal bytes, |
| one thread each, and reports the aggregate MB/s (all bytes over the slowest thread's time) and the |
| scaling efficiency, aggregate / (N × one thread). The threads' spans hash equal to one thread's. C2 with |
| the cascade gate, `--labels identifiers`, median of 3 interleaved repeats (`--runs 3` on the VM, 2 on |
| the M3), two corpora: the bench corpus, and 100 MB of synthetic logs (`piitag-train logcorpus`: syslog, |
| nginx and apache, JSON app logs, k8s events and stack traces, 9% of bytes PII). On the VM each point is |
| restricted (`taskset`) to N distinct physical cores while N ≤ 8, then to their SMT siblings. |
| |
| | threads | EPYC-Milan, documents MB/s | efficiency | EPYC-Milan, logs MB/s | efficiency | M3 Pro, documents MB/s | efficiency | M3 Pro, logs MB/s | efficiency | |
| |---|---|---|---|---|---|---|---|---| |
| | 1 | 49.1 | 0.99 | 41.5 | 1.00 | 81.2 | 1.00 | 66.2 | 1.02 | |
| | 2 | 95.6 | 0.99 | 82.3 | 0.99 | 153.5 | 0.96 | 130.6 | 0.98 | |
| | 4 | 185.9 | 0.96 | 158.1 | 0.95 | 309.2 | 0.97 | 256.9 | 0.98 | |
| | 5 (the M3's P-cores) | | | | | 360.7 | 0.91 | 314.6 | 0.96 | |
| | 6 | 264.6 | 0.90 | 236.6 | 0.95 | 397.2 | 0.83 | 324.2 | 0.82 | |
| | 8 (the VM's cores) | 365.7 | 0.92 | 296.5 | 0.89 | 449.8 | 0.70 | 385.8 | 0.73 | |
| | 8 on 4 cores (SMT pairs) | 258.5 | 0.65 | 218.2 | 0.66 | | | | | |
| | 11 (every M3 core) | | | | | 466.9 | 0.53 | 433.0 | 0.59 | |
| | 12 | 388.6 | 0.65 | 330.8 | 0.66 | | | | | |
| | 16 (every VM thread) | 517.0 | 0.65 | 422.2 | 0.63 | | | | | |
| |
| - **One thread per core scales almost linearly**: 0.89–0.96 at 8 cores on the VM and 0.91–0.96 at the |
| M3's 5 P-cores. With 8 threads on 8 cores the cycles per byte equal one thread's (75.2 against 73.7 on |
| documents, 88.4 against 87.9 on logs, `perf stat`), and loads served from L3 or DRAM stay under 0.05 |
| per byte at every N: memory bandwidth and the shared L3 are not the limit, although 16 caches of |
| 4 MiB exceed the 32 MiB L3. |
| - **The bend is SMT, and on the M3 the E-cores.** Two threads on one core each run at IPC 2.0 instead |
| of 3.0 (cycles/byte 109–111 against 74, instructions unchanged): the loop is execution-bound, so a |
| core's second thread adds about a third. On the M3 an E-core adds a quarter (documents) to a third (logs) of a P-core. |
| - **Equal shards waste the faster threads.** The aggregate is set by the slowest thread: at 12 threads |
| on the VM, 4 cores run two threads, and the other 4 wait (slowest thread 32 MB/s, mean 39). A caller |
| that hands out records from a shared queue instead of fixed shards keeps every thread busy. |
| - **The token cache** hits 90.8% of tokens on logs against 79.5% on documents, but logs cost more per |
| byte (278 against 222 instructions): they hold more tokens per byte (0.146 against 0.133) and more of |
| them are PII-shaped. Per thread the hit rate falls slightly with N (79.5% → 77.4% at 16 threads), |
| since each shard starts cold. At 16 threads a 2 MiB cache is 7% slower than 4 MiB (hit 73.2%) and an |
| 8 MiB cache 1–5% faster. |
| |
| `scripts/phase-icount.sh` splits the instructions by phase; `CONTRIBUTING.md` says how to measure. |
| |
| ## Quality |
| |
| Every number is the library's output (model ∪ rules) from the integer reference at each model's |
| operating point, with the gate on. The full model's checkpoint is scored at `pii_q` −266, where |
| its prose validation leak is the same as at the shipped −273. The test sets were scored once, after the models were chosen. |
| Each model is scored against its own labels: the narrow model's gold is the identifier labels, the |
| full model's is every label, so the full model's span leak counts dates, places and organizations |
| too. |
|
|
| - **span leak**: the share of gold PII spans with any byte left unredacted. |
| - **non-PII redacted**: the share of non-PII bytes redacted. |
| - **name false-hit share**: the share of redacted person-name tokens that are not PII. |
|
|
| | set | narrow: span leak | non-PII redacted | name false-hit share | full: span leak | non-PII redacted | name false-hit share | |
| |---|---|---|---|---|---|---| |
| | prose (Nemotron-PII test) | 0.00969 | 0.00144 | 0.0299 | 0.06735 | 0.00450 | 0.0120 | |
| | hard cases (held-out vocabulary) | 0.01597 | 0.01769 | 0.2378 | – | – | – | |
| | JSON documents | 0.00490 | 0.00090 | 0.0080 | 0.02858 | 0.00257 | 0.0034 | |
| | JSON, hard shapes | 0.00755 | 0.00255 | 0.0250 | 0.04222 | 0.00410 | 0.0136 | |
| | out-of-domain sample | 0.00901 | 0.00142 | 0.0819 | 0.07025 | 0.00409 | 0.0246 | |
|
|
| The hard cases are role words before names and capitalized non-names, labelled for the narrow |
| scope; under full labels some of their negatives are PII, so the full model has no such set. |
|
|
| **The release candidate, per preset.** Its held-out suites go through the file (`piitag detect |
| --labels L`, the same 20,000 records per suite `int-eval --held-out` samples, seed 0), scored by |
| `labelset.evaluate` against the preset's own gold types. F2 is character-level, recall-weighted; in |
| parentheses, the previous candidate (anneal20-s1, `pii_q` −332), scored the same way by the same |
| build. The test sets were scored once, after the candidate was chosen. |
|
|
| | preset | prose F2 | prose span leak | JSON F2 | JSON span leak | JSON hard F2 | JSON hard span leak | |
| |---|---|---|---|---|---|---| |
| | `all` | 0.9543 (0.9598) | 0.0662 (0.0591) | 0.9801 (0.9848) | 0.0286 (0.0260) | 0.9745 (0.9773) | 0.0408 (0.0365) | |
| | `identifiers` | 0.9925 (0.9922) | 0.0125 (0.0130) | 0.9942 (0.9948) | 0.0120 (0.0109) | 0.9920 (0.9928) | 0.0145 (0.0133) | |
| | `contact` | 0.9904 (0.9901) | 0.0165 (0.0168) | 0.9911 (0.9924) | 0.0169 (0.0147) | 0.9881 (0.9898) | 0.0196 (0.0172) | |
| | `financial` | 0.9760 (0.9746) | 0.0384 (0.0401) | 0.9744 (0.9765) | 0.0454 (0.0417) | 0.9757 (0.9738) | 0.0417 (0.0439) | |
| | `secrets` | 0.9772 (0.9831) | 0.0599 (0.0520) | 0.9850 (0.9880) | 0.0508 (0.0429) | 0.9829 (0.9877) | 0.0519 (0.0446) | |
| | `government_ids` | 0.9848 (0.9871) | 0.0187 (0.0175) | 0.9649 (0.9570) | 0.0474 (0.0577) | 0.9697 (0.9650) | 0.0393 (0.0475) | |
|
|
| It is behind the previous candidate in 12 of 18 F2 cells, by at most 0.0059 (`secrets` on prose), |
| and ahead in 6. **The trade is Nemotron for PIIMB:** C2's main phase is 65% Nemotron-PII where the |
| previous candidate's was 75%, and these suites are Nemotron-PII's test split and its JSON renderings, |
| while PIIMB's six-task mean rose 0.008. **`government_ids` on JSON**, the previous candidate's one |
| regression against the full file, is now ahead of both: F2 0.9649 against 0.9570 (full file 0.9631) |
| and span leak 0.0474 against 0.0577 (full 0.0496). |
| |
| ## Intended use |
| |
| Redacting PII and credentials from English logs, traces, JSON and command lines before they are |
| stored or shared, on CPU, at line rate: the `identifiers` labels for operational data, `all` for text |
| that leaves your hands. |
| |
| ## Limitations |
| |
| - **English first.** Both priors are English. The narrow model trains on English text only |
| (Russian PII 0.381); the full model also trains on OpenPII-1M's 23 languages and MAPA (Russian PII |
| 0.625, PIIMB ai4privacy-multi 0.977). |
| - **First names the prior does not know leak more.** On the out-of-domain sample, the narrow model |
| leaks 15 of 288 such names (5.2%), against 94 of 4,075 (2.3%) for names it knows. |
| - **The full model leaks more spans.** On prose, 6.7% of its gold spans (every label) keep a byte, |
| against 1.0% for the narrow model on its identifier labels. On PIIMB's nemotron-pii it masks 99.3% |
| of identifier characters and 92.3% of context characters. |
| - **The full model on identifier benchmarks:** its context spans read as false positives there (see |
| the benchmark table). Choose the scope by what you need gone, not by those numbers. |
| - **The release candidate trails the previous one on Nemotron-PII's own text.** Its held-out prose |
| span leak under `all` is 6.6% against 5.9%, and its `secrets` prose F2 0.9772 against 0.9831: the |
| training mix gave Nemotron-PII 65% of the main phase instead of 75% to gain on PIIMB's other tasks. |
| - **The context is ±4 tokens.** A cue further away than that (a column header, a key two fields |
| back) does not reach the model. |
| - **Segments restrict where hits land, not what the model reads** (see API). |
| - **A secret is exact only after a key that names it,** or in a known format. A bare password |
| anywhere else is the model's guess. |
| |
| ## The embedded model |
| |
| `cargo build` embeds the model named in `weights.lock`'s `[model]` section (repo, commit sha, file, |
| sha256 and size), read by `build.rs`. `Model::embedded()` returns it. |
| |
| - **First build**: downloads the file with `curl` from |
| `https://huggingface.co/{repo}/resolve/{revision}/{file}` into a cache shared by every build. It |
| checks the size and sha256, and only then moves the file into place. |
| - **Later builds**: reuse the cached file offline. |
| - **Cache location**: `CYNCH_PIITAG_CACHE`, else `$XDG_CACHE_HOME/cynch-piitag`, else |
| `~/.cache/cynch-piitag`. There is one file per model, named by its sha256. |
| |
| Other ways to build: |
| |
| - `CYNCH_PIITAG_MODEL=<path>` embeds a local `.piitag` instead. A build warning says so. The build refuses a file |
| that is not a `.piitag`. |
| - `--no-default-features` (the `embedded-model` feature off) and docs.rs builds embed nothing. |
| `Model::embedded()` then returns `Error::NotEmbedded` with the reason, and `Model::load(path)` |
| reads a model file at run time. |
| - A lock whose download fails, or that lacks its section, is a build error that names the |
| remedy, never a quiet fallback. |
| |
| `weights.lock` is written by `piitag-train publish lock`; `fixtures/weights.lock.example` shows |
| its shape. It pins commits rather than a branch, so a |
| later upload never changes what an existing checkout embeds. |
| |
| ## The model file (`.piitag`) |
| |
| One file holds everything: |
| |
| - the magic `PIITAG\0\1` (format version 1); |
| - a u32 header length, then a compact JSON header (shape, types, `pii_q`, `gate_q`, quantization |
| exponents, and the array table); |
| - 25 little-endian arrays, 64-byte aligned. These are the feature table, the layers, the cascade gate, |
| and last the lexical prior table (`prior`). |
| |
| The Python writer is `train/src/piitag_train/intattn.py`; the Rust reader is `src/attn_file.rs`. |
| |
| **The shipped models** share one net: d 16 with 2 attention heads, feed-forward 32 and a head |
| layer of 16, over a feature table 64 wide. They differ in the table, the prior and the type head. |
| |
| - **Narrow:** 2,101,607 parameters in a 4,139,490-byte file: a 2^15-row table (2,097,152 |
| parameters), 34 types and a 2,034,594-byte prior (key version 1). Its operating point is `pii_q` |
| −1160 and its gate `gate_q` −3270. |
| - **Full:** 529,356 parameters in a 5,663,354-byte file: a 2^13-row table (524,288 parameters) and |
| 55 types. The file also carries a 5,130,682-byte English prior (key version 2), which is a word |
| table, not counted as parameters. Its operating point is `pii_q` −273 and its gate `gate_q` −703. |
| Its header has `"id_rules": "defer"`. |
| |
| **The cascade gate** is a small net (16 → 16 → 1) on each token's embedding. Within a record, an |
| 8-token block with no token at or above `gate_q` skips attention, the feed-forward layer and the |
| heads. The tokens it keeps are scored bit-identically to the ungated model. `gate_q` is chosen at |
| 99.9% recall of the model's identifier redactions. |
|
|
| **The neighbour gate** (optional; `train/README.md` `fit-ngate`) replaces the cascade in a file that |
| carries it (`ngate0.w`, `ngate0.b`, `ngate1.w`, `ngate1.b`, `ngate_t` after the prior). Per token it |
| reads its own embedding and its two neighbours' (48 → 16 → one output per type and one for any type). |
| A token none of the label set's outputs keeps skips q, attention, the feed-forward layer and the |
| heads, and scores as skipped. Every output's threshold keeps 99.9% of its redacted validation tokens. |
| A kept token scores exactly as with no gate (`tests/ngate.rs`). A file carries one gate, not both. |
|
|
| ## Training |
|
|
| `train/README.md` rebuilds both models from a clean checkout: the data, the teacher and the |
| distilled student, the gate, the export, the validation that chose them, the scorecard and the |
| upload. `train/recipe_final.json` (narrow) and `train/recipe.json` (the release candidate, whose recipe is |
| the full model's plus an 8-group type head and a prose anneal) hold every command and setting. |
|
|
| ## Training data and attribution |
|
|
| - [`nvidia/Nemotron-PII`](https://huggingface.co/datasets/nvidia/Nemotron-PII) |
| ([CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/)), labels cleaned. The narrow model |
| learns its identifier labels; the full model learns every label. |
| - A JSON-rendered mix of it (10% of training tokens). |
| - Synthetic hard cases generated by the training code: role words before names, capitalized |
| non-names, non-PII values, and credentials after their keys. |
| - 12.5% of the tokens from the train splits of CoNLL-2003 (Reuters RCV1), OntoNotes 5.0 |
| (LDC2013T19) and the Text Anonymization Benchmark (ECHR judgments, Pilán et al. 2022), labels |
| mapped and cleaned. These corpora restrict redistributing their text; each model file contains |
| none of it. |
| - The full model and the release candidate, 25% of its tokens in all (`train/recipe.json`): those three, plus the |
| train splits of |
| [`ai4privacy/open-pii-masking-1m`](https://huggingface.co/datasets/ai4privacy/open-pii-masking-1m) |
| (OpenPII-1M, 23 languages, [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), © Ai4Privacy |
| / Ai Suisse SA), privy (synthetic API traffic, relabelled by key), MAPA (EU legal text) and |
| Gretel's English PII set, and synthetic infrastructure ids in logs and headers (2%). |
|
|
| The lexical prior comes from: |
|
|
| - ENABLE (public domain); |
| - the OpenSubtitles 2018 word counts of |
| [`hermitdave/FrequencyWords`](https://github.com/hermitdave/FrequencyWords) |
| (content [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/)); |
| - the Social Security Administration's baby names; |
| - the Census 2000 surname table. |
|
|
| Each source is sha256-pinned. Only a one-byte word/frequency/popularity code per key reaches the |
| model file. |
|
|
| The weights are published under [CC-BY-SA-4.0](https://creativecommons.org/licenses/by-sa/4.0/). |
|
|
