BrewSLM export 37: sensitive-data span tagger, run #143 (deberta-v3-base, 30 types)

README.md

@@ -0,0 +1,155 @@
+---
+library_name: transformers
+pipeline_tag: token-classification
+base_model: microsoft/deberta-v3-base
+tags:
+- token-classification
+- span-extraction
+- brewslm
+license: mit
+language:
+- en
+---
+
+# Sensitive-data tagger (PCI / GDPR / HIPAA + custom) span tagger
+
+A token-classification model that finds spans of text and labels each with a type, fine-tuned from `microsoft/deberta-v3-base`.
+Trained with BrewSLM (project “Sensitive-data tagger (PCI / GDPR / HIPAA + custom)”, run #143).
+
+## Model
+
+- **Base model:** `microsoft/deberta-v3-base`
+- **Task:** token classification (BIO tags → character spans)
+- **Window:** 512 tokens (the tokenizer's `model_max_length`); longer texts are read in overlapping windows
+
+## Entity types
+
+30 types: `address`, `api_key`, `bank_account`, `card_expiry`, `credit_card`, `customer_id`, `cvv`, `date_of_birth`, `device_id`, `diagnosis`, `email`, `employee_id`, `health_plan_id`, `iban`, `ip_address`, `licence_number`, `license_plate`, `medical_record_number`, `medication`, `national_id`, `passport`, `person_name`, `phone`, `provider_name`, `routing_number`, `tax_id`, `ticket_id`, `transaction_id`, `username`, `visit_date`.
+Labels are BIO: `O`, then `B-<type>` / `I-<type>` per type.
+
+## Training data
+
+- **Training rows:** 1620 (1612 with at least one span)
+- **Validation rows:** 202
+- **Data kinds:** synthetic
+- **Row sources:** `passage_labels:cloud:anthropic:claude-sonnet-5-5` 919; `teacher_llm` 335; `passage_labels:ollama` 223; `passage_labels:cloud:anthropic:claude-haiku-4-5-20251001` 101; `document_labels:ollama` 32; `teacher_llm+regex` 10
+- **Labelled by:** `cloud:anthropic:claude-sonnet-5-5` 1620
+- **Augmentation:** 1616 OCR-noise copies (character noise rate 0.1)
+- **Epochs:** 5
+- **Seed selection:** Picked seed 45 (run #143) of 3 by validation span F1 (tuned thresholds) (higher is better; values: seed 45 0.9467, seed 46 0.9452, seed 47 0.9437); test examples, the answer key and external benchmarks were not used to pick.
+
+## Evaluation
+
+Strict span match: a prediction counts only when its type, start and end all equal a reference span.
+
+### Answer key (44 texts)
+
+Reference spans written by `cloud:anthropic:claude-sonnet-5-5`.
+
+Human review: 11 of 44 rows checked by a person (1 corrected). On the checked rows: precision 0.90 · recall 0.95 · F1 0.92 (11 texts).
+
+Overall: precision 0.97 · recall 0.98 · F1 0.97
+
+0 of 24 types have enough reference spans to show a 0.05 F1 difference; the others can only show 0.82 or more (medical_record_number: 1 span, needs 152).
+
+| Type | Precision | Recall | F1 | Gold spans |
+|---|---|---|---|---|
+| `address` | 1.00 | 1.00 | 1.00 | 4 |
+| `api_key` | 1.00 | 1.00 | 1.00 | 2 |
+| `bank_account` | 0.62 | 1.00 | 0.77 | 5 |
+| `card_expiry` | 1.00 | 1.00 | 1.00 | 16 |
+| `credit_card` | 1.00 | 0.91 | 0.95 | 23 |
+| `cvv` | 1.00 | 1.00 | 1.00 | 9 |
+| `date_of_birth` | 0.70 | 1.00 | 0.82 | 7 |
+| `diagnosis` | 1.00 | 1.00 | 1.00 | 16 |
+| `email` | 1.00 | 1.00 | 1.00 | 27 |
+| `employee_id` | 1.00 | 1.00 | 1.00 | 3 |
+| `health_plan_id` | 1.00 | 1.00 | 1.00 | 7 |
+| `iban` | 1.00 | 1.00 | 1.00 | 7 |
+| `ip_address` | 1.00 | 1.00 | 1.00 | 6 |
+| `medical_record_number` | 1.00 | 1.00 | 1.00 | 1 |
+| `medication` | 0.86 | 0.90 | 0.88 | 20 |
+| `national_id` | 1.00 | 1.00 | 1.00 | 13 |
+| `passport` | 1.00 | 1.00 | 1.00 | 6 |
+| `person_name` | 1.00 | 1.00 | 1.00 | 37 |
+| `phone` | 1.00 | 1.00 | 1.00 | 25 |
+| `provider_name` | 1.00 | 1.00 | 1.00 | 23 |
+| `routing_number` | 0.00 | 0.00 | 0.00 | 3 |
+| `ticket_id` | 1.00 | 1.00 | 1.00 | 9 |
+| `username` | 1.00 | 1.00 | 1.00 | 1 |
+| `visit_date` | 0.94 | 1.00 | 0.97 | 16 |
+
+### Test examples (44 texts)
+
+Overall: precision 0.96 · recall 0.98 · F1 0.97
+
+0 of 24 types have enough reference spans to show a 0.05 F1 difference; the others can only show 1.00 or more (medical_record_number: 1 span, needs 152).
+
+| Type | Precision | Recall | F1 | Gold spans |
+|---|---|---|---|---|
+| `address` | 1.00 | 1.00 | 1.00 | 4 |
+| `api_key` | 1.00 | 1.00 | 1.00 | 2 |
+| `bank_account` | 0.62 | 1.00 | 0.77 | 5 |
+| `card_expiry` | 1.00 | 1.00 | 1.00 | 16 |
+| `credit_card` | 0.95 | 0.87 | 0.91 | 23 |
+| `cvv` | 1.00 | 1.00 | 1.00 | 9 |
+| `date_of_birth` | 0.70 | 1.00 | 0.82 | 7 |
+| `diagnosis` | 1.00 | 1.00 | 1.00 | 16 |
+| `email` | 1.00 | 1.00 | 1.00 | 27 |
+| `employee_id` | 1.00 | 1.00 | 1.00 | 3 |
+| `health_plan_id` | 1.00 | 1.00 | 1.00 | 7 |
+| `iban` | 1.00 | 1.00 | 1.00 | 7 |
+| `ip_address` | 1.00 | 1.00 | 1.00 | 6 |
+| `medical_record_number` | 1.00 | 1.00 | 1.00 | 1 |
+| `medication` | 0.86 | 0.90 | 0.88 | 20 |
+| `national_id` | 1.00 | 1.00 | 1.00 | 13 |
+| `passport` | 1.00 | 1.00 | 1.00 | 6 |
+| `person_name` | 1.00 | 1.00 | 1.00 | 37 |
+| `phone` | 1.00 | 1.00 | 1.00 | 25 |
+| `provider_name` | 1.00 | 1.00 | 1.00 | 23 |
+| `routing_number` | 0.00 | 0.00 | 0.00 | 2 |
+| `ticket_id` | 1.00 | 1.00 | 1.00 | 9 |
+| `username` | 1.00 | 1.00 | 1.00 | 1 |
+| `visit_date` | 0.94 | 1.00 | 0.97 | 16 |
+
+## External benchmarks
+
+Labelled sets made by someone else, mapped onto this model's types; only the mapped types are scored (a prediction of another type is neither a hit nor a false positive).
+
+| Benchmark | Source | Licence | Texts | Types scored | Precision | Recall | F1 |
+|---|---|---|---|---|---|---|---|
+| CoNLL-2003 people (news, human-labelled) | `eriktks/conll2003:test` | other | 400 | person_name | 0.93 | 0.70 | 0.80 |
+| Gretel PII (synthetic, independent) | `gretelai/gretel-pii-masking-en-v1:test` | apache-2.0 | 500 | address, api_key, bank_account, credit_card, date_of_birth, email, employee_id, health_plan_id, ip_address, license_plate, medical_record_number, national_id, person_name, phone | 0.76 | 0.80 | 0.78 |
+| CoNLL-2003 people, large (validation, 2000 rows) | `eriktks/conll2003:validation` | other | 2000 | person_name | 0.93 | 0.74 | 0.83 |
+
+## Usage
+
+```python
+from transformers import pipeline
+
+tagger = pipeline("token-classification", model="path/to/model", aggregation_strategy="simple", stride=128)
+
+def find_spans(text):
+    spans = []
+    for ent in tagger(text):
+        start, end = ent["start"], ent["end"]
+        # The tokenizer can attach the preceding space to a span; trim it.
+        while start < end and text[start].isspace():
+            start += 1
+        spans.append({"type": ent["entity_group"], "start": start, "end": end,
+                      "text": text[start:end], "score": float(ent["score"])})
+    return spans
+```
+
+`stride=128` matters: without it the pipeline cuts a text at the 512-token window and never tags anything after it. With it, long texts are read in overlapping windows.
+
+`thresholds.json` holds per-type minimum scores (all 0.0 = keep every predicted span) and a `recall_bias`. To favour precision for a type, drop its spans whose score is under that type's value. BrewSLM's redact endpoint and scored decoder read this file; there the score is the span's mean per-token probability of its type, and `recall_bias` (0–1) is subtracted from each token's `O` probability so borderline tokens are tagged (more recall, less precision).
+
+## Limitations
+
+- Scores come from small held-out sets — 44 texts (answer key), 44 texts (test examples) — so expect a few points of noise either way.
+- Types with fewer than 10 reference spans in an evaluation (`address`, `api_key`, `bank_account`, `cvv`, `date_of_birth`, `employee_id`, `health_plan_id`, `iban`, `ip_address`, `medical_record_number`, `passport`, `routing_number`, `ticket_id`, `username`) have unreliable per-type scores.
+- The training data is synthetic or model-labelled, not human-annotated; real documents with other layouts, formats or languages may score lower.
+- It finds what it was trained to find: entity types outside the list above are never tagged.
+- On the Gretel PII benchmark (103 employee_id, 55 license_plate, 17 bank_account reference spans — directional numbers): employee_id precision 0.95 / recall 0.70, license_plate F1 0.69, bank_account precision 0.50 (false positives are mostly routing numbers, device identifiers and timestamps that Gretel leaves unlabelled). Compared with the previous export of this project (run #135), employee_id recall is 0.06 lower and bank_account precision 0.21 lower; employee_id precision is 0.15 higher and license_plate 0.10 higher. Plate false positives are still UK National Insurance numbers, medical record numbers and postcodes.
+- Benchmark hygiene: 32 Gretel gold values (identifiers, plates, street names) had been copied verbatim into earlier training rows by the data-seeding prompts (591 occurrences in 248 rows). They were replaced with invented values of the same shape before this run trained, so its Gretel figures are measured on data the model never saw. The previous export's employee_id figure was overstated by an estimated 0.03–0.15 F1 for that reason.

config.json

@@ -0,0 +1,165 @@
+{
+  "architectures": [
+    "DebertaV2ForTokenClassification"
+  ],
+  "attention_probs_dropout_prob": 0.1,
+  "bos_token_id": null,
+  "dtype": "float32",
+  "eos_token_id": null,
+  "hidden_act": "gelu",
+  "hidden_dropout_prob": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "O",
+    "1": "B-address",
+    "2": "I-address",
+    "3": "B-api_key",
+    "4": "I-api_key",
+    "5": "B-bank_account",
+    "6": "I-bank_account",
+    "7": "B-card_expiry",
+    "8": "I-card_expiry",
+    "9": "B-credit_card",
+    "10": "I-credit_card",
+    "11": "B-customer_id",
+    "12": "I-customer_id",
+    "13": "B-cvv",
+    "14": "I-cvv",
+    "15": "B-date_of_birth",
+    "16": "I-date_of_birth",
+    "17": "B-device_id",
+    "18": "I-device_id",
+    "19": "B-diagnosis",
+    "20": "I-diagnosis",
+    "21": "B-email",
+    "22": "I-email",
+    "23": "B-employee_id",
+    "24": "I-employee_id",
+    "25": "B-health_plan_id",
+    "26": "I-health_plan_id",
+    "27": "B-iban",
+    "28": "I-iban",
+    "29": "B-ip_address",
+    "30": "I-ip_address",
+    "31": "B-licence_number",
+    "32": "I-licence_number",
+    "33": "B-license_plate",
+    "34": "I-license_plate",
+    "35": "B-medical_record_number",
+    "36": "I-medical_record_number",
+    "37": "B-medication",
+    "38": "I-medication",
+    "39": "B-national_id",
+    "40": "I-national_id",
+    "41": "B-passport",
+    "42": "I-passport",
+    "43": "B-person_name",
+    "44": "I-person_name",
+    "45": "B-phone",
+    "46": "I-phone",
+    "47": "B-provider_name",
+    "48": "I-provider_name",
+    "49": "B-routing_number",
+    "50": "I-routing_number",
+    "51": "B-tax_id",
+    "52": "I-tax_id",
+    "53": "B-ticket_id",
+    "54": "I-ticket_id",
+    "55": "B-transaction_id",
+    "56": "I-transaction_id",
+    "57": "B-username",
+    "58": "I-username",
+    "59": "B-visit_date",
+    "60": "I-visit_date"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "B-address": 1,
+    "B-api_key": 3,
+    "B-bank_account": 5,
+    "B-card_expiry": 7,
+    "B-credit_card": 9,
+    "B-customer_id": 11,
+    "B-cvv": 13,
+    "B-date_of_birth": 15,
+    "B-device_id": 17,
+    "B-diagnosis": 19,
+    "B-email": 21,
+    "B-employee_id": 23,
+    "B-health_plan_id": 25,
+    "B-iban": 27,
+    "B-ip_address": 29,
+    "B-licence_number": 31,
+    "B-license_plate": 33,
+    "B-medical_record_number": 35,
+    "B-medication": 37,
+    "B-national_id": 39,
+    "B-passport": 41,
+    "B-person_name": 43,
+    "B-phone": 45,
+    "B-provider_name": 47,
+    "B-routing_number": 49,
+    "B-tax_id": 51,
+    "B-ticket_id": 53,
+    "B-transaction_id": 55,
+    "B-username": 57,
+    "B-visit_date": 59,
+    "I-address": 2,
+    "I-api_key": 4,
+    "I-bank_account": 6,
+    "I-card_expiry": 8,
+    "I-credit_card": 10,
+    "I-customer_id": 12,
+    "I-cvv": 14,
+    "I-date_of_birth": 16,
+    "I-device_id": 18,
+    "I-diagnosis": 20,
+    "I-email": 22,
+    "I-employee_id": 24,
+    "I-health_plan_id": 26,
+    "I-iban": 28,
+    "I-ip_address": 30,
+    "I-licence_number": 32,
+    "I-license_plate": 34,
+    "I-medical_record_number": 36,
+    "I-medication": 38,
+    "I-national_id": 40,
+    "I-passport": 42,
+    "I-person_name": 44,
+    "I-phone": 46,
+    "I-provider_name": 48,
+    "I-routing_number": 50,
+    "I-tax_id": 52,
+    "I-ticket_id": 54,
+    "I-transaction_id": 56,
+    "I-username": 58,
+    "I-visit_date": 60,
+    "O": 0
+  },
+  "layer_norm_eps": 1e-07,
+  "legacy": true,
+  "max_position_embeddings": 512,
+  "max_relative_positions": -1,
+  "model_type": "deberta-v2",
+  "norm_rel_ebd": "layer_norm",
+  "num_attention_heads": 12,
+  "num_hidden_layers": 12,
+  "pad_token_id": 0,
+  "pooler_dropout": 0,
+  "pooler_hidden_act": "gelu",
+  "pooler_hidden_size": 768,
+  "pos_att_type": [
+    "p2c",
+    "c2p"
+  ],
+  "position_biased_input": false,
+  "position_buckets": 256,
+  "relative_attention": true,
+  "share_att_key": true,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.2.0",
+  "type_vocab_size": 0,
+  "use_cache": false,
+  "vocab_size": 128100
+}

label_space.json

@@ -0,0 +1,97 @@
+{
+  "labels": [
+    "O",
+    "B-address",
+    "I-address",
+    "B-api_key",
+    "I-api_key",
+    "B-bank_account",
+    "I-bank_account",
+    "B-card_expiry",
+    "I-card_expiry",
+    "B-credit_card",
+    "I-credit_card",
+    "B-customer_id",
+    "I-customer_id",
+    "B-cvv",
+    "I-cvv",
+    "B-date_of_birth",
+    "I-date_of_birth",
+    "B-device_id",
+    "I-device_id",
+    "B-diagnosis",
+    "I-diagnosis",
+    "B-email",
+    "I-email",
+    "B-employee_id",
+    "I-employee_id",
+    "B-health_plan_id",
+    "I-health_plan_id",
+    "B-iban",
+    "I-iban",
+    "B-ip_address",
+    "I-ip_address",
+    "B-licence_number",
+    "I-licence_number",
+    "B-license_plate",
+    "I-license_plate",
+    "B-medical_record_number",
+    "I-medical_record_number",
+    "B-medication",
+    "I-medication",
+    "B-national_id",
+    "I-national_id",
+    "B-passport",
+    "I-passport",
+    "B-person_name",
+    "I-person_name",
+    "B-phone",
+    "I-phone",
+    "B-provider_name",
+    "I-provider_name",
+    "B-routing_number",
+    "I-routing_number",
+    "B-tax_id",
+    "I-tax_id",
+    "B-ticket_id",
+    "I-ticket_id",
+    "B-transaction_id",
+    "I-transaction_id",
+    "B-username",
+    "I-username",
+    "B-visit_date",
+    "I-visit_date"
+  ],
+  "entity_types": [
+    "address",
+    "api_key",
+    "bank_account",
+    "card_expiry",
+    "credit_card",
+    "customer_id",
+    "cvv",
+    "date_of_birth",
+    "device_id",
+    "diagnosis",
+    "email",
+    "employee_id",
+    "health_plan_id",
+    "iban",
+    "ip_address",
+    "licence_number",
+    "license_plate",
+    "medical_record_number",
+    "medication",
+    "national_id",
+    "passport",
+    "person_name",
+    "phone",
+    "provider_name",
+    "routing_number",
+    "tax_id",
+    "ticket_id",
+    "transaction_id",
+    "username",
+    "visit_date"
+  ]
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fdefab12891f005e8d0ac933ff871764fcd074d74bd1231baabb3866b5bad7a1
+size 735538204

thresholds.json

@@ -0,0 +1,451 @@
+{
+  "version": 1,
+  "score": "mean over a span's tokens of P(B-type) + P(I-type)",
+  "default_min_score": 0.0,
+  "recall_bias": 0.0,
+  "per_type": {
+    "address": 0.9,
+    "api_key": 0.0,
+    "bank_account": 0.6,
+    "card_expiry": 0.0,
+    "credit_card": 0.7,
+    "customer_id": 0.7,
+    "cvv": 0.6,
+    "date_of_birth": 0.8,
+    "device_id": 0.9,
+    "diagnosis": 0.0,
+    "email": 0.0,
+    "employee_id": 0.9,
+    "health_plan_id": 0.5,
+    "iban": 0.9,
+    "ip_address": 0.0,
+    "licence_number": 0.0,
+    "license_plate": 0.0,
+    "medical_record_number": 0.8,
+    "medication": 0.0,
+    "national_id": 0.7,
+    "passport": 0.0,
+    "person_name": 0.0,
+    "phone": 0.7,
+    "provider_name": 0.9,
+    "routing_number": 0.0,
+    "tax_id": 0.0,
+    "ticket_id": 0.9,
+    "transaction_id": 0.6,
+    "username": 0.0,
+    "visit_date": 0.9
+  },
+  "recall_floors": {},
+  "tuned": {
+    "split": "val",
+    "rows": 202,
+    "at": "2026-10-17T15:09:36.228497+00:00",
+    "micro_f1_before": 0.9288,
+    "micro_f1_after": 0.9467,
+    "micro_before": {
+      "precision": 0.9114,
+      "recall": 0.9468,
+      "f1": 0.9288
+    },
+    "micro_after": {
+      "precision": 0.9481,
+      "recall": 0.9454,
+      "f1": 0.9467
+    },
+    "recall_bias": 0.0,
+    "recall_bias_suggestion": null,
+    "floors": {},
+    "report": {
+      "address": {
+        "threshold": 0.9,
+        "f1_before": 0.8873,
+        "f1_after": 0.9065,
+        "precision_before": 0.8514,
+        "precision_after": 0.8873,
+        "recall_before": 0.9265,
+        "recall_after": 0.9265,
+        "support": 68,
+        "predicted": 74,
+        "floor": null,
+        "floor_status": null
+      },
+      "api_key": {
+        "threshold": 0.0,
+        "f1_before": 1.0,
+        "f1_after": 1.0,
+        "precision_before": 1.0,
+        "precision_after": 1.0,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 4,
+        "predicted": 4,
+        "floor": null,
+        "floor_status": null
+      },
+      "bank_account": {
+        "threshold": 0.6,
+        "f1_before": 0.8829,
+        "f1_after": 0.9333,
+        "precision_before": 0.8033,
+        "precision_after": 0.8909,
+        "recall_before": 0.98,
+        "recall_after": 0.98,
+        "support": 50,
+        "predicted": 61,
+        "floor": null,
+        "floor_status": null
+      },
+      "card_expiry": {
+        "threshold": 0.0,
+        "f1_before": 1.0,
+        "f1_after": 1.0,
+        "precision_before": 1.0,
+        "precision_after": 1.0,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 28,
+        "predicted": 28,
+        "floor": null,
+        "floor_status": null
+      },
+      "credit_card": {
+        "threshold": 0.7,
+        "f1_before": 0.9103,
+        "f1_after": 0.9342,
+        "precision_before": 0.8987,
+        "precision_after": 0.9467,
+        "recall_before": 0.9221,
+        "recall_after": 0.9221,
+        "support": 77,
+        "predicted": 79,
+        "floor": null,
+        "floor_status": null
+      },
+      "customer_id": {
+        "threshold": 0.7,
+        "f1_before": 0.9434,
+        "f1_after": 0.9804,
+        "precision_before": 0.8929,
+        "precision_after": 0.9615,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 25,
+        "predicted": 28,
+        "floor": null,
+        "floor_status": null
+      },
+      "cvv": {
+        "threshold": 0.6,
+        "f1_before": 0.9333,
+        "f1_after": 0.9655,
+        "precision_before": 0.9333,
+        "precision_after": 1.0,
+        "recall_before": 0.9333,
+        "recall_after": 0.9333,
+        "support": 15,
+        "predicted": 15,
+        "floor": null,
+        "floor_status": null
+      },
+      "date_of_birth": {
+        "threshold": 0.8,
+        "f1_before": 0.8842,
+        "f1_after": 0.9333,
+        "precision_before": 0.875,
+        "precision_after": 0.9767,
+        "recall_before": 0.8936,
+        "recall_after": 0.8936,
+        "support": 47,
+        "predicted": 48,
+        "floor": null,
+        "floor_status": null
+      },
+      "device_id": {
+        "threshold": 0.9,
+        "f1_before": 0.9,
+        "f1_after": 0.9474,
+        "precision_before": 0.9,
+        "precision_after": 1.0,
+        "recall_before": 0.9,
+        "recall_after": 0.9,
+        "support": 10,
+        "predicted": 10,
+        "floor": null,
+        "floor_status": null
+      },
+      "diagnosis": {
+        "threshold": 0.0,
+        "f1_before": 0.8667,
+        "f1_after": 0.8667,
+        "precision_before": 0.8667,
+        "precision_after": 0.8667,
+        "recall_before": 0.8667,
+        "recall_after": 0.8667,
+        "support": 30,
+        "predicted": 30,
+        "floor": null,
+        "floor_status": null
+      },
+      "email": {
+        "threshold": 0.0,
+        "f1_before": 0.9756,
+        "f1_after": 0.9756,
+        "precision_before": 0.9756,
+        "precision_after": 0.9756,
+        "recall_before": 0.9756,
+        "recall_after": 0.9756,
+        "support": 41,
+        "predicted": 41,
+        "floor": null,
+        "floor_status": null
+      },
+      "employee_id": {
+        "threshold": 0.9,
+        "f1_before": 0.9688,
+        "f1_after": 0.9947,
+        "precision_before": 0.949,
+        "precision_after": 1.0,
+        "recall_before": 0.9894,
+        "recall_after": 0.9894,
+        "support": 94,
+        "predicted": 98,
+        "floor": null,
+        "floor_status": null
+      },
+      "health_plan_id": {
+        "threshold": 0.5,
+        "f1_before": 0.96,
+        "f1_after": 0.9796,
+        "precision_before": 0.96,
+        "precision_after": 1.0,
+        "recall_before": 0.96,
+        "recall_after": 0.96,
+        "support": 25,
+        "predicted": 25,
+        "floor": null,
+        "floor_status": null
+      },
+      "iban": {
+        "threshold": 0.9,
+        "f1_before": 0.9259,
+        "f1_after": 0.9434,
+        "precision_before": 0.8929,
+        "precision_after": 0.9259,
+        "recall_before": 0.9615,
+        "recall_after": 0.9615,
+        "support": 26,
+        "predicted": 28,
+        "floor": null,
+        "floor_status": null
+      },
+      "ip_address": {
+        "threshold": 0.0,
+        "f1_before": 1.0,
+        "f1_after": 1.0,
+        "precision_before": 1.0,
+        "precision_after": 1.0,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 5,
+        "predicted": 5,
+        "floor": null,
+        "floor_status": null
+      },
+      "licence_number": {
+        "threshold": 0.0,
+        "f1_before": 1.0,
+        "f1_after": 1.0,
+        "precision_before": 1.0,
+        "precision_after": 1.0,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 2,
+        "predicted": 2,
+        "floor": null,
+        "floor_status": null
+      },
+      "license_plate": {
+        "threshold": 0.0,
+        "f1_before": 0.9686,
+        "f1_after": 0.9686,
+        "precision_before": 0.9625,
+        "precision_after": 0.9625,
+        "recall_before": 0.9747,
+        "recall_after": 0.9747,
+        "support": 79,
+        "predicted": 80,
+        "floor": null,
+        "floor_status": null
+      },
+      "medical_record_number": {
+        "threshold": 0.8,
+        "f1_before": 0.907,
+        "f1_after": 0.9398,
+        "precision_before": 0.8864,
+        "precision_after": 0.9512,
+        "recall_before": 0.9286,
+        "recall_after": 0.9286,
+        "support": 42,
+        "predicted": 44,
+        "floor": null,
+        "floor_status": null
+      },
+      "medication": {
+        "threshold": 0.0,
+        "f1_before": 0.8824,
+        "f1_after": 0.8824,
+        "precision_before": 0.8824,
+        "precision_after": 0.8824,
+        "recall_before": 0.8824,
+        "recall_after": 0.8824,
+        "support": 17,
+        "predicted": 17,
+        "floor": null,
+        "floor_status": null
+      },
+      "national_id": {
+        "threshold": 0.7,
+        "f1_before": 0.878,
+        "f1_after": 0.9474,
+        "precision_before": 0.8182,
+        "precision_after": 0.9474,
+        "recall_before": 0.9474,
+        "recall_after": 0.9474,
+        "support": 19,
+        "predicted": 22,
+        "floor": null,
+        "floor_status": null
+      },
+      "passport": {
+        "threshold": 0.0,
+        "f1_before": 0.9333,
+        "f1_after": 0.9333,
+        "precision_before": 0.875,
+        "precision_after": 0.875,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 7,
+        "predicted": 8,
+        "floor": null,
+        "floor_status": null
+      },
+      "person_name": {
+        "threshold": 0.0,
+        "f1_before": 0.9711,
+        "f1_after": 0.9711,
+        "precision_before": 0.9627,
+        "precision_after": 0.9627,
+        "recall_before": 0.9797,
+        "recall_after": 0.9797,
+        "support": 395,
+        "predicted": 402,
+        "floor": null,
+        "floor_status": null
+      },
+      "phone": {
+        "threshold": 0.7,
+        "f1_before": 0.8649,
+        "f1_after": 0.9057,
+        "precision_before": 0.8727,
+        "precision_after": 0.96,
+        "recall_before": 0.8571,
+        "recall_after": 0.8571,
+        "support": 56,
+        "predicted": 55,
+        "floor": null,
+        "floor_status": null
+      },
+      "provider_name": {
+        "threshold": 0.9,
+        "f1_before": 0.8556,
+        "f1_after": 0.8994,
+        "precision_before": 0.8191,
+        "precision_after": 0.9157,
+        "recall_before": 0.8953,
+        "recall_after": 0.8837,
+        "support": 86,
+        "predicted": 94,
+        "floor": null,
+        "floor_status": null
+      },
+      "routing_number": {
+        "threshold": 0.0,
+        "f1_before": 1.0,
+        "f1_after": 1.0,
+        "precision_before": 1.0,
+        "precision_after": 1.0,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 20,
+        "predicted": 20,
+        "floor": null,
+        "floor_status": null
+      },
+      "tax_id": {
+        "threshold": 0.0,
+        "f1_before": 0.0,
+        "f1_after": 0.0,
+        "precision_before": null,
+        "precision_after": null,
+        "recall_before": null,
+        "recall_after": null,
+        "support": 0,
+        "predicted": 0,
+        "floor": null,
+        "floor_status": null
+      },
+      "ticket_id": {
+        "threshold": 0.9,
+        "f1_before": 0.8706,
+        "f1_after": 0.8916,
+        "precision_before": 0.925,
+        "precision_after": 0.9737,
+        "recall_before": 0.8222,
+        "recall_after": 0.8222,
+        "support": 45,
+        "predicted": 40,
+        "floor": null,
+        "floor_status": null
+      },
+      "transaction_id": {
+        "threshold": 0.6,
+        "f1_before": 0.8525,
+        "f1_after": 0.8814,
+        "precision_before": 0.8125,
+        "precision_after": 0.8667,
+        "recall_before": 0.8966,
+        "recall_after": 0.8966,
+        "support": 29,
+        "predicted": 32,
+        "floor": null,
+        "floor_status": null
+      },
+      "username": {
+        "threshold": 0.0,
+        "f1_before": 0.963,
+        "f1_after": 0.963,
+        "precision_before": 0.9286,
+        "precision_after": 0.9286,
+        "recall_before": 1.0,
+        "recall_after": 1.0,
+        "support": 13,
+        "predicted": 14,
+        "floor": null,
+        "floor_status": null
+      },
+      "visit_date": {
+        "threshold": 0.9,
+        "f1_before": 0.8571,
+        "f1_after": 0.8649,
+        "precision_before": 0.8049,
+        "precision_after": 0.8421,
+        "recall_before": 0.9167,
+        "recall_after": 0.8889,
+        "support": 36,
+        "predicted": 41,
+        "floor": null,
+        "floor_status": null
+      }
+    }
+  }
+}

tokenizer_config.json

@@ -0,0 +1,23 @@
+{
+  "add_prefix_space": true,
+  "backend": "tokenizers",
+  "bos_token": "[CLS]",
+  "cls_token": "[CLS]",
+  "do_lower_case": false,
+  "eos_token": "[SEP]",
+  "extra_special_tokens": [
+    "[PAD]",
+    "[CLS]",
+    "[SEP]"
+  ],
+  "is_local": false,
+  "mask_token": "[MASK]",
+  "model_max_length": 512,
+  "pad_token": "[PAD]",
+  "sep_token": "[SEP]",
+  "split_by_punct": false,
+  "tokenizer_class": "DebertaV2Tokenizer",
+  "unk_id": 3,
+  "unk_token": "[UNK]",
+  "vocab_type": "spm"
+}