Instructions to use EasyOctopus/german-support-pii-model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use EasyOctopus/german-support-pii-model with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="EasyOctopus/german-support-pii-model")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("EasyOctopus/german-support-pii-model") model = AutoModelForTokenClassification.from_pretrained("EasyOctopus/german-support-pii-model", device_map="auto") - Notebooks
- Google Colab
- Kaggle
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForTokenClassification
tokenizer = AutoTokenizer.from_pretrained("EasyOctopus/german-support-pii-model")
model = AutoModelForTokenClassification.from_pretrained("EasyOctopus/german-support-pii-model", device_map="auto")German support PII experimental checkpoint
Detect personally identifiable information (PII) with beginning, inside and outside token labels. The example returns Unicode offsets and redacted text. It processes overflow in non-overlapping 256-token windows. Boundary context and missed spans limit privacy guarantees.
Selected variant scenarios-hard by the frozen validation rule. All parameters were updated on one NVIDIA A100-SXM4-80GB. See provenance.json for initialization, label policy, data revisions, seed and controlled alternatives.
| Candidate | Complete spans | Collateral characters | Flagged negatives |
|---|---|---|---|
| replay | 214/256 | 170/10,805 (1.57%) | 16/48 |
| scenarios | 255/256 | 292/10,805 (2.70%) | 17/48 |
| scenarios-hard | 255/256 | 55/10,805 (0.51%) | 2/48 |
| prior | 214/256 | 178/10,805 (1.65%) | 16/48 |
| live-adapted | 178/256 | 245/10,805 (2.27%) | 46/48 |
| pilot-union | 240/256 | 275/10,805 (2.55%) | 46/48 |
Replay continues training on previously seen synthetic data. Scenarios adds novel positive examples. Scenarios-hard also swaps in hard negatives. Prior is the unmodified starting checkpoint. Live-adapted combines a CPU model, rules and a small tagger. Pilot-union combines that baseline with the starting checkpoint.
On the earlier independent suite, the selected checkpoint fully protected 94/108 spans, compared with 86 for its initializer. On the earlier confirmation suite, the selected checkpoint fully protected 64/70 spans, compared with 63 for its initializer.
The training messages and newly generated test messages are fictional. Repeated slot draws within a template are correlated. Split families are disjoint but use the same authoring process. These results do not measure protection or dictation quality on customer messages.
Download and use
Fine-tuned by Mahir Isikli (EasyOctopus). The companion dataset is EasyOctopus/german-support-pii-data.
Load the tokenizer and classification head with standard Transformers. The model runs locally on a CPU or GPU.
from transformers import AutoModelForTokenClassification, AutoTokenizer
repo_id = "EasyOctopus/german-support-pii-model"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForTokenClassification.from_pretrained(repo_id).eval()
For the exact decoder used in the evaluation, download the repository and run the included example. This needs Python 3.11 or later, PyTorch and Transformers. It sends no message text to the Hub.
import subprocess
import sys
from pathlib import Path
from huggingface_hub import snapshot_download
folder = Path(snapshot_download("EasyOctopus/german-support-pii-model", local_dir="german-support-pii"))
subprocess.run([sys.executable, str(folder / "infer.py"), "Bitte schickt die Bestätigung an beispiel@example.org."], check=True)
subprocess.run([sys.executable, str(folder / "infer.py"), "--self-test"], check=True)
See LICENSE and NOTICE for source attribution. The test predictions and validation selection are documented in eval_results.json and provenance.json.
Retrain the controlled variants
Download this repository with snapshot_download as shown above. From the downloaded directory, run the commands below. The script fetches only the training and validation files it needs from the companion dataset. It verifies the initializer and input fingerprints before fitting. Use --data-dir to use a local dataset checkout.
uv run --script training/retrain.py --prepare-only --output smoke-results
CUDA_VISIBLE_DEVICES=0 uv run --script training/retrain.py --variant all --output ablation-results
The PII initializer is included in training/initial/. Its Apache-2.0 provenance is in training/initial-provenance.json. Initial weight SHA-256: a30303f35671526f97590f30206469ac286b014d11e74133123a758883e993dc. Each variant uses seed 20261118, 3 epochs, batch size 24 and AdamW with a constant learning rate of 1.5e-05, weight decay 0.01 and gradient clipping at norm 1. Training uses BF16 autocast when the card supports it. The script retains every final-epoch checkpoint and records the validation-selected variant. CUDA, hardware and library differences can affect numerical results. The original run used PyTorch 2.6.0 with CUDA 12.4 and Transformers 4.49.0.
- Downloads last month
- 19
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="EasyOctopus/german-support-pii-model")