Instructions to use EasyOctopus/german-ticket-privacy-filter with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use EasyOctopus/german-ticket-privacy-filter with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="EasyOctopus/german-ticket-privacy-filter")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("EasyOctopus/german-ticket-privacy-filter") model = AutoModelForTokenClassification.from_pretrained("EasyOctopus/german-ticket-privacy-filter", device_map="auto") - Notebooks
- Google Colab
- Kaggle
German ticket privacy filter
A compact contextual redactor trained by Mahir Isikli for German support tickets. It reads an optional support question and its customer reply, then returns private spans in the reply. This checkpoint starts from German Support PII, which descends from the German cased DistilBERT. Digitflow's separate German fine-tune of OpenAI Privacy Filter was evaluated as an external reference. Its weights, outputs and training labels were absent from our training and selection.
Run locally
Download this repository, install PyTorch and Transformers and run the included entry point. The generic Hugging Face token-classification pipeline accepts standalone text. The included entry point implements the question/reply layout, the selected decoder and reply-relative offsets.
hf download EasyOctopus/german-ticket-privacy-filter --local-dir german-ticket-privacy-filter
cd german-ticket-privacy-filter
uv run --no-project --with torch==2.6.0 --with transformers==4.49.0 python runtime.py --model . <<'JSON'
{"context":"Wie lautet Ihre Kundennummer?","text":"Meine Kennung ist KL-04281."}
JSON
The entry point reads JSON lines, loads local files, and returns redacted text and half-open Unicode code-point offsets. Its maximum input is 12,000 reply characters and 512 question characters. At most 80 question subwords are retained. Long replies use overlapping windows, and a token gets the prediction from the window with the larger edge margin. Scores represent model confidence, not a calibrated probability of privacy.
Training
The selected checkpoint used 4,200 newly generated fictional conversations plus 650 records from the earlier public training split. The new development split contains 420 records. Gradient loss applies only to reply tokens. We trained four epochs on one NVIDIA A100-SXM4-80GB with AdamW, seed 181153 and a learning rate of 0.000028. Development selected epoch 2, cutoff 0.55 and within-word expansion set to True. The full weight SHA-256 is 4b11077b8ed8c33af653df2440236091e420ed6bb68be9f3c9a18c68d9ec65f8. Training corpus
Independent evaluation
The new independent panel contains 121 fictional tickets with 26 standalone replies. Different authors created the training and test material. No normalised question/reply pair crossed the new train, development and blind test splits. A complete mention must have every non-whitespace character masked. A clean ticket is flagged if it has any redaction. Both members of a contrast pair must have the intended privacy decision. These measures ignore type mismatches. The operating point was frozen before scoring the blind test.
| Method | Fully masked private mentions | Clean tickets flagged | Both matched replies correct |
|---|---|---|---|
| Earlier compact browser | 60/73 | 22/56 | 4/28 |
| Digitflow German Privacy Filter | 58/73 | 33/56 | 5/28 |
| New contextual model | 63/73 | 4/56 | 22/28 |
| New model with the preceding question omitted | 55/73 | 22/56 | 0/28 |
| Earlier private batch with Digitflow | 63/73 | 26/56 | 5/28 |
| Private batch with the new model | 65/73 | 23/56 | 7/28 |
Earlier 90-dialogue evaluation
| Method | Fully masked private mentions | Clean tickets flagged | Both matched replies correct |
|---|---|---|---|
| Earlier compact browser | 31/44 | 17/49 | 3/25 |
| Digitflow German Privacy Filter | 36/44 | 27/49 | 2/25 |
| New contextual model | 40/44 | 5/49 | 23/25 |
| New model with the preceding question omitted | 34/44 | 21/49 | 0/25 |
| Earlier private batch with Digitflow | 36/44 | 20/49 | 6/25 |
| Private batch with the new model | 39/44 | 20/49 | 7/25 |
The private batch comparison uses the same existing fast detector and router for each secondary model. Primary masks are preserved by construction. Passed the preregistered criteria for replacing the earlier private batch model. A better average is insufficient if it hides regressions on previously protected mentions. Consult full paired results before adopting it in another product. The published browser demo has its own operating point and runtime.
CPU inference on the new panel took 78 ms at the median and 107 ms at the 95th percentile, warm on two Contabo CPU threads. The two synthetic evaluations cover only a fraction of German support language. The model can expose private details and redact ordinary text. Performance on customer tickets is unknown.
External German finance transfer
On 453 independently generated German finance documents from Gretel, the new standalone model completely masked 568 of 1,025 in-scope mentions. The earlier browser masked 663 of 1,025. The external annotations can omit private values, so this test does not estimate false alarms. It did not select the training checkpoint. Gretel German finance corpus
- Downloads last month
- 19
Model tree for EasyOctopus/german-ticket-privacy-filter
Base model
distilbert/distilbert-base-german-cased