README.md

---
language:
- ko
- en
license: apache-2.0
library_name: onnxruntime
pipeline_tag: token-classification
inference: false
base_model: openai/privacy-filter
tags:
- onnx
- onnxruntime
- int8
- weight-only
- token-classification
- ner
- pii
- privacy
- pii-masking
- korean
- finance
- bioes
- viterbi
- mixture-of-experts
datasets:
- BCCard/pii-masking-openpii-finance
metrics:
- f1
- precision
- recall
---

# 1. Overview
This repository provides the INT8 weight-only ONNX Runtime serving artifact for a Korean/English PII detection model for the finance domain. The model was built by full fine-tuning [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) - a 1.4B-parameter MoE with 50M active parameters - on synthetic finance-domain PII data. It tags 18 PII entity types using 73
BIOES token classes at token level.

The model is designed as the NER layer of a multi-layer PII-masking gateway in front of LLM services. It should be deployed with a deterministic regex backstop for fully structured identifiers and must not be treated as a standalone anonymization, compliance or privacy guarantee. High-sensitivity deployments require in-domain evaluation and ongoing monitoring.

This repository does not contain PyTorch weights. `AutoModelForTokenClassification.from_pretrained()`
and the standard Transformers token-classification pipeline therefore do not load this artifact. Use ONNX Runtime as shown in [Usage](#13-usage).

The serving chain is:

```text
text
-> tokenizer
-> INT8 weight-only ONNX graph
-> FP32 logits
-> constrained BIOES Viterbi
-> character-offset spans
-> whitespace boundary refinement
```

## 1.1. TL;DR
* **Base model**: [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter) — 1.4B-parameter MoE (128 experts, 50M active), 8 layers, hidden 640, bidirectional banded attention (±128), o200k tokenizer
* **Domain / Language**: Finance (BC Card — cards, accounts, national IDs, customer service text) / Korean + English
* **Task**: Token classification (BIOES) → character-offset PII spans → masking
* **Labels (18)**: `PERSON, RRN, FRN, CARD_NUMBER, ACCOUNT_NUMBER, SECRET, USER_ID, EMAIL, PHONE, PASSPORT, DRIVER_LICENSE, GENERIC_ID, ADDRESS, ZIPCODE, DATE, CARD_EXPIRY, CVC, IPIN`
* **Method**: Full fine-tuning (all parameters incl. experts & router) with a re-initialized 73-class head (rows copied from the base head by taxonomy mapping)
* **Decoding**: **constrained BIOES Viterbi** (not per-token argmax) + whitespace span refinement — the bundled `viterbi_calibration.json` exposes precision↔recall operating-point biases without retraining
* **Format**: INT8 weight-only ONNX (`MatMulNBits`, `QMoE`, `GatherBlockQuantized`) with FP32 activations and logits; graph + external tensor data + tokenizer + label taxonomy + Viterbi calibration sidecar
* **Sequence length**: trained on sequences ≤768 tokens — chunk longer inputs
* **Intended use**
  - In-house **PII masking gateway** (detect → mask before text reaches an LLM)
  - Korean-centric finance text with mixed English (IDs, e-mails, card numbers)

## 1.2. Label Taxonomy (N=18)
The 18 labels re-map the upstream ai4privacy source labels to the granularity a Korean financial masking policy needs - merging fragments into single spans (`GIVENNAME`/`SURNAME` → `PERSON`, `CITY`/`STREET`/`BUILDINGNUM` → `ADDRESS`) and adding Korea-specific classes absent upstream (`RRN`, `FRN`, `IPIN`, `CARD_EXPIRY`, `CVC`, `SECRET`). `data source` records the row-source buckets in which each label occurs: `ko` means `openpii-1.5m-ko`, `en` means `openpii-1.5m-en`, and `domain` means locally synthesized rows.

| label | description | data source |
|-------|-------------|-------------|
| `PERSON` | full name (surname + given, single span) | ko, en, domain |
| `RRN` | resident registration number (Korea) | ko, domain |
| `FRN` | foreign registration number | domain |
| `CARD_NUMBER` | credit/debit card PAN | ko, en, domain |
| `ACCOUNT_NUMBER` | bank account number | ko, domain |
| `SECRET` | auth secret (password / API key / token) | ko, domain |
| `USER_ID` | online member ID | ko, en, domain |
| `EMAIL` | email address | ko, en, domain |
| `PHONE` | phone number (mobile / landline) | ko, en, domain |
| `PASSPORT` | passport number | ko, en, domain |
| `DRIVER_LICENSE` | driver's license number | ko, en, domain |
| `GENERIC_ID` | generic identifier (no KO counterpart) | ko, en, domain |
| `ADDRESS` | address (city / street / building, single span) | ko, en, domain |
| `ZIPCODE` | postal code | ko, en, domain |
| `DATE` | date / time | ko, en, domain |
| `CARD_EXPIRY` | card expiry date | domain |
| `CVC` | card verification code | domain |
| `IPIN` | I-PIN number | domain |

Each entity type has `B-`, `I-`, `E-` and `S-` boundary classes, plus the background class `O`. This yields 73 output classes. The bundled `label-taxonomy.yaml` and `config.json` must remain in the same label order.

## 1.3. Usage
Install the CPU runtime and client dependencies:

```bash
pip install "onnxruntime>=1.28,<1.29" "huggingface-hub>=1.5" "transformers>=5.6" numpy
```

The following example downloads the repository and runs the INT8 ONNX graph. To reproduce the validated output, use the functions in [Viterbi Reference Implementation](#43-viterbi-reference-implementation)
to apply constrained BIOES decoding and recover refined character-offset spans. Independent token argmax is not equivalent to the validated decoding chain.

```python
import json
from pathlib import Path

import numpy as np
import onnxruntime as ort

from huggingface_hub import snapshot_download
from transformers import AutoTokenizer

model_id = "BCCard/MoAI-Privacy-Filter-INT8"
model_dir = Path(snapshot_download(repo_id=model_id))
tokenizer = AutoTokenizer.from_pretrained(model_dir)
config = json.loads((model_dir / "config.json").read_text(encoding="utf-8"))
labels = tuple(
    config["id2label"][str(index)]
    for index in range(len(config["id2label"]))
)
calibration = json.loads(
    (model_dir / "viterbi_calibration.json").read_text(encoding="utf-8")
)
biases = calibration["operating_points"]["default"]["biases"]

session = ort.InferenceSession(
    str(model_dir / "model_quantized.onnx"),
    providers=["CPUExecutionProvider"],
)
text = "고객 모아이님(000000-0000000)께서 010-0000-0000로 연락 요청하셨습니다."
encoding = tokenizer(
    text,
    return_offsets_mapping=True,
    add_special_tokens=False,
    return_tensors="np",
)
offsets = encoding.pop("offset_mapping")[0].tolist()
feeds = {
    model_input.name: np.asarray(encoding[model_input.name], dtype=np.int64)
    for model_input in session.get_inputs()
}
logits = session.run(["logits"], feeds)[0]
length = int(feeds["attention_mask"][0].sum())

# Copy constrained_viterbi() and decode_spans() from Section 4.3.
path = constrained_viterbi(logits[0, :length], labels, biases)
tags = [labels[class_id] for class_id in path]
spans = decode_spans(tags, offsets[:length], text)

print(logits.shape)
print(spans)
```

Output from the INT8 graph:

```text
(1, 30, 73)
[
  {'start': 3, 'end': 6, 'label': 'PERSON'},
  {'start': 8, 'end': 22, 'label': 'RRN'},
  {'start': 26, 'end': 39, 'label': 'PHONE'}
]
```

The offsets use Python's half-open character interval `[start, end)`. Masking is downstream policy logic. For example, the spans above can produce:

```text
고객 [PERSON]님([RRN])께서 [PHONE]로 연락 요청하셨습니다.
```

For batches, enable right padding and pass only `input_ids` and `attention_mask` to the graph. `offset_mapping` stays outside ONNX and is used only to map decoded token tags back to the original text. Inputs are INT64 and the graph always returns FP32 logits.


## 1.4. Training Data
| Dataset | Role | Size |
|---------|------|------|
| (Public) [BCCard/pii-masking-openpii-finance](https://huggingface.co/datasets/BCCard/pii-masking-openpii-finance) (v2) | Training / validation | ~58.5k train rows · ~14.5k validation rows |
| (Private) BCCard/pii-masking-openpii-finance-test (v2) | Golden Set (release gate; never used for training/tuning) | 2,000 rows (ko 1,460 / en 540) |

* Sources: curated Korean subset of `ai4privacy/pii-masking-openpii-1.5m` (label taxonomy remapped, name spans merged & naturalized) + finance-domain synthetic templates + **~30% English replay** (forgetting guard)
* Hard-example design baked into v2: surface-similar non-PII decoys (FP suppression), label-confusion pairs in one sentence (RRN↔FRN, DRIVER_LICENSE↔GENERIC_ID), weak-context true PII (FN suppression), long-span address boundary variants
* All values are synthetic; validity-pattern collisions with real identifiers are removed at generation time (e.g. card numbers are forced to fail Luhn)


## 1.5. Training Procedure
| Item | Value |
|------|-------|
| Method | Full fine-tuning (1.4B params — experts and router included) |
| Head | 33-class base head → 73-class head, initialized by copying base rows via taxonomy mapping |
| Loss | Token-level cross-entropy |
| Batch | effective 16 (per-device × world × accum), fixed across hardware layouts |
| LR / scheduler | 1e-4 / linear decay, warmup 3% |
| Optimizer | AdamW (fused), weight decay 0.0, max_grad_norm 1.0 |
| Epochs | 5 — best checkpoint by validation span micro-F1, **decoded with the same constrained Viterbi as deployment** |
| Precision | FP32 master weights + BF16 autocast; MoE router/experts explicitly kept FP32 during compute |
| Hardware | 1× NVIDIA H100 (~5h) |

<div align="center">
  <img src="figures/evaluation-train-1-1.png" alt="Training loss, learning-rate and gradient-norm curves for the v1 and v2 models" >
</div>

<div align="center">
  <img src="figures/evaluation-test-1-1.png" alt="Training-time validation metric curves for the v1 and v2 models" >
</div>

<br>

# 2. Evaluation
## 2.1. INT8 Validation Results
The INT8 artifact was evaluated on all 14,543 v2 validation rows with the same tokenizer, actual character offsets, constrained Viterbi decoder and whitespace refinement used in the serving example.

| Metric | INT8 ONNX |
|---|---:|
| strict micro F1 | 0.9599 |
| macro F1 | 0.9607 |
| ko strict micro F1 | 0.9560 |
| ko macro F1 | 0.9571 |
| en strict micro F1 | 0.9689 |
| en macro F1 | 0.9641 |
| masking coverage - diagnostic | 0.9980 |
| ko masking coverage - diagnostic | 0.9987 |
| en masking coverage - diagnostic | 0.9967 |

Masking coverage is the proportion of gold PII characters covered by the union of predicted spans, regardless of predicted label. It is a diagnostic value and was not used as the INT8 pass or fail gate.

## 2.2. INT8 Parity Against FP32 ONNX
| Metric | FP32 ONNX | INT8 ONNX | Delta |
|---|---:|---:|---:|
| strict micro F1 | 0.959801 | 0.959886 | +0.000085 |
| macro F1 | 0.960264 | 0.960662 | +0.000398 |
| masking coverage - diagnostic | 0.997960 | 0.997980 | +0.000020 |
| ko strict micro F1 | 0.955986 | 0.956047 | +0.000061 |
| en strict micro F1 | 0.968768 | 0.968910 | +0.000142 |

INT8 predictions reached strict micro F1 0.998568 against FP32 predictions and matched exactly on 14,429 of 14,543 rows. Overall PERSON recall changed by -0.000276 and ADDRESS recall changed by +0.000116. All thresholds fixed before the full run passed. Small positive quality deltas do not show that INT8 is intrinsically more accurate. Quantization moved a small number of boundary decisions in both directions.

## 2.3. Limitations and Deployment Guidance
* The graph uses Microsoft contrib operators. ONNX Runtime 1.28.0 with `CPUExecutionProvider` was validated. Confirm operator placement and fallback behavior before selecting another provider.
* The M4 validation run was a functional observation, not a serving benchmark. Measure warmup, latency percentiles, throughput, RSS and peak memory on target hardware.
* Independent token argmax can create invalid BIOES paths and does not reproduce the reported metrics. Use constrained Viterbi decoding.
* Weak-context Korean person names remain the main known miss channel. Names without honorifics, particles or nearby role cues require particular monitoring.
* Alphanumeric identifiers can swap among `USER_ID`, `SECRET`, `GENERIC_ID` and `ACCOUNT_NUMBER`. The value may still be masked even when the semantic label is wrong.
* Training and evaluation use synthetic data. Robustness to real customer text, slang, OCR noise and previously unseen credential formats has not been established.
* The label policy is fixed to the 18 types above. Changing taxonomy or boundary policy requires fine-tuning.
* Inputs beyond 768 tokens were not part of this release validation. Chunk longer inputs with overlap and reconcile spans at chunk boundaries.
* Use deterministic pattern rules as a backstop for structured identifiers and retain review paths for high-sensitivity workflows.

<br>

# 3. Future Work
* Benchmark the INT8 artifact on the actual serving hardware and execution provider.
* Improve weak-context PERSON coverage and collect privacy-safe shadow-mode failure patterns for a future data revision.

<br>

# 4. Meta Info
## 4.1. Citation
```bibtex
@misc{bccard2026moaiprivacyfilter,
  title        = {MoAI Privacy Filter INT8: A Korean Finance-Domain PII Detection Model},
  author       = {BC Card AX Team},
  year         = {2026},
  howpublished = {https://huggingface.co/BCCard/MoAI-Privacy-Filter-INT8},
  note         = {INT8 weight-only ONNX artifact of a full fine-tune of openai/privacy-filter}
}
```

## 4.2. See Also
* Base model - [`openai/privacy-filter`](https://huggingface.co/openai/privacy-filter)
* Parent BF16 model - [`BCCard/MoAI-Privacy-Filter`](https://huggingface.co/BCCard/MoAI-Privacy-Filter)
* Training dataset - [`BCCard/pii-masking-openpii-finance`](https://huggingface.co/datasets/BCCard/pii-masking-openpii-finance)
* Source data attribution - `ai4privacy/pii-masking-openpii-1.5m` under CC-BY-4.0

## 4.3. Viterbi Reference Implementation
These functions apply the BIOES transition constraints and calibration biases used for validation, then recover character-offset spans with whitespace boundary refinement. Keep `config.json`, `label-taxonomy.yaml` and `viterbi_calibration.json` in their bundled label order and configuration.

<details>
<summary>Show the constrained Viterbi and span-decoding reference code</summary>

```python
import numpy as np


def transition_bias(
    from_prefix: str,
    to_prefix: str,
    biases: dict[str, float],
) -> float:
    if from_prefix == "O" and to_prefix == "O":
        return biases["transition_bias_background_stay"]
    if from_prefix == "O" and to_prefix in {"B", "S"}:
        return biases["transition_bias_background_to_start"]
    if from_prefix in {"E", "S"} and to_prefix == "O":
        return biases["transition_bias_end_to_background"]
    if from_prefix in {"E", "S"} and to_prefix in {"B", "S"}:
        return biases["transition_bias_end_to_start"]
    if from_prefix in {"B", "I"} and to_prefix == "I":
        return biases["transition_bias_inside_to_continue"]
    if from_prefix in {"B", "I"} and to_prefix == "E":
        return biases["transition_bias_inside_to_end"]
    raise ValueError(f"Unsupported transition family: {from_prefix} -> {to_prefix}")


def constrained_viterbi(
    logits: np.ndarray,
    labels: tuple[str, ...],
    biases: dict[str, float],
) -> list[int]:
    scores = np.asarray(logits, dtype=np.float32)
    num_classes = len(labels)
    parsed = [
        ("O", None) if tag == "O" else tuple(tag.split("-", 1))
        for tag in labels
    ]
    start_scores = np.full(num_classes, -np.inf, dtype=np.float32)
    end_scores = np.full(num_classes, -np.inf, dtype=np.float32)
    transition_scores = np.full(
        (num_classes, num_classes),
        -np.inf,
        dtype=np.float32,
    )

    for class_id, (tag_prefix, _) in enumerate(parsed):
        if tag_prefix in {"O", "B", "S"}:
            start_scores[class_id] = 0.0
        if tag_prefix in {"O", "E", "S"}:
            end_scores[class_id] = 0.0

    for from_id, (from_prefix, from_label) in enumerate(parsed):
        for to_id, (to_prefix, to_label) in enumerate(parsed):
            allowed = (
                from_prefix in {"O", "E", "S"}
                and to_prefix in {"O", "B", "S"}
            ) or (
                from_prefix in {"B", "I"}
                and to_prefix in {"I", "E"}
                and from_label == to_label
            )
            if allowed:
                transition_scores[from_id, to_id] = transition_bias(
                    from_prefix,
                    to_prefix,
                    biases,
                )

    delta = start_scores + scores[0]
    backpointers = np.zeros((scores.shape[0], num_classes), dtype=np.int64)
    for token_id in range(1, scores.shape[0]):
        candidates = delta[:, None] + transition_scores
        backpointers[token_id] = np.argmax(candidates, axis=0)
        delta = np.max(candidates, axis=0) + scores[token_id]

    last_class = int(np.argmax(delta + end_scores))
    path = [last_class]
    for token_id in range(scores.shape[0] - 1, 0, -1):
        path.append(int(backpointers[token_id, path[-1]]))
    return list(reversed(path))


def decode_spans(
    tags: list[str],
    offsets: list[list[int]],
    text: str,
) -> list[dict[str, int | str]]:
    spans = []
    current = None

    def flush() -> None:
        nonlocal current
        if current is None:
            return
        start = current["start"]
        end = current["end"]
        while start < end and text[start].isspace():
            start += 1
        while end > start and text[end - 1].isspace():
            end -= 1
        if start < end:
            spans.append(
                {
                    "start": start,
                    "end": end,
                    "label": current["label"],
                }
            )
        current = None

    for tag, (start, end) in zip(tags, offsets, strict=True):
        if tag == "O":
            flush()
            continue
        tag_prefix, label = tag.split("-", 1)
        if tag_prefix in {"B", "S"} or current is None or current["label"] != label:
            flush()
            current = {
                "start": start,
                "end": end,
                "label": label,
            }
        else:
            current["end"] = end
        if tag_prefix in {"E", "S"}:
            flush()
    flush()
    return spans
```

</details>