---
license: mit
language:
- zh
library_name: onnxruntime
pipeline_tag: token-classification
tags:
- onnx
- punctuation-restoration
- input-method
- traditional-chinese
---

# 0c D512 Punctuation Restoration (Bidirectional Encoder)

**Project website:** [0c.biggo.com](https://0c.biggo.com)

Restores punctuation in unpunctuated Traditional Chinese text. The model inserts punctuation while preserving
all source characters. It does not append punctuation at the end of the text.
Labels: ， 。 ？ ！ 、 ；, plus a "no insertion" class.

- Architecture: a bidirectional Transformer encoder with 8 layers, hidden size 512, 8 attention heads, FFN size 768. A D512 backbone pretrained from random initialization was adapted to bidirectional attention and fine-tuned for punctuation restoration.
- Window: 128 characters; stride: 64
- Format: ONNX (opset 18), CPU inference
- Vocabulary: 27,857 characters (`vocab.json`)

## Files

| File | Description |
|---|---|
| `model.onnx` | Punctuation classification model; SHA-256 `4db3ed40df777347d9c455da01285b01de192d4c210c5f7cd9f0646c727880d8` |
| `vocab.json` | Special tokens and character vocabulary; SHA-256 `556dffaffbf4c05211a846dcee55ef735cea68c5754f9bffcc6f9588124172b3` |
| `manifest.json` | Labels, threshold, window settings, and file hashes |

## Evaluation

Export validation used 1,171 development segments that had been seen during training,
rather than an unseen test set. At threshold 0.63, precision is 0.897,
recall is 0.755, and F1 is 0.820. ONNX and PyTorch produce identical punctuation insertions for every validation segment.

## Installation

Requires Python 3.9 or later. The examples use CPU inference.

```sh
pip install onnxruntime numpy huggingface_hub
```

The first run downloads the model from Hugging Face (approximately 130 MB). Subsequent runs use the local cache.

## Usage

Save the complete example below as `punctuate.py` and run it. Threshold 0.63 is an operating point
with precision around 0.9 in the reported validation. Lower thresholds insert more punctuation but also increase incorrect insertions.

```python
import json
import re

import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download

path = snapshot_download("Funmula/0c-punct-d512")
manifest = json.load(open(f"{path}/manifest.json", encoding="utf-8"))
vocab = json.load(open(f"{path}/vocab.json", encoding="utf-8"))
CHARS = {c: i + 8 for i, c in enumerate(vocab["characters"])}   # IDs 0–7 are special tokens; 4 = UNK
LABELS = manifest["labels"]                                       # Class 0 = no insertion
THRESHOLD, WINDOW, STRIDE = manifest["threshold"], manifest["window"], manifest["stride"]
session = ort.InferenceSession(f"{path}/model.onnx", providers=["CPUExecutionProvider"])

URL = re.compile(r"(?:https?://|www\.)[^\s，。！？、；「」『』（）【】]+", re.I)
ENUMERATORS = set("一二三四五六七八九十壹貳參肆伍陸柒捌玖拾甲乙丙丁戊己庚辛壬癸")


def is_han(c):
    return "㐀" <= c <= "鿿" or "\U00020000" <= c <= "\U000323af"


def allowed(text):
    # Insert after a Han character outside URLs, followed by Han text or whitespace;
    # the next non-whitespace character must be Han or ASCII alphanumeric.
    protected = {i for m in URL.finditer(text) for i in range(m.start(), m.end())}
    out = []
    for i, c in enumerate(text):
        nxt = text[i + 1:i + 2]
        visible = text[i + 1:].lstrip()[:1]
        out.append(is_han(c) and i not in protected
                   and (not nxt or is_han(nxt) or nxt.isspace())
                   and (not visible or is_han(visible) or visible.isascii() and visible.isalnum()))
    return out


def window_probs(text):
    ids = np.array([[1, 7] + [CHARS.get(c, 4) for c in text]], dtype=np.int64)   # [BOS][TASK] source text
    return session.run(["probs"], {"ids": ids})[0][2:]   # Row i: label probabilities after character i


def probabilities(text):
    # For long text, use sliding windows and keep the most central prediction at each position.
    n = len(text)
    starts = [0] if n <= WINDOW else list(range(0, n - WINDOW, STRIDE)) + [n - WINDOW]
    out, best = [None] * n, [-1] * n
    for s in starts:
        probs = window_probs(text[s:s + WINDOW])
        for j in range(len(probs)):
            margin = min(j, len(probs) - 1 - j) if n > WINDOW else 0
            if margin > best[s + j]:
                best[s + j], out[s + j] = margin, probs[j]
    return np.stack(out)


def punctuate(text):
    # Insert punctuation while preserving every source character; leave the final boundary unchanged.
    if not text:
        return text
    ok, probs = allowed(text), probabilities(text)
    out = []
    for i, c in enumerate(text):
        out.append(c)
        if i == len(text) - 1 or not ok[i]:
            continue
        k = int(probs[i, 1:].argmax()) + 1
        if i == 0 and LABELS[k] == "、" and c in ENUMERATORS:   # Skip a list comma after a single initial enumerator
            continue
        if probs[i, k] >= THRESHOLD:
            out.append(LABELS[k])
    return "".join(out)


print(punctuate("好的我知道了明天見"))   # 好的我知道了，明天見
```

The input-method integration processes at most 256 characters per request. The example itself
has no length limit and uses sliding windows for longer text.

## License and Release Scope

Model weights and the accompanying inference examples are released under the MIT license (see `LICENSE`).
This release covers this D512 checkpoint only. Training code and training materials are not included.