README.md

---
license: mit
language:
- zh
library_name: onnxruntime
tags:
- onnx
- bopomofo
- zhuyin
- input-method
- traditional-chinese
---

# 0c D512 Zhuyin Candidate Reranker (PF512)

**Project website:** [0c.biggo.com](https://0c.biggo.com)

A neural candidate reranker for a Traditional Chinese Zhuyin (Bopomofo) input method.
Given confirmed preceding text (up to 64 characters), Zhuyin readings, and supplied candidates,
the model computes log P(candidate | context, reading). These scores can be combined with an input-method engine's
scores to decide whether to override its default candidate. A candidate source is required; the model does not generate candidates itself.

- Architecture: a dedicated causal Transformer decoder, 8 layers, hidden size 512, 8 attention heads, FFN size 768, pretrained from random initialization and then fine-tuned for candidate ranking
- Format: ONNX (opset 18), CPU inference
- Vocabulary: 27,857 characters and 416 Zhuyin bases (`vocab.json`)

## Files

| File | Description |
|---|---|
| `model.onnx` | Candidate scoring model; SHA-256 `71af1f0e77b2752cc71cbc04bf19c8e8f2e765662246f766e65a2822bab4e66e` |
| `vocab.json` | Special tokens, character vocabulary, and Zhuyin bases; SHA-256 `556dffaffbf4c05211a846dcee55ef735cea68c5754f9bffcc6f9588124172b3` |
| `manifest.json` | Operating points, input limits, and file hashes |

## Installation

Requires Python 3.9 or later. The examples use CPU inference.

```sh
pip install onnxruntime numpy huggingface_hub
```

The first run downloads the model from Hugging Face (approximately 130 MB). Subsequent runs use the local cache.

## Usage

Save the complete example below as `rank.py` and run it. Supply one string per Zhuyin syllable
(for example, `ㄍㄨㄥ` and `ㄩㄢˊ`). Each candidate must have exactly one character per syllable.

- With supplied candidates only: use `rank()` to order homophonic candidates by model score and find the best match for the context.
- With an input-method engine: use `choose()` to combine engine and model scores, overriding the engine default only when the required score margin is met.

```python
import json

import numpy as np
import onnxruntime as ort
from huggingface_hub import snapshot_download

path = snapshot_download("Funmula/0c-sequence-d512")
manifest = json.load(open(f"{path}/manifest.json", encoding="utf-8"))
vocab = json.load(open(f"{path}/vocab.json", encoding="utf-8"))
CHARS = {c: i + 8 for i, c in enumerate(vocab["characters"])}   # IDs 0–7 are special tokens; 4 = UNK
BASES = {b: i + 2 for i, b in enumerate(vocab["bases"])}
TONES = {"ˊ": 2, "ˇ": 3, "ˋ": 4, "˙": 5}                         # No tone mark = first tone
NEUTRAL = set(manifest.get("neutral_tolerant_last", []))        # Final particles are often typed with first tone
session = ort.InferenceSession(f"{path}/model.onnx", providers=["CPUExecutionProvider"])


def syllable(s):
    base = s[:-1] if s[-1] in TONES else s
    return BASES[base], TONES.get(s[-1], 1)


def log_prob(context, reading, candidate):
    # [BOS] context [SEP] reading×n [OUT] first n−1 candidate characters.
    # Return log P(candidate | context, reading).
    head = [1] + [CHARS.get(c, 4) for c in context[-64:]] + [2]
    sounds = [syllable(s) for s in reading]
    cand = [CHARS[c] for c in candidate]   # Unknown candidate characters raise KeyError for caller fallback
    n, out = len(cand), len(head) + len(sounds)
    ids = head + [0] * n + [3] + cand[:-1]
    bases = [0] * len(head) + [b for b, _ in sounds] + [0] * n
    tones = [0] * len(head) + [t for _, t in sounds] + [0] * n
    types = [0] * len(head) + [1] * n + [0] + [2] * (n - 1)
    L = len(ids)
    row = lambda x: np.array([x], dtype=np.int64)
    feeds = {"ids": row(ids), "bases": row(bases), "tones": row(tones), "types": row(types),
             "positions": row(list(range(L))), "mask": np.tril(np.ones((L, L), dtype=bool))[None, None],
             "proj_pos": np.arange(out, L, dtype=np.int64), "target_node": row(list(range(n))),
             "target_ids": row([x - 6 for x in cand])}
    return float(session.run(["scores"], feeds)[0][0])


def score(context, reading, candidate):
    s = log_prob(context, reading, candidate)
    last = reading[-1]
    base = last[:-1] if last[-1] in TONES else last
    if base in NEUTRAL and last[-1] != "˙":   # Also score the final syllable with neutral tone and keep the higher score
        s = max(s, log_prob(context, reading[:-1] + [base + "˙"], candidate))
    return s


def choose(context, reading, candidates, engine_scores, w=manifest["weight"], tau=manifest["margin"]):
    # candidates[0] is the engine default; engine_scores are engine log scores.
    # Return the selected candidate index.
    try:
        neural = [score(context, reading, c) for c in candidates]
    except KeyError:
        return 0
    fused = [e + w * s for e, s in zip(engine_scores, neural)]
    best = max(range(len(fused)), key=fused.__getitem__)
    return best if best != 0 and fused[best] - fused[0] > tau else 0


def rank(context, reading, candidates):
    # Rank supplied homophonic candidates by model score alone, highest first.
    return sorted(candidates, key=lambda c: score(context, reading, c), reverse=True)


print(rank("下週就是", ["ㄑㄧˊ", "ㄓㄨㄥ", "ㄎㄠˇ"], ["其中考", "期中考"]))   # ['期中考', '其中考']
print(rank("", ["ㄐㄧㄣ", "ㄊㄧㄢ", "ㄊㄧㄢ", "ㄑㄧˋ", "ㄏㄣˇ", "ㄏㄠˇ"],
           ["金天天氣很好", "今天天氣很好", "今天天器很好"]))                   # ['今天天氣很好', ...]

# With an input-method engine, override candidates[0] only when the fused score warrants it.
candidates = ["公元", "公園"]
print(candidates[choose("今天天氣很好我們去", ["ㄍㄨㄥ", "ㄩㄢˊ"], candidates, [-7.0, -7.5])])   # 公園
```

## Operating Points and Limitations

- Fused score = engine score + w × model score, with w = 2. Override only if another candidate has the highest score and exceeds the engine default by more than τ = 0.
- Supported operating range: 2–32 syllables, at most 16 candidates, and the last 64 context characters.
- If any candidate contains a character outside the vocabulary, `choose()` retains the engine default for the entire case.
- Training candidates came from the 0c dictionary (`dictionary_sha256 = 517e6c8bb5cbb145c0554edc3cc91c2b55fc753dd42e11194d83f6b0e119aea3`). Calibrate w and τ again when using a different engine's score scale.
- Shared candidate prefixes can also be scored together as a tree, using an ancestor attention mask and multiple `target_node`/`target_ids` rows. Scores match individual candidate scoring.

## License and Release Scope

Model weights and the accompanying inference examples are released under the MIT license (see `LICENSE`).
This release covers this D512 checkpoint only. Training code and training materials are not included.