Add the iPhone 16 Pro Max and keyboard-extension measurements to the model card

8cc5ef3 verified 1 day ago

15.7 kB

---
language:
- zh
license: gpl-3.0
pipeline_tag: text-generation
tags:
- pytorch
- coreml
- chinese
- pinyin
- input-method
- causal-lm
- contextual-reranking
- tiny-gpt
- rmsnorm
- swiglu
---
# VimeML Tiny Chinese GPT v1

A **20,036,736-parameter** Simplified Chinese decoder-only language model for local contextual reranking of pinyin input-method candidates and short phrase prediction. This release contains the **vimeml-tiny-zh-v1 step 844,730** FP32 inference bundle, its **INT8 block32, FP32-compute Core ML** stateless package, and a matching **chunked KV-cache** Core ML package.

**Status: release for iOS deployment.** The model is being integrated into the Vime input method on iOS. The Core ML packages pass the Mac alignment gates. They have been measured on iPhone 16 Pro Max in a test host and inside the Vime keyboard extension, where the model reranks pinyin candidates (see Performance).

This is a custom PyTorch model, not a Transformers `AutoModel` integration or an instruction/chat model. SentencePiece, candidate retrieval (librime with rime-ice), candidate scoring and search run outside the Core ML graph. The architecture, loader and scoring code are included under `source/`; no `trust_remote_code` or training checkpoint is required.

## Files and identity

| Path | Contents |
| --- | --- |
| `inference/` | Unique FP32 state_dict, config, tokenizer and provenance manifest; no optimizer |
| `coreml/ios18-int8-block32/` | Stateless Core ML `.mlpackage` and manifest |
| `coreml/ios18-int8-block32-kvchunk/` | Chunked KV-cache Core ML `.mlpackage` and manifest |
| `source/` | Frozen VimeML Python source, scripts, configurations and requirements |
| `infer.py` | FP32 candidate-scoring and phrase-prediction wrapper using the zh-v1 bundle loader |
| `evaluation/summary.json` | Aggregate quality, alignment and measurement results; no benchmark texts or traces |
| `RELEASE.json`, `SHA256SUMS.txt`, `verify_release.py` | Release inventory and optional byte verification |

- Source: [Voltline/VimeML](https://github.com/Voltline/VimeML), commit `54ace39b7fbbce4c362d55677c638ea2db7b83c6`.
- FP32 inference manifest: `9d9dad19c573c84c84e6a78060d1999a0a7e978c90a9bd78ca844da565a6ae74`.
- Core ML manifests: stateless `50a5524592117ff2c4775792213009e6aeb9f388524746fb8d8eda99fa248276`, chunked KV `c5bc759d4a224cf389d9ae28304df194e7e1b0ba3eb3945a94ecea9ddd82abc7`.
- Tokenizer: `68bd37c45bcb018b32b2eef5b241561ab5b6dc1b77c1c3aadcc1b41c4bf47e5a`.
- Original training checkpoint: `44c423a1e7cb0e32161afc94d6167cabb439a3d25fd9de6d1fbb150a927542d4`; the optimizer-bearing checkpoint is not distributed. The inference weights equal the checkpoint's model tensors exactly (the tied LM head is stored once).

These identities are SHA256 digests. Original model manifests remain unchanged. The internal codename of this model family in code and manifests is `zh13` (`tiny_gpt_zh13`).

## Architecture and tokenization

Pre-Norm **RMSNorm/SwiGLU**, bias-free Linear layers, learned absolute positions and causal attention. Vocabulary 12,288; context 128; width 384; 9 layers; 6 heads of dimension 64; SwiGLU width 960; dropout 0; RMSNorm epsilon 1e-5. Input embedding and LM head share one parameter.

SentencePiece Unigram, 12,288 pieces, trained on a sample of the cleaned corpus: identity normalization, whitespace preservation, byte fallback and `add_dummy_prefix=false`. PAD/UNK/BOS/EOS IDs are 0/1/2/3. IDs 4–6 are reserved markers `<CTX>`, `<READ>`, `<OUT>` that the model never trains on; ID 7 is the newline. One 12K token covers 1.40 Chinese characters on average.

## Quick start: downloaded files

Python≥3.11. Install the included requirements after selecting a suitable PyTorch build. FP32 inference does not invoke Apple runtime services.

```sh
python -m pip install -r source/requirements.txt
python verify_release.py
python infer.py --context "我们明天一起去" --candidate "吃饭" --candidate "迟返"
python infer.py --context "我们明天一起去"
python infer.py --previous "周末有什么安排？" --context "我想去"
```

The candidate command ranks supplied strings; it does not convert pinyin or retrieve dictionary candidates. Without candidates, `infer.py` runs the phrase-prediction decoder described below. The loader reads weights with `torch.load(..., weights_only=True)`, restores the shared head, and checks the model configuration, parameter count and tokenizer contract. `verify_release.py` checks the complete inventory and sizes without running a model; `--hashes` additionally checks all file bytes against the recorded digests. Checksums are not a digital signature. Keep `source/`, the tokenizer and inference files together.

## Candidate scoring

Encode `BOS + context + candidate` jointly. Find the common token prefix of the context and all candidate sequences, then sum full-vocabulary log-softmax over the remaining suffix. Do not append candidate-final EOS; ties keep original order. This suffix likelihood is a scoring proxy, not exact string conditional probability. In deployment, librime with rime-ice produces the pinyin candidates and the model reorders them by pure LM score.

Phrase prediction (experimental) uses one beam decoder (`source/src/vimeml/benchmarks/predict_zh.py`): earlier turns are joined as separate `BOS turn EOS` sequences, the last pure-Chinese token of the typed text is taken back and re-spelled (token healing), beam width 16 over pure-Chinese tokens, and the top five 2–6 character phrases that are not prefixes of one another are returned.

## Training

Random initialization (seed 1591996171), one pass over fresh data, 844,730 updates, batch 512, BF16 with FP32 weights, AdamW (β 0.9/0.95, weight decay 0.1, clip 1.0), cosine learning rate 6e-4→6e-5 after 1,000 warmup updates, and a 30% prefix crop on eligible windows. Total exposure is 14,519,248,976 supervised next-token targets; training took 3.9 hours on a single GPU.

- **Web text:** 21 of the 370 train files of [FineWeb2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) `cmn_Hani`. Traditional-script documents were converted to Simplified Chinese with OpenCC; rule filters removed spam, gambling, pornographic and illegal-service pages and spam sites; flagged documents were kept only after review. A fixed hash-selected pool of 16,551,843 documents (21.28 billion characters) was seen once, split into paragraphs.
- **Dialogue:** independent utterances from [LCCC-base](https://github.com/thu-coai/CDial-GPT) (16,326,323 training utterances, 209 million characters), sampled to 5% of supervised targets, which repeats the pool about four times.

Every paragraph and utterance is its own `BOS text EOS` sequence; the model never saw multi-turn context during training. The objective is ordinary next-token loss. This model is the release arm (`12k-fresh-s1`) of a preregistered eight-arm experiment comparing fresh data with repeated data and an 8K with a 12K tokenizer. None of the five contrasts was effective, so the preregistered defaults (fresh data, 12K) apply; the final checkpoint was used as registered.

FP32 validation BPC is **4.354450** on held-out web text and **5.690730** on held-out LCCC dialogue (total NLL including EOS divided by characters × ln 2).

## Evaluation

`ime-standard-zh-v1-r1` reranks frozen librime + rime-ice candidate pools for real dialogue turns from CrossWOZ, RiSAWOZ and KdConv, using the left context; correct answers are not added to the pools. Labels are **AI-reviewed, not native-speaker gold**. The blind split was opened once for a preregistered confirmation, which passed; the baselines were scored afterwards with blind as a reference test.

| System | Parameters | Development Top-1 | Blind Top-1 | Blind Top-3 | Blind MRR |
| --- | ---: | ---: | ---: | ---: | ---: |
| **vimeml-tiny-zh-v1** | 20M | **93.6%** (1,363/1,456) | **93.8%** (1,355/1,445) | **97.2%** | **0.954** |
| Qwen2.5-0.5B (same pools and scoring) | 494M | 91.3% | 91.8% | 96.1% | 0.940 |
| Qwen3-0.6B-Base (same pools and scoring) | 596M | 89.8% | 89.2% | 95.4% | 0.924 |
| rime-ice + wanxiang grammar model | — | 79.0% | 82.2% | 83.7% | 0.829 |
| rime-ice | — | 68.8% | 72.2% | 73.6% | 0.729 |
| luna_pinyin_simp | — | 68.3% | 68.9% | 70.5% | 0.696 |

Against the rime-ice first candidate on blind: +329/−19, exact McNemar p = 3.6e-74. All five paired comparisons on both splits remain significant after Holm correction; the closest system is Qwen2.5-0.5B (+28 on blind, p = 0.005). Abbreviated pinyin (initials only) is much harder: blind Top-1 35.0%, against 31.0% for Qwen2.5-0.5B and 15.7% for rime-ice + grammar; pool coverage limits it to about 41%.

Most of the advantage over Rime comes from using left context: with empty context the model scores 81.9% on development, close to rime-ice + grammar. Whether Qwen saw these public dialogue datasets during training is unknown.

## Core ML variants

**Weight-only symmetric INT8, block size 32; FP32 computation; CPU_ONLY; minimum iOS 18.** No activation quantization or INT8 arithmetic is claimed. 47 matrices are compressed (token and position embeddings, fused QKV, attention projection and the three FFN matrices of every layer); RMSNorm weights, the causal mask and a converter-inserted zero head bias remain FP32. The stateless package is **22,792,958 bytes**, versus 80,367,198 bytes uncompressed (about 71.6% smaller).
- **Stateless:** `input_ids` INT32 `[1,T]`, 1≤T≤128, right PAD → `logits` FLOAT32 `[1,T,12288]`.
- **Chunked KV (22,830,199 bytes):** `input_ids` INT32 `[1,T]`, `position` INT32 `[1]` (current cache length, position+T≤128), `key_cache`/`value_cache` FLOAT32 `(9,1,6,128,64)` → logits for the new tokens plus updated caches. The caller owns cache snapshots and rollback.

```sh
xcrun coremlcompiler compile coreml/ios18-int8-block32/model.mlpackage compiled-int8 --platform ios --deployment-target 18.0
```

Uncompressed conversions pass frozen logits alignment (maximum absolute error 3.9e-5 stateless, 4.6e-5 chunked over 22 checks). **INT8 fails strict FP32 logits alignment** (maximum 0.392, mean 0.046 at T=128; V4.2 Japanese reached 0.477) at unchanged atol/rtol 3e-4; padding and future-token invariants are exact. The chunked INT8 graph matches the stateless INT8 package within 4.2e-5 with identical rankings. On development, INT8 versus FP32 Top-1 is 1,364 vs 1,363 (+2/−1) on full pinyin and 335 vs 337 (+0/−2) on abbreviated pinyin; Top-3 is unchanged on both. The strict failure is recorded as lossy quantization drift, not numerical equivalence.

## Performance evidence and limits

Apple M3 Mac, Core ML CPU_ONLY, measured interleaved with Japanese V4.2 in the same harness (median over five fresh processes):
| Metric | zh-v1 INT8 | V4.2 INT8 |
| --- | ---: | ---: |
| Stateless `predict` p50, T=1 / T=128 | 1.65 / 8.49 ms | 1.29 / 6.43 ms |
| Chunked KV prefill of 119 tokens / single-token decode, p50 | 9.33 / 2.71 ms | 7.18 / 2.05 ms |
| Model peak memory delta in the host process | 18.4 MiB | 13.4 MiB |
zh-v1 is about 1.3× V4.2 in size and latency, in line with its larger vocabulary and two extra layers. INT8 reduces size, not CPU latency, because computation stays FP32. A simulator functional check (iOS 27.2) passed: resource identity, logits shape, native SentencePiece tokenization and joint suffix scores within 0.002 of Mac Core ML INT8.
**iPhone 16 Pro Max, iOS 27.3, CPU_ONLY.** Vime test host, four rounds alternating the two models, medians:
| Metric | zh-v1 INT8 | V4.2 INT8 |
| --- | ---: | ---: |
| Stateless `predict` p50, T=1 / T=128 | 1.70 / 6.82 ms | 1.25 / 4.90 ms |
| Load / reload | 195 / 7.4 ms | 155 / 5.9 ms |
| Memory delta after load and 50 predictions | 14.1 MiB | 11.3 MiB |
The chunked KV package loads in 236–280 ms the first time. A single-token step takes 2.4–2.9 ms p50 and a 128-token prefill 7.5–7.6 ms p50. For 3–6 short candidates the stateless joint scoring is faster (p50 about 9 ms versus 13.5–14 ms with the KV package), so the keyboard uses the stateless package.
**Inside the keyboard extension** (Vime integration branch, not yet merged). Rime proposes the candidates. The model rescores at most the first 8 complete conversions after the sentence before the cursor; each place lower in Rime's order costs 1.0 log-probability. The candidate bar waits at most 60 ms for scores, then shows Rime's order. Learned phrases, partial conversions and excluded terms are never moved. Two five-minute runs of XCUITest taps on the installed keyboard (about 1.2 taps/s):
| Measured in the extension process | Model on | Model off |
| --- | ---: | ---: |
| Key → candidates tappable, mean / p95 / max | 26.1 / 45.0 / 49.7 ms | 20.3 / 26.1 / 27.4 ms |
| Model scoring, mean / p95 / max | 13.2 / 36.3 / 40.8 ms | — |
| Physical footprint median / max; process peak | 20.0 / 29.5; 29.9 MiB | 24.3 / 25.3; 25.7 MiB |
| Model load (once) | 253 ms | — |
p95 is the largest per-window p95 (windows of 256 calls). 221 of 222 scoring opportunities used the model's scores (the other showed Rime's order); 173 of the 221 lists had some order within the top 8 changed, which does not mean the first candidate changed. Scoring is several times slower in the extension than in the test host (per-keystroke scoring p50 1.9–2.0 ms, p95 9.2–9.8 ms over 300 scorings); the cause has not been identified. **Chinese next-word prediction in the keyboard, behavior under memory pressure and energy use have not been measured.**
## Intended use and limitations
Simplified Chinese pinyin candidate reranking with left context, and experimental short phrase prediction, primarily for local input methods. This small non-instruction-tuned model can repeat, drift or invent text. It is not a standalone pinyin converter, general assistant or factual reference.
- Benchmark labels are AI-reviewed, and the data are dialogue turns rather than real keyboard traffic; candidate-pool coverage bounds achievable accuracy.
- The model was trained on single paragraphs and single utterances, so multi-turn context is outside its training distribution.
- Web and social-media text carry their own biases. Cleaning and review reduce but do not remove inappropriate content; applications should keep their own output filtering.
- Public training overlap with evaluation data was checked by 16-character n-gram matching for the benchmark; semantic near-duplicates were not audited.
## License and attribution
The weights, tokenizer, Core ML packages and included VimeML code are released under **GNU GPL v3.0 or later** with an additional permission under GPLv3 section 7 for conveyance through application distribution services such as the App Store. See `LICENSE` (GPLv3), `LICENSE-EXCEPTION` (copyright, additional permission and the source form of model weights) and `NOTICE.md` (training data attribution that applications bundling this model must show). Training sources and third-party dependencies retain their own terms; they are not relicensed.
Training data: FineWeb2 `cmn_Hani` (ODC-By 1.0, subject to the Common Crawl Terms of Use) and LCCC-base by thu-coai (Wang et al., *A Large-Scale Chinese Short-Text Conversation Dataset*, NLPCC 2020), whose repository and dataset card declare the MIT license and whose README states that the dataset and pretrained dialogue models are provided for research purposes. Evaluation sources: CrossWOZ (Apache-2.0), RiSAWOZ (MIT), KdConv (Apache-2.0) and CPED (Apache-2.0). This repository distributes the model and aggregate evaluation metrics only; it contains no corpus, dialogue or benchmark text. Optimizer/RNG state, user inputs, raw device traces, credentials and the separate Vime client are excluded.

	---
	language:
	- zh
	license: gpl-3.0
	pipeline_tag: text-generation
	tags:
	- pytorch
	- coreml
	- chinese
	- pinyin
	- input-method
	- causal-lm
	- contextual-reranking
	- tiny-gpt
	- rmsnorm
	- swiglu
	---

	# VimeML Tiny Chinese GPT v1

	A 20,036,736-parameter Simplified Chinese decoder-only language model for local contextual reranking of pinyin input-method candidates and short phrase prediction. This release contains the vimeml-tiny-zh-v1 step 844,730 FP32 inference bundle, its INT8 block32, FP32-compute Core ML stateless package, and a matching chunked KV-cache Core ML package.

	Status: release for iOS deployment. The model is being integrated into the Vime input method on iOS. The Core ML packages pass the Mac alignment gates. They have been measured on iPhone 16 Pro Max in a test host and inside the Vime keyboard extension, where the model reranks pinyin candidates (see Performance).

	This is a custom PyTorch model, not a Transformers `AutoModel` integration or an instruction/chat model. SentencePiece, candidate retrieval (librime with rime-ice), candidate scoring and search run outside the Core ML graph. The architecture, loader and scoring code are included under `source/`; no `trust_remote_code` or training checkpoint is required.

	## Files and identity

	\| Path \| Contents \|
	\| --- \| --- \|
	\| `inference/` \| Unique FP32 state_dict, config, tokenizer and provenance manifest; no optimizer \|
	\| `coreml/ios18-int8-block32/` \| Stateless Core ML `.mlpackage` and manifest \|
	\| `coreml/ios18-int8-block32-kvchunk/` \| Chunked KV-cache Core ML `.mlpackage` and manifest \|
	\| `source/` \| Frozen VimeML Python source, scripts, configurations and requirements \|
	\| `infer.py` \| FP32 candidate-scoring and phrase-prediction wrapper using the zh-v1 bundle loader \|
	\| `evaluation/summary.json` \| Aggregate quality, alignment and measurement results; no benchmark texts or traces \|
	\| `RELEASE.json`, `SHA256SUMS.txt`, `verify_release.py` \| Release inventory and optional byte verification \|

	- Source: [Voltline/VimeML](https://github.com/Voltline/VimeML), commit `54ace39b7fbbce4c362d55677c638ea2db7b83c6`.
	- FP32 inference manifest: `9d9dad19c573c84c84e6a78060d1999a0a7e978c90a9bd78ca844da565a6ae74`.
	- Core ML manifests: stateless `50a5524592117ff2c4775792213009e6aeb9f388524746fb8d8eda99fa248276`, chunked KV `c5bc759d4a224cf389d9ae28304df194e7e1b0ba3eb3945a94ecea9ddd82abc7`.
	- Tokenizer: `68bd37c45bcb018b32b2eef5b241561ab5b6dc1b77c1c3aadcc1b41c4bf47e5a`.
	- Original training checkpoint: `44c423a1e7cb0e32161afc94d6167cabb439a3d25fd9de6d1fbb150a927542d4`; the optimizer-bearing checkpoint is not distributed. The inference weights equal the checkpoint's model tensors exactly (the tied LM head is stored once).

	These identities are SHA256 digests. Original model manifests remain unchanged. The internal codename of this model family in code and manifests is `zh13` (`tiny_gpt_zh13`).

	## Architecture and tokenization

	Pre-Norm RMSNorm/SwiGLU, bias-free Linear layers, learned absolute positions and causal attention. Vocabulary 12,288; context 128; width 384; 9 layers; 6 heads of dimension 64; SwiGLU width 960; dropout 0; RMSNorm epsilon 1e-5. Input embedding and LM head share one parameter.

	SentencePiece Unigram, 12,288 pieces, trained on a sample of the cleaned corpus: identity normalization, whitespace preservation, byte fallback and `add_dummy_prefix=false`. PAD/UNK/BOS/EOS IDs are 0/1/2/3. IDs 4–6 are reserved markers `<CTX>`, `<READ>`, `<OUT>` that the model never trains on; ID 7 is the newline. One 12K token covers 1.40 Chinese characters on average.

	## Quick start: downloaded files

	Python≥3.11. Install the included requirements after selecting a suitable PyTorch build. FP32 inference does not invoke Apple runtime services.

	```sh
	python -m pip install -r source/requirements.txt
	python verify_release.py
	python infer.py --context "我们明天一起去" --candidate "吃饭" --candidate "迟返"
	python infer.py --context "我们明天一起去"
	python infer.py --previous "周末有什么安排？" --context "我想去"
	```

	The candidate command ranks supplied strings; it does not convert pinyin or retrieve dictionary candidates. Without candidates, `infer.py` runs the phrase-prediction decoder described below. The loader reads weights with `torch.load(..., weights_only=True)`, restores the shared head, and checks the model configuration, parameter count and tokenizer contract. `verify_release.py` checks the complete inventory and sizes without running a model; `--hashes` additionally checks all file bytes against the recorded digests. Checksums are not a digital signature. Keep `source/`, the tokenizer and inference files together.

	## Candidate scoring

	Encode `BOS + context + candidate` jointly. Find the common token prefix of the context and all candidate sequences, then sum full-vocabulary log-softmax over the remaining suffix. Do not append candidate-final EOS; ties keep original order. This suffix likelihood is a scoring proxy, not exact string conditional probability. In deployment, librime with rime-ice produces the pinyin candidates and the model reorders them by pure LM score.

	Phrase prediction (experimental) uses one beam decoder (`source/src/vimeml/benchmarks/predict_zh.py`): earlier turns are joined as separate `BOS turn EOS` sequences, the last pure-Chinese token of the typed text is taken back and re-spelled (token healing), beam width 16 over pure-Chinese tokens, and the top five 2–6 character phrases that are not prefixes of one another are returned.

	## Training

	Random initialization (seed 1591996171), one pass over fresh data, 844,730 updates, batch 512, BF16 with FP32 weights, AdamW (β 0.9/0.95, weight decay 0.1, clip 1.0), cosine learning rate 6e-4→6e-5 after 1,000 warmup updates, and a 30% prefix crop on eligible windows. Total exposure is 14,519,248,976 supervised next-token targets; training took 3.9 hours on a single GPU.

	- Web text: 21 of the 370 train files of [FineWeb2](https://huggingface.co/datasets/HuggingFaceFW/fineweb-2) `cmn_Hani`. Traditional-script documents were converted to Simplified Chinese with OpenCC; rule filters removed spam, gambling, pornographic and illegal-service pages and spam sites; flagged documents were kept only after review. A fixed hash-selected pool of 16,551,843 documents (21.28 billion characters) was seen once, split into paragraphs.
	- Dialogue: independent utterances from [LCCC-base](https://github.com/thu-coai/CDial-GPT) (16,326,323 training utterances, 209 million characters), sampled to 5% of supervised targets, which repeats the pool about four times.

	Every paragraph and utterance is its own `BOS text EOS` sequence; the model never saw multi-turn context during training. The objective is ordinary next-token loss. This model is the release arm (`12k-fresh-s1`) of a preregistered eight-arm experiment comparing fresh data with repeated data and an 8K with a 12K tokenizer. None of the five contrasts was effective, so the preregistered defaults (fresh data, 12K) apply; the final checkpoint was used as registered.

	FP32 validation BPC is 4.354450 on held-out web text and 5.690730 on held-out LCCC dialogue (total NLL including EOS divided by characters × ln 2).

	## Evaluation

	`ime-standard-zh-v1-r1` reranks frozen librime + rime-ice candidate pools for real dialogue turns from CrossWOZ, RiSAWOZ and KdConv, using the left context; correct answers are not added to the pools. Labels are AI-reviewed, not native-speaker gold. The blind split was opened once for a preregistered confirmation, which passed; the baselines were scored afterwards with blind as a reference test.

	\| System \| Parameters \| Development Top-1 \| Blind Top-1 \| Blind Top-3 \| Blind MRR \|
	\| --- \| ---: \| ---: \| ---: \| ---: \| ---: \|
	\| vimeml-tiny-zh-v1 \| 20M \| 93.6% (1,363/1,456) \| 93.8% (1,355/1,445) \| 97.2% \| 0.954 \|
	\| Qwen2.5-0.5B (same pools and scoring) \| 494M \| 91.3% \| 91.8% \| 96.1% \| 0.940 \|
	\| Qwen3-0.6B-Base (same pools and scoring) \| 596M \| 89.8% \| 89.2% \| 95.4% \| 0.924 \|
	\| rime-ice + wanxiang grammar model \| — \| 79.0% \| 82.2% \| 83.7% \| 0.829 \|
	\| rime-ice \| — \| 68.8% \| 72.2% \| 73.6% \| 0.729 \|
	\| luna_pinyin_simp \| — \| 68.3% \| 68.9% \| 70.5% \| 0.696 \|

	Against the rime-ice first candidate on blind: +329/−19, exact McNemar p = 3.6e-74. All five paired comparisons on both splits remain significant after Holm correction; the closest system is Qwen2.5-0.5B (+28 on blind, p = 0.005). Abbreviated pinyin (initials only) is much harder: blind Top-1 35.0%, against 31.0% for Qwen2.5-0.5B and 15.7% for rime-ice + grammar; pool coverage limits it to about 41%.

	Most of the advantage over Rime comes from using left context: with empty context the model scores 81.9% on development, close to rime-ice + grammar. Whether Qwen saw these public dialogue datasets during training is unknown.

	## Core ML variants

	Weight-only symmetric INT8, block size 32; FP32 computation; CPU_ONLY; minimum iOS 18. No activation quantization or INT8 arithmetic is claimed. 47 matrices are compressed (token and position embeddings, fused QKV, attention projection and the three FFN matrices of every layer); RMSNorm weights, the causal mask and a converter-inserted zero head bias remain FP32. The stateless package is 22,792,958 bytes, versus 80,367,198 bytes uncompressed (about 71.6% smaller).

	- Stateless: `input_ids` INT32 `[1,T]`, 1≤T≤128, right PAD → `logits` FLOAT32 `[1,T,12288]`.
	- Chunked KV (22,830,199 bytes): `input_ids` INT32 `[1,T]`, `position` INT32 `[1]` (current cache length, position+T≤128), `key_cache`/`value_cache` FLOAT32 `(9,1,6,128,64)` → logits for the new tokens plus updated caches. The caller owns cache snapshots and rollback.

	```sh
	xcrun coremlcompiler compile coreml/ios18-int8-block32/model.mlpackage compiled-int8 --platform ios --deployment-target 18.0
	```

	Uncompressed conversions pass frozen logits alignment (maximum absolute error 3.9e-5 stateless, 4.6e-5 chunked over 22 checks). INT8 fails strict FP32 logits alignment (maximum 0.392, mean 0.046 at T=128; V4.2 Japanese reached 0.477) at unchanged atol/rtol 3e-4; padding and future-token invariants are exact. The chunked INT8 graph matches the stateless INT8 package within 4.2e-5 with identical rankings. On development, INT8 versus FP32 Top-1 is 1,364 vs 1,363 (+2/−1) on full pinyin and 335 vs 337 (+0/−2) on abbreviated pinyin; Top-3 is unchanged on both. The strict failure is recorded as lossy quantization drift, not numerical equivalence.

	## Performance evidence and limits

	Apple M3 Mac, Core ML CPU_ONLY, measured interleaved with Japanese V4.2 in the same harness (median over five fresh processes):

	\| Metric \| zh-v1 INT8 \| V4.2 INT8 \|
	\| --- \| ---: \| ---: \|
	\| Stateless `predict` p50, T=1 / T=128 \| 1.65 / 8.49 ms \| 1.29 / 6.43 ms \|
	\| Chunked KV prefill of 119 tokens / single-token decode, p50 \| 9.33 / 2.71 ms \| 7.18 / 2.05 ms \|
	\| Model peak memory delta in the host process \| 18.4 MiB \| 13.4 MiB \|

	zh-v1 is about 1.3× V4.2 in size and latency, in line with its larger vocabulary and two extra layers. INT8 reduces size, not CPU latency, because computation stays FP32. A simulator functional check (iOS 27.2) passed: resource identity, logits shape, native SentencePiece tokenization and joint suffix scores within 0.002 of Mac Core ML INT8.

	iPhone 16 Pro Max, iOS 27.3, CPU_ONLY. Vime test host, four rounds alternating the two models, medians:

	\| Metric \| zh-v1 INT8 \| V4.2 INT8 \|
	\| --- \| ---: \| ---: \|
	\| Stateless `predict` p50, T=1 / T=128 \| 1.70 / 6.82 ms \| 1.25 / 4.90 ms \|
	\| Load / reload \| 195 / 7.4 ms \| 155 / 5.9 ms \|
	\| Memory delta after load and 50 predictions \| 14.1 MiB \| 11.3 MiB \|

	The chunked KV package loads in 236–280 ms the first time. A single-token step takes 2.4–2.9 ms p50 and a 128-token prefill 7.5–7.6 ms p50. For 3–6 short candidates the stateless joint scoring is faster (p50 about 9 ms versus 13.5–14 ms with the KV package), so the keyboard uses the stateless package.

	Inside the keyboard extension (Vime integration branch, not yet merged). Rime proposes the candidates. The model rescores at most the first 8 complete conversions after the sentence before the cursor; each place lower in Rime's order costs 1.0 log-probability. The candidate bar waits at most 60 ms for scores, then shows Rime's order. Learned phrases, partial conversions and excluded terms are never moved. Two five-minute runs of XCUITest taps on the installed keyboard (about 1.2 taps/s):

	\| Measured in the extension process \| Model on \| Model off \|
	\| --- \| ---: \| ---: \|
	\| Key → candidates tappable, mean / p95 / max \| 26.1 / 45.0 / 49.7 ms \| 20.3 / 26.1 / 27.4 ms \|
	\| Model scoring, mean / p95 / max \| 13.2 / 36.3 / 40.8 ms \| — \|
	\| Physical footprint median / max; process peak \| 20.0 / 29.5; 29.9 MiB \| 24.3 / 25.3; 25.7 MiB \|
	\| Model load (once) \| 253 ms \| — \|

	p95 is the largest per-window p95 (windows of 256 calls). 221 of 222 scoring opportunities used the model's scores (the other showed Rime's order); 173 of the 221 lists had some order within the top 8 changed, which does not mean the first candidate changed. Scoring is several times slower in the extension than in the test host (per-keystroke scoring p50 1.9–2.0 ms, p95 9.2–9.8 ms over 300 scorings); the cause has not been identified. Chinese next-word prediction in the keyboard, behavior under memory pressure and energy use have not been measured.

	## Intended use and limitations

	Simplified Chinese pinyin candidate reranking with left context, and experimental short phrase prediction, primarily for local input methods. This small non-instruction-tuned model can repeat, drift or invent text. It is not a standalone pinyin converter, general assistant or factual reference.

	- Benchmark labels are AI-reviewed, and the data are dialogue turns rather than real keyboard traffic; candidate-pool coverage bounds achievable accuracy.
	- The model was trained on single paragraphs and single utterances, so multi-turn context is outside its training distribution.
	- Web and social-media text carry their own biases. Cleaning and review reduce but do not remove inappropriate content; applications should keep their own output filtering.
	- Public training overlap with evaluation data was checked by 16-character n-gram matching for the benchmark; semantic near-duplicates were not audited.

	## License and attribution

	The weights, tokenizer, Core ML packages and included VimeML code are released under GNU GPL v3.0 or later with an additional permission under GPLv3 section 7 for conveyance through application distribution services such as the App Store. See `LICENSE` (GPLv3), `LICENSE-EXCEPTION` (copyright, additional permission and the source form of model weights) and `NOTICE.md` (training data attribution that applications bundling this model must show). Training sources and third-party dependencies retain their own terms; they are not relicensed.

	Training data: FineWeb2 `cmn_Hani` (ODC-By 1.0, subject to the Common Crawl Terms of Use) and LCCC-base by thu-coai (Wang et al., A Large-Scale Chinese Short-Text Conversation Dataset, NLPCC 2020), whose repository and dataset card declare the MIT license and whose README states that the dataset and pretrained dialogue models are provided for research purposes. Evaluation sources: CrossWOZ (Apache-2.0), RiSAWOZ (MIT), KdConv (Apache-2.0) and CPED (Apache-2.0). This repository distributes the model and aggregate evaluation metrics only; it contains no corpus, dialogue or benchmark text. Optimizer/RNG state, user inputs, raw device traces, credentials and the separate Vime client are excluded.