|
Download README.md from subdiox/genmai-90M: direct link, hf CLI and curl.
- Browser
- Download file 7.36 kB
-
https://huggingface.co/subdiox/genmai-90M/resolve/main/README.md
- Command line
-
hf download hf://subdiox/genmai-90M/README.md
-
curl -L -o README.md https://huggingface.co/subdiox/genmai-90M/resolve/main/README.md
7.36 kB
| license: cc-by-sa-4.0 | |
| language: ja | |
| base_model: Miwa-Keita/zenz-v3.2-small-gguf | |
| pipeline_tag: text-generation | |
| tags: | |
| - japanese | |
| - input-method | |
| - kana-kanji-conversion | |
| # genmai-90M | |
| A small Japanese **kana-kanji conversion** model: it converts a reading (katakana) into Japanese text, optionally | |
| conditioned on the text before the cursor. It is the light-weight variant of | |
| [subdiox/genmai-310M](https://huggingface.co/subdiox/genmai-310M), distilled from it, meant for devices with less | |
| memory (e.g. iPhone). It uses the same prompt format, tokenizer and file layout as genmai-310M. | |
| - Architecture: GPT-2, character-level vocabulary (6,000), 12 layers, 768 hidden, 90M parameters | |
| - Base model: zenz-v3.2-small ([Miwa-Keita/zenz-v3.2-small-gguf](https://huggingface.co/Miwa-Keita/zenz-v3.2-small-gguf), | |
| Apache-2.0), dequantized from the Q5_K_M GGUF | |
| - Teacher: genmai-310M | |
| ## Files | |
| Releases are git tags (`v1.0.1`); use a tag to pin a version. `main` is the latest release. | |
| | file | contents | | |
| |---|---| | |
| | `model.safetensors` | fp32 weights in the Hugging Face GPT-2 layout (362 MB), for transformers | | |
| | `config.json` | GPT-2 config; the prompt format is recorded as `"genmai_prompt_format": "zenzlc"` | | |
| | `tokenizer.json`, `tokenizer_config.json` | the tokenizer | | |
| | `model-coreml.zip` | what the genmai input method runs: the model as a stateful Core ML package for the Apple Neural Engine (82 MB; see below): `genmai.mlpackage` (int8 weights) and `genmai_coreml.json` (its static shapes: `format` `genmai-coreml-1`, `prompt_slots` 256, `output_steps` 64, `rows` 5, `prefill_chunk` 32) | | |
| ```python | |
| from transformers import AutoModelForCausalLM, AutoTokenizer | |
| m = AutoModelForCausalLM.from_pretrained("subdiox/genmai-90M", revision="v1.0.1") | |
| tok = AutoTokenizer.from_pretrained("subdiox/genmai-90M", revision="v1.0.1") | |
| x = tok("<s>\uee00" + "ใใใใใฌใคใฟใคใฉใทใค" + "\uee01", return_tensors="pt", add_special_tokens=False) | |
| y = m.generate(**x, num_beams=5, num_return_sequences=5, length_penalty=0.0, max_new_tokens=64) | |
| print([tok.decode(s[x.input_ids.shape[1]:], skip_special_tokens=True) for s in y]) # ๆฏใฏๆญฏใ็ใใใใ, ... | |
| ``` | |
| Rank beams by the plain sum of log-probabilities (`length_penalty=0.0`): with transformers' default length | |
| normalisation, long degenerate outputs such as ใใใใใฌใคใฟใคใใใ can win. | |
| ## Prompt format (`zenzlc`) | |
| Left context first (up to the last 500 characters), then the reading, then the output: | |
| ``` | |
| <s>[U+EE02]{left context}[U+EE00]{reading in katakana}[U+EE01]{output}</s> | |
| ``` | |
| `[U+EE0x]` are the private-use marker characters U+EE02 (context), U+EE00 (input) and U+EE01 (output). | |
| Without context the `[U+EE02]{left context}` part is omitted. Spaces are written as U+3000 and newlines are | |
| removed. Decode with beam search (5 beams) and stop at EOS; long readings are split at ใใ into chunks of at most | |
| 40 characters, each decoded after the previous output. | |
| ## Core ML (Apple Neural Engine) | |
| `genmai.mlpackage` in `model-coreml.zip` is the same model converted with coremltools 9 (macOS 15 / iOS 18 or later) for the | |
| genmai input method's Core ML decoder. It is one multifunction package with two functions that share one set of | |
| states (per layer, the keys and values in fp16): | |
| - `prefill`: `ids`, `pos` (1, 32) int32, `start` (1,) int32, `mask` (32, 256) fp16 additive โ `logp` (32, 6000): | |
| writes 32 prompt tokens to prompt slots `start ..< start + 32` and returns the log-probabilities after each. | |
| - `decode`: `tok` (5, 1) int32, `pos` (1,) int32, `wsel` (320, 5) / `keep` (320, 1) fp16, `pmask` (1, 256) / | |
| `smask` (5, 320) fp16 additive โ `logp` (5, 6000): one token for each of 5 beams. Beam r's token of step t is | |
| stored at output slot t ยท 5 + r (`wsel` is the one-hot placement, `keep` = 1 โ its row sums); each beam attends to | |
| the prompt slots in `pmask` and to its ancestors' output slots in `smask`, so reordering beams copies nothing. | |
| Linear weights are int8 (symmetric, per channel), the rest fp16. Prompts are limited to 256 tokens and outputs to | |
| 64 tokens per chunk. The conversion script is `Tools/coreml/genmai_coreml.py` of | |
| [genmai-ime](https://github.com/subdiox/genmai-ime). | |
| Layer norms divide their input by its maximum magnitude first (epsilon divided by its square; exact): the first | |
| token carries a residual value of about 5000, whose square exceeds fp16, and macOS 26 computes a layer norm's | |
| variance in fp16 (v1.0.0 gave garbage there; fixed in v1.0.1). | |
| On an M4 Pro (5 beams): the same top-1 as the weights in 8 bits on the GPU for 101 / 101 everyday sentences, | |
| 199 / 200 AJIMEE items and 27 / 28 hard sentences; about 4 ms per decode step on the Neural Engine; process footprint | |
| about 50โ90 MB. The first load compiles the model for | |
| the Neural Engine (about 20 s; macOS caches the result). | |
| ## Training | |
| Knowledge distillation from genmai-310M: at every output position the loss is | |
| 0.5 ยท cross-entropy to the reference + 0.5 ยท KL divergence to the teacher's next-character distribution (both models | |
| share the vocabulary), plus an auxiliary next-character language-model loss (weight 0.3) on the text before the | |
| cursor. Stages: | |
| 1. ~3M automatically read sentences (Wikipedia, FineWeb-2 Japanese, Aozora Bunko, Diet minutes) and sentences | |
| containing rare Mozc dictionary words; | |
| 2. ~3.2M further Wikipedia / FineWeb-2 sentences; | |
| 3. final stage on curated data (hand-checked items, synthetic and homophone sentences generated with Claude, | |
| LLM-judged corrections) with replay of the earlier data. | |
| ## Evaluation | |
| Top-1 accuracy, beam size 5; candidates whose reading does not match the input are dropped. AJIMEE, everyday | |
| sentences and ime_v1 dev are judged leniently (orthographic variants accepted); homophones = word-level accuracy | |
| on held-out homophone sentences; hard = 28 sentences written to stress an IME (exact match up to orthography). | |
| | set | genmai-90M (bf16) | genmai-310M (bf16) | Zenzai (zenz-v3.2-small, azooKey settings) | | |
| |---|---|---|---| | |
| | AJIMEE-Bench (200) | 91.0 % | 95.0 % | 85.5 % | | |
| | everyday sentences (101) | 98.0 % | 98.0 % | 97.0 % | | |
| | ime_v1 dev (544) | 94.3 % | 93.6 % | โ | | |
| | homophone sentences (1,607) | 89.6 % | 93.1 % | โ | | |
| | average of the four | 93.2 % | 94.9 % | โ | | |
| | hard IME sentences (28) | 22 / 28 | 22 / 28 | โ | | |
| Speed (RTX 3090, bf16, 5 beams, everyday sentences): median 18 ms per conversion vs. 36 ms for genmai-310M. In the | |
| input method (Core ML, M4 Pro), the conversion key answers in a median 16 / 22 ms (everyday sentences / AJIMEE). | |
| ## Limitations | |
| - Compared with genmai-310M it is weaker on knowledge-heavy text (rare proper nouns, technical terms) and on fine | |
| homophone distinctions (about 3โ4 points lower on AJIMEE and the homophone set). | |
| - Readings that repeat one sound (ใใใใฎใใใใใใใใงใใใใใ) tend to map every occurrence to the same word. | |
| - The model copies the left context strongly: a wrong conversion earlier in the context can be repeated. | |
| ## License and attribution | |
| CC BY-SA 4.0. Base model zenz-v3.2-small by Miwa-Keita (Apache-2.0). The training data include Wikipedia | |
| (CC BY-SA), FineWeb-2 (ODC-By), Aozora Bunko, the National Diet minutes, and the Mozc dictionary (BSD-3-Clause, | |
| Copyright Google Inc.; contains data derived from IPAdic, Copyright Nara Institute of Science and Technology). | |