genmai-90M

A small Japanese kana-kanji conversion model: it converts a reading (katakana) into Japanese text, optionally conditioned on the text before the cursor. It is the light-weight variant of subdiox/genmai-310M, distilled from it, meant for devices with less memory (e.g. iPhone). It uses the same prompt format, tokenizer and file layout as genmai-310M.

• Architecture: GPT-2, character-level vocabulary (6,000), 12 layers, 768 hidden, 90M parameters
• Base model: zenz-v3.2-small (Miwa-Keita/zenz-v3.2-small-gguf, Apache-2.0), dequantized from the Q5_K_M GGUF
• Teacher: genmai-310M

Files

Releases are git tags (v1.0.1); use a tag to pin a version. main is the latest release.

file	contents
model.safetensors	fp32 weights in the Hugging Face GPT-2 layout (362 MB), for transformers
config.json	GPT-2 config; the prompt format is recorded as "genmai_prompt_format": "zenzlc"
tokenizer.json, tokenizer_config.json	the tokenizer
model-coreml.zip	what the genmai input method runs: the model as a stateful Core ML package for the Apple Neural Engine (82 MB; see below): genmai.mlpackage (int8 weights) and genmai_coreml.json (its static shapes: format genmai-coreml-1, prompt_slots 256, output_steps 64, rows 5, prefill_chunk 32)

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("subdiox/genmai-90M", revision="v1.0.1")
tok = AutoTokenizer.from_pretrained("subdiox/genmai-90M", revision="v1.0.1")
x = tok("<s>\uee00" + "ハハハハガイタイラシイ" + "\uee01", return_tensors="pt", add_special_tokens=False)
y = m.generate(**x, num_beams=5, num_return_sequences=5, length_penalty=0.0, max_new_tokens=64)
print([tok.decode(s[x.input_ids.shape[1]:], skip_special_tokens=True) for s in y])  # 母は歯が痛いらしい, ...

Rank beams by the plain sum of log-probabilities (length_penalty=0.0): with transformers' default length normalisation, long degenerate outputs such as ハハハハガイタイらしい can win.

Prompt format (zenzlc)

Left context first (up to the last 500 characters), then the reading, then the output:

<s>[U+EE02]{left context}[U+EE00]{reading in katakana}[U+EE01]{output}</s>

[U+EE0x] are the private-use marker characters U+EE02 (context), U+EE00 (input) and U+EE01 (output). Without context the [U+EE02]{left context} part is omitted. Spaces are written as U+3000 and newlines are removed. Decode with beam search (5 beams) and stop at EOS; long readings are split at 、。 into chunks of at most 40 characters, each decoded after the previous output.

Core ML (Apple Neural Engine)

genmai.mlpackage in model-coreml.zip is the same model converted with coremltools 9 (macOS 15 / iOS 18 or later) for the genmai input method's Core ML decoder. It is one multifunction package with two functions that share one set of states (per layer, the keys and values in fp16):

• prefill: ids, pos (1, 32) int32, start (1,) int32, mask (32, 256) fp16 additive → logp (32, 6000): writes 32 prompt tokens to prompt slots start ..< start + 32 and returns the log-probabilities after each.
• decode: tok (5, 1) int32, pos (1,) int32, wsel (320, 5) / keep (320, 1) fp16, pmask (1, 256) / smask (5, 320) fp16 additive → logp (5, 6000): one token for each of 5 beams. Beam r's token of step t is stored at output slot t · 5 + r (wsel is the one-hot placement, keep = 1 − its row sums); each beam attends to the prompt slots in pmask and to its ancestors' output slots in smask, so reordering beams copies nothing.

Linear weights are int8 (symmetric, per channel), the rest fp16. Prompts are limited to 256 tokens and outputs to 64 tokens per chunk. The conversion script is Tools/coreml/genmai_coreml.py of genmai-ime.

Layer norms divide their input by its maximum magnitude first (epsilon divided by its square; exact): the first token carries a residual value of about 5000, whose square exceeds fp16, and macOS 26 computes a layer norm's variance in fp16 (v1.0.0 gave garbage there; fixed in v1.0.1).

On an M4 Pro (5 beams): the same top-1 as the weights in 8 bits on the GPU for 101 / 101 everyday sentences, 199 / 200 AJIMEE items and 27 / 28 hard sentences; about 4 ms per decode step on the Neural Engine; process footprint about 50–90 MB. The first load compiles the model for the Neural Engine (about 20 s; macOS caches the result).

Training

Knowledge distillation from genmai-310M: at every output position the loss is 0.5 · cross-entropy to the reference + 0.5 · KL divergence to the teacher's next-character distribution (both models share the vocabulary), plus an auxiliary next-character language-model loss (weight 0.3) on the text before the cursor. Stages:

1. ~3M automatically read sentences (Wikipedia, FineWeb-2 Japanese, Aozora Bunko, Diet minutes) and sentences containing rare Mozc dictionary words;
2. ~3.2M further Wikipedia / FineWeb-2 sentences;
3. final stage on curated data (hand-checked items, synthetic and homophone sentences generated with Claude, LLM-judged corrections) with replay of the earlier data.

Evaluation

Top-1 accuracy, beam size 5; candidates whose reading does not match the input are dropped. AJIMEE, everyday sentences and ime_v1 dev are judged leniently (orthographic variants accepted); homophones = word-level accuracy on held-out homophone sentences; hard = 28 sentences written to stress an IME (exact match up to orthography).

set	genmai-90M (bf16)	genmai-310M (bf16)	Zenzai (zenz-v3.2-small, azooKey settings)
AJIMEE-Bench (200)	91.0 %	95.0 %	85.5 %
everyday sentences (101)	98.0 %	98.0 %	97.0 %
ime_v1 dev (544)	94.3 %	93.6 %	–
homophone sentences (1,607)	89.6 %	93.1 %	–
average of the four	93.2 %	94.9 %	–
hard IME sentences (28)	22 / 28	22 / 28	–

Speed (RTX 3090, bf16, 5 beams, everyday sentences): median 18 ms per conversion vs. 36 ms for genmai-310M. In the input method (Core ML, M4 Pro), the conversion key answers in a median 16 / 22 ms (everyday sentences / AJIMEE).

Limitations

• Compared with genmai-310M it is weaker on knowledge-heavy text (rare proper nouns, technical terms) and on fine homophone distinctions (about 3–4 points lower on AJIMEE and the homophone set).
• Readings that repeat one sound (きしゃのきしゃがきしゃできしゃした) tend to map every occurrence to the same word.
• The model copies the left context strongly: a wrong conversion earlier in the context can be repeated.

License and attribution

CC BY-SA 4.0. Base model zenz-v3.2-small by Miwa-Keita (Apache-2.0). The training data include Wikipedia (CC BY-SA), FineWeb-2 (ODC-By), Aozora Bunko, the National Diet minutes, and the Mozc dictionary (BSD-3-Clause, Copyright Google Inc.; contains data derived from IPAdic, Copyright Nara Institute of Science and Technology).