| --- |
| license: apache-2.0 |
| license_link: https://ai.google.dev/gemma/docs/gemma_4_license |
| base_model: google/gemma-4-E2B |
| base_model_relation: finetune |
| library_name: mlx |
| pipeline_tag: text-generation |
| tags: |
| - mlx |
| - gemma4 |
| - autocomplete |
| - grammar-correction |
| - lora |
| --- |
| |
| # Pretype One |
|
|
| One model for typing, fixes and replies. It is a smaller Gemma 4 E2B (4-bit MLX) for [Pretype](https://github.com/nikiomori/Pretype), a macOS autocomplete app. |
| One 1.5 GB download does four jobs: |
|
|
| - raw-text autocomplete (no chat template), with the weights in `model.safetensors`; |
| - typo fixes, replies and rewrites through the chat template, with the LoRA adapter in |
| `adapters.safetensors` switched on. |
|
|
| The adapter is not fused: a fuse into 4-bit weights erases it. Load it at run time. With the adapter |
| off, the model is the autocomplete model. |
|
|
|  |
|
|
| ## How it was made |
|
|
| 1. **Smaller token tables.** [mlx-community/gemma-4-e2b-4bit](https://huggingface.co/mlx-community/gemma-4-e2b-4bit) |
| without the audio and vision towers. The vocabulary is cut from 262,144 to 82,000 tokens: the |
| tokens most used in public subtitles, Wikipedia text and code. The tokenizer merges of a |
| removed token are removed too, so its text splits into smaller tokens that stay. The file falls |
| from 2.45 GiB to 1.46 GiB. |
| 2. **Distillation.** Gemma 4 E4B (8-bit) is the teacher, on 10 million tokens of public text in 20 |
| languages (subtitles, Wikipedia, code, and messages that Gemma 4 E4B-it wrote for this |
| purpose). The loss follows the teacher's top 16 tokens. Attention weights, per-layer gates and |
| the down projections learn; the weights are 4-bit again at the end. |
| 3. **Chat adapter.** LoRA on `q_proj` and `v_proj` of all 35 layers, rank 64, scale 20, 5500 steps, |
| on the frozen 4-bit model. Data, 43,500 rows: typo fixes with synthetic typos on subtitle |
| sentences, replies and rewrites that Gemma 4 E4B-it wrote on public subtitle text. |
|
|
| Sentences equal to public evaluation text were dropped. No user data is in any step. |
|
|
| ## Measurements |
|
|
| Pretype's evaluation harness, 2026-11-02 to 2026-11-12. Greedy decoding. The comparison is Gemma 4 |
| E2B 4-bit for autocomplete and its instruct build for the chat jobs: two files, 2.5 GB + 3.0 GB. |
|
|
| | | Comparison | This model | |
| |---|--:|--:| |
| | Autocomplete: first word right, 6,000 rows, 20 languages | 21.2% | 24.2% | |
| | The same, only rows not met in the training text | 21.3% | 23.9% | |
| | Autocomplete: first word right, 5,649 rows of other sources | 20.8% | 22.8% | |
| | Fix: typo line restored exactly, 510 rows | 49% | 79% | |
| | Fix: change proposed on a clean line | 52 of 170 | 6 of 170 | |
| | Reply: answered, in the right language, 380 conversations | 96% | 96% | |
| | Reply: on topic, by a judge model, 380 conversations | 85% | 90% (p=0.04, trend) | |
| | Rewrite: answered, right language, right length, 400 rows | 74% | 86% | |
| | Peak memory of the process | 6.0 GiB | 2.1 GiB | |
| | Energy per suggestion, one session | 1048 mJ | 946 mJ | |
| | Energy per fix, one session | 2291 and 2474 mJ | 2224 mJ | |
|
|
| ## Limits |
|
|
| - **Rewrites can come back unchanged.** 34 of 400 rewrite requests gave the input text back. This |
| occurs most where a small edit is correct. |
| - **Rewrites are measured for form, not for sense.** The test looks at the language and the |
| length. No judge read the rewrites. |
| - **Rewrites are slower than on the instruct build:** 2.5 ms for each output character against |
| 1.8 ms. The adapter is the cause. |
| - **Rare tokens are gone.** Text that needs a removed token costs more tokens. News text lost |
| some accuracy from the cut before the distillation. |
| - **Not a chat model.** The adapter learned three short tasks. It was not tested on open questions. |
| - Energy differs by about 15% between sessions. The energy rows show "not more", not a gain. |
|
|
| ## Use |
|
|
| mlx-lm rejects the unused key/value projections of the KV-shared layers in strict mode, so load |
| without it: |
|
|
| ```python |
| from pathlib import Path |
| from huggingface_hub import snapshot_download |
| from mlx_lm import generate |
| from mlx_lm.utils import load_model, load_tokenizer |
| |
| path = Path(snapshot_download("nikimoriOne/Pretype-One")) |
| model, _ = load_model(path, strict=False) |
| tokenizer = load_tokenizer(path) |
| print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20)) |
| ``` |
|
|
| For the chat jobs, add the adapter (`mlx_lm.tuner.utils.load_adapters(model, str(path))`) and use |
| the chat template. |
|
|
| License: Apache-2.0, the license of Gemma 4. |
|
|
