Blame for README.md

---
license: apache-2.0
license_link: https://ai.google.dev/gemma/docs/gemma_4_license
base_model: google/gemma-4-E2B
base_model_relation: finetune
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- gemma4
- autocomplete
- grammar-correction
- lora
---

# Pretype One

One model for typing, fixes and replies. It is a smaller Gemma 4 E2B (4-bit MLX) for [Pretype](https://github.com/nikiomori/Pretype), a macOS autocomplete app.
One 1.5 GB download does four jobs:

- raw-text autocomplete (no chat template), with the weights in `model.safetensors`;
- typo fixes, replies and rewrites through the chat template, with the LoRA adapter in
  `adapters.safetensors` switched on.

The adapter is not fused: a fuse into 4-bit weights erases it. Load it at run time. With the adapter
off, the model is the autocomplete model.

![Pretype One against the other models of Pretype: download size, autocomplete, typo fixes and replies](benchmark.png)

## How it was made

1. **Smaller token tables.** [mlx-community/gemma-4-e2b-4bit](https://huggingface.co/mlx-community/gemma-4-e2b-4bit)
   without the audio and vision towers. The vocabulary is cut from 262,144 to 82,000 tokens: the
   tokens most used in public subtitles, Wikipedia text and code. The tokenizer merges of a
   removed token are removed too, so its text splits into smaller tokens that stay. The file falls
   from 2.45 GiB to 1.46 GiB.
2. **Distillation.** Gemma 4 E4B (8-bit) is the teacher, on 10 million tokens of public text in 20
   languages (subtitles, Wikipedia, code, and messages that Gemma 4 E4B-it wrote for this
   purpose). The loss follows the teacher's top 16 tokens. Attention weights, per-layer gates and
   the down projections learn; the weights are 4-bit again at the end.
3. **Chat adapter.** LoRA on `q_proj` and `v_proj` of all 35 layers, rank 64, scale 20, 5500 steps,
   on the frozen 4-bit model. Data, 43,500 rows: typo fixes with synthetic typos on subtitle
   sentences, replies and rewrites that Gemma 4 E4B-it wrote on public subtitle text.

Sentences equal to public evaluation text were dropped. No user data is in any step.

## Measurements

Pretype's evaluation harness, 2026-11-02 to 2026-11-12. Greedy decoding. The comparison is Gemma 4
E2B 4-bit for autocomplete and its instruct build for the chat jobs: two files, 2.5 GB + 3.0 GB.

| | Comparison | This model |
|---|--:|--:|
| Autocomplete: first word right, 6,000 rows, 20 languages | 21.2% | 24.2% |
| The same, only rows not met in the training text | 21.3% | 23.9% |
| Autocomplete: first word right, 5,649 rows of other sources | 20.8% | 22.8% |
| Fix: typo line restored exactly, 510 rows | 49% | 79% |
| Fix: change proposed on a clean line | 52 of 170 | 6 of 170 |
| Reply: answered, in the right language, 380 conversations | 96% | 96% |
| Reply: on topic, by a judge model, 380 conversations | 85% | 90% (p=0.04, trend) |
| Rewrite: answered, right language, right length, 400 rows | 74% | 86% |
| Peak memory of the process | 6.0 GiB | 2.1 GiB |
| Energy per suggestion, one session | 1048 mJ | 946 mJ |
| Energy per fix, one session | 2291 and 2474 mJ | 2224 mJ |

## Limits

- **Rewrites can come back unchanged.** 34 of 400 rewrite requests gave the input text back. This
  occurs most where a small edit is correct.
- **Rewrites are measured for form, not for sense.** The test looks at the language and the
  length. No judge read the rewrites.
- **Rewrites are slower than on the instruct build:** 2.5 ms for each output character against
  1.8 ms. The adapter is the cause.
- **Rare tokens are gone.** Text that needs a removed token costs more tokens. News text lost
  some accuracy from the cut before the distillation.
- **Not a chat model.** The adapter learned three short tasks. It was not tested on open questions.
- Energy differs by about 15% between sessions. The energy rows show "not more", not a gain.

## Use

mlx-lm rejects the unused key/value projections of the KV-shared layers in strict mode, so load
without it:

```python
from pathlib import Path
from huggingface_hub import snapshot_download
from mlx_lm import generate
from mlx_lm.utils import load_model, load_tokenizer

path = Path(snapshot_download("nikimoriOne/Pretype-One"))
model, _ = load_model(path, strict=False)
tokenizer = load_tokenizer(path)
print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20))
```

For the chat jobs, add the adapter (`mlx_lm.tuner.utils.load_adapters(model, str(path))`) and use
the chat template.

License: Apache-2.0, the license of Gemma 4.