nikimori commited on
Commit
0adb812
·
verified ·
1 Parent(s): 70cea30

Card: the name Pretype One, the judged reply figure

Browse files
Files changed (1) hide show
  1. README.md +6 -5
README.md CHANGED
@@ -13,9 +13,9 @@ tags:
13
  - lora
14
  ---
15
 
16
- # Gemma 4 E2B Pretype, 4-bit MLX
17
 
18
- A smaller Gemma 4 E2B for [Pretype](https://github.com/nikiomori/Pretype), a macOS autocomplete app.
19
  One 1.5 GB download does four jobs:
20
 
21
  - raw-text autocomplete (no chat template), with the weights in `model.safetensors`;
@@ -55,6 +55,7 @@ E2B 4-bit for autocomplete and its instruct build for the chat jobs: two files,
55
  | Fix: typo line restored exactly, 510 rows | 49% | 79% |
56
  | Fix: change proposed on a clean line | 52 of 170 | 6 of 170 |
57
  | Reply: answered, in the right language, 380 conversations | 96% | 96% |
 
58
  | Rewrite: answered, right language, right length, 400 rows | 74% | 86% |
59
  | Peak memory of the process | 6.0 GiB | 2.1 GiB |
60
  | Energy per suggestion, one session | 1048 mJ | 946 mJ |
@@ -64,8 +65,8 @@ E2B 4-bit for autocomplete and its instruct build for the chat jobs: two files,
64
 
65
  - **Rewrites can come back unchanged.** 34 of 400 rewrite requests gave the input text back. This
66
  occurs most where a small edit is correct.
67
- - **Replies and rewrites are measured for form, not for sense.** The tests look at the language
68
- and the length. No judge read the answers.
69
  - **Rewrites are slower than on the instruct build:** 2.5 ms for each output character against
70
  1.8 ms. The adapter is the cause.
71
  - **Rare tokens are gone.** Text that needs a removed token costs more tokens. News text lost
@@ -84,7 +85,7 @@ from huggingface_hub import snapshot_download
84
  from mlx_lm import generate
85
  from mlx_lm.utils import load_model, load_tokenizer
86
 
87
- path = Path(snapshot_download("nikimoriOne/gemma-4-e2b-Pretype-4bit"))
88
  model, _ = load_model(path, strict=False)
89
  tokenizer = load_tokenizer(path)
90
  print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20))
 
13
  - lora
14
  ---
15
 
16
+ # Pretype One
17
 
18
+ One model for typing, fixes and replies. It is a smaller Gemma 4 E2B (4-bit MLX) for [Pretype](https://github.com/nikiomori/Pretype), a macOS autocomplete app.
19
  One 1.5 GB download does four jobs:
20
 
21
  - raw-text autocomplete (no chat template), with the weights in `model.safetensors`;
 
55
  | Fix: typo line restored exactly, 510 rows | 49% | 79% |
56
  | Fix: change proposed on a clean line | 52 of 170 | 6 of 170 |
57
  | Reply: answered, in the right language, 380 conversations | 96% | 96% |
58
+ | Reply: on topic, by a judge model, 380 conversations | 85% | 90% (p=0.04, trend) |
59
  | Rewrite: answered, right language, right length, 400 rows | 74% | 86% |
60
  | Peak memory of the process | 6.0 GiB | 2.1 GiB |
61
  | Energy per suggestion, one session | 1048 mJ | 946 mJ |
 
65
 
66
  - **Rewrites can come back unchanged.** 34 of 400 rewrite requests gave the input text back. This
67
  occurs most where a small edit is correct.
68
+ - **Rewrites are measured for form, not for sense.** The test looks at the language and the
69
+ length. No judge read the rewrites.
70
  - **Rewrites are slower than on the instruct build:** 2.5 ms for each output character against
71
  1.8 ms. The adapter is the cause.
72
  - **Rare tokens are gone.** Text that needs a removed token costs more tokens. News text lost
 
85
  from mlx_lm import generate
86
  from mlx_lm.utils import load_model, load_tokenizer
87
 
88
+ path = Path(snapshot_download("nikimoriOne/Pretype-One"))
89
  model, _ = load_model(path, strict=False)
90
  tokenizer = load_tokenizer(path)
91
  print(generate(model, tokenizer, prompt="The quick brown fox", max_tokens=20))