Paris99 commited on
Commit
9eadf89
·
verified ·
1 Parent(s): 7ed5dc8

Update model card

Browse files
Files changed (1) hide show
  1. README.md +10 -9
README.md CHANGED
@@ -39,15 +39,16 @@ for the harnesses, repetitions and other models.
39
  | Build | Size | Bits per weight | Same next token as bf16 | KL vs bf16 | GSM8K, 50 questions | Peak memory, 8K-token prompt | Recommended Mac | Tool call / coding / thinking / vision |
40
  |---|---|---|---|---|---|---|---|---|
41
  | [6-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-6bit) (recommended) | 23.1 GB | 6.66 | 98.7% | 0.0014 | 44/50 | 23.4 GB | 36 GB+ | pass / 3/3 / 2/2 / pass |
42
- | [8-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-8bit) | 29.9 GB | 8.63 | 99.5% | 0.0004 | – | 29.7 GB | 48 GB+ | pass / 3/3 / 2/2 / pass |
43
  | [4-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-4bit) | 16.3 GB | 4.70 | 95.8% | 0.0207 | 45/50 | 17.1 GB | 32 GB+ | pass / 3/3 / 2/2 / pass |
44
  | [3-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-3bit) | 13.8 GB | 3.91 | 92.1% | 0.0814 | 43/50 | 14.6 GB | 24 GB+ | pass / 3/3 / 2/2 / pass |
 
45
  | [bf16](https://huggingface.co/netprojects/DaisyRose-27B-MLX-bf16) | 55.6 GB | 16 | reference | – | – | 53.4 GB | 96 GB+ | – / – / – / pass |
46
 
47
  Measured on our builds as described under "What was checked" (GSM8K: thinking off, greedy; peak memory: mlx-lm, text). "Recommended Mac" is the smallest Mac whose default GPU memory limit (about two thirds of RAM up to 36 GB, three quarters above) holds that peak; longer contexts need more.
48
 
49
  On a smaller Mac you can raise the GPU memory limit until the next restart, for example `sudo sysctl iogpu.wired_limit_mb=27000` on a 32 GB Mac
50
- (leave several GB for macOS and your apps). With a raised limit and short contexts the next size down usually works: 6-bit on 32 GB, 4-bit on 24 GB. The 6-bit build is the recommended balance of size and fidelity; 8-bit is closest to bf16; 4-bit and 3-bit are for Macs with less memory (the 3-bit build fits 24 GB Macs).
51
 
52
  ## Use with mlx-lm
53
 
@@ -98,13 +99,13 @@ Every drafted token is checked by the full model. With greedy decoding the text
98
  On a MacBook Pro M5 Max (128 GB), 19 November 2026, with mlx 0.32.4, mlx-lm 0.32.2 and mlx-vlm 0.7.11.
99
 
100
  - **Source.** Every file of RadixArk/DaisyRose-27B at revision `293a7a2` matches the Hub's SHA-256 hashes; its config and tensor list are identical to Qwen3.8-27B's.
101
- - **Quantization.** Each quantized build is checked to be exactly the affine (group 64) quantization of the bf16 build.
102
- - **Agreement with bf16** (12 short prompts, 9 with thinking off and 3 with thinking on, 953 teacher-forced positions): the 8-bit build picks the same next token as bf16 on 99.5% of positions, 6-bit on 98.7%, 4-bit on 95.8% and 3-bit on 92.1% (mean KL 0.0004 / 0.0014 / 0.0207 / 0.0814).
103
- - **Tool calling and coding** (thinking off): one agent-style prompt with `bash` and `read` tools and three short Python tasks run against asserts: 8-bit passed, 6-bit passed, 4-bit passed, 3-bit passed.
104
- - **Thinking** (two short maths prompts, reasoning effort low): the reply closes `</think>` once and the answer is right: 8-bit 2/2, 6-bit 2/2, 4-bit 2/2, 3-bit 2/2.
105
- - **GSM8K, 50 fixed test questions** (thinking off, greedy, 512-token cap): 6-bit 44/50, 4-bit 45/50, 3-bit 43/50.
106
- - **Vision** (mlx-vlm): asked to name the shapes and colours in a test image, every build answered correctly and answered a text-only question in the same session.
107
- - **MTP layer.** The bundled MTP tensors equal DaisyRose-27B's `mtp.*` weights (norms in mlx-vlm's stored form), quantized exactly. The 8-bit, 6-bit, 4-bit and 3-bit builds were loaded in mlx-lm and mlx-vlm with the extra file present and decoded with their bundled MTP head (the 3-bit build keeps an 8-bit head, declared in `config.json` as `mtplx_mtp_quantization`).
108
 
109
  ## Credits
110
 
 
39
  | Build | Size | Bits per weight | Same next token as bf16 | KL vs bf16 | GSM8K, 50 questions | Peak memory, 8K-token prompt | Recommended Mac | Tool call / coding / thinking / vision |
40
  |---|---|---|---|---|---|---|---|---|
41
  | [6-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-6bit) (recommended) | 23.1 GB | 6.66 | 98.7% | 0.0014 | 44/50 | 23.4 GB | 36 GB+ | pass / 3/3 / 2/2 / pass |
42
+ | [8-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-8bit) | 29.9 GB | 8.63 | 99.5% | 0.0004 | 45/50 | 29.7 GB | 48 GB+ | pass / 3/3 / 2/2 / pass |
43
  | [4-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-4bit) | 16.3 GB | 4.70 | 95.8% | 0.0207 | 45/50 | 17.1 GB | 32 GB+ | pass / 3/3 / 2/2 / pass |
44
  | [3-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-3bit) | 13.8 GB | 3.91 | 92.1% | 0.0814 | 43/50 | 14.6 GB | 24 GB+ | pass / 3/3 / 2/2 / pass |
45
+ | [2.5-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-2.5bit) | 11.8 GB | 3.65 | 87.6% | 0.2138 | 42/50 | 13.8 GB | 24 GB+ | pass / 2/3 / 2/2 / text-only |
46
  | [bf16](https://huggingface.co/netprojects/DaisyRose-27B-MLX-bf16) | 55.6 GB | 16 | reference | – | – | 53.4 GB | 96 GB+ | – / – / – / pass |
47
 
48
  Measured on our builds as described under "What was checked" (GSM8K: thinking off, greedy; peak memory: mlx-lm, text). "Recommended Mac" is the smallest Mac whose default GPU memory limit (about two thirds of RAM up to 36 GB, three quarters above) holds that peak; longer contexts need more.
49
 
50
  On a smaller Mac you can raise the GPU memory limit until the next restart, for example `sudo sysctl iogpu.wired_limit_mb=27000` on a 32 GB Mac
51
+ (leave several GB for macOS and your apps). With a raised limit and short contexts the next size down usually works: 6-bit on 32 GB, 4-bit on 24 GB. The 6-bit build is the recommended balance of size and fidelity; 8-bit is closest to bf16; 4-bit, 3-bit and the text-only 2.5-bit are for Macs with less memory (3-bit and 2.5-bit fit 24 GB Macs).
52
 
53
  ## Use with mlx-lm
54
 
 
99
  On a MacBook Pro M5 Max (128 GB), 19 November 2026, with mlx 0.32.4, mlx-lm 0.32.2 and mlx-vlm 0.7.11.
100
 
101
  - **Source.** Every file of RadixArk/DaisyRose-27B at revision `293a7a2` matches the Hub's SHA-256 hashes; its config and tensor list are identical to Qwen3.8-27B's.
102
+ - **Quantization.** Each quantized build is checked to be exactly the affine (group 64) quantization of the bf16 build, with the bit widths described above.
103
+ - **Agreement with bf16** (12 short prompts, 9 with thinking off and 3 with thinking on, 953 teacher-forced positions): share of positions where each build picks the same next token as bf16: 8-bit 99.5% (KL 0.0004); 6-bit 98.7% (KL 0.0014); 4-bit 95.8% (KL 0.0207); 3-bit 92.1% (KL 0.0814); 2.5-bit 87.6% (KL 0.2138).
104
+ - **Tool calling and coding** (thinking off): one agent-style prompt with `bash` and `read` tools and three short Python tasks run against asserts: 8-bit tool call passed, coding 3/3; 6-bit tool call passed, coding 3/3; 4-bit tool call passed, coding 3/3; 3-bit tool call passed, coding 3/3; 2.5-bit tool call passed, coding 2/3.
105
+ - **Thinking** (two short maths prompts, reasoning effort low): the reply closes `</think>` once and the answer is right: 8-bit 2/2; 6-bit 2/2; 4-bit 2/2; 3-bit 2/2; 2.5-bit 2/2.
106
+ - **GSM8K, 50 fixed test questions** (thinking off, greedy, 512-token cap): 8-bit 45/50, 6-bit 44/50, 4-bit 45/50, 3-bit 43/50, 2.5-bit 42/50.
107
+ - **Vision** (mlx-vlm): asked to name the shapes and colours in a test image, every build with vision (all but the text-only 2.5-bit) answered correctly and answered a text-only question in the same session.
108
+ - **MTP layer.** The bundled MTP tensors equal DaisyRose-27B's `mtp.*` weights (norms in mlx-vlm's stored form), quantized exactly. Every quantized build was loaded in mlx-lm and mlx-vlm with the extra file present and decoded with its bundled MTP head (the 3-bit build keeps an 8-bit head and the 2.5-bit build a 4-bit head, declared in `config.json` as `mtplx_mtp_quantization`).
109
 
110
  ## Credits
111