Paris99 commited on
Commit
7ed5dc8
·
verified ·
1 Parent(s): bc061b5

Update model card

Browse files
Files changed (1) hide show
  1. README.md +17 -13
README.md CHANGED
@@ -36,14 +36,18 @@ image understanding and 256K context, and thinks before it answers by default. R
36
  Publisher's figures for the original BF16/FP8 weights; see the [original model card](https://huggingface.co/RadixArk/DaisyRose-27B)
37
  for the harnesses, repetitions and other models.
38
 
39
- | Build | Size | Same next token as bf16 | KL vs bf16 | Peak memory, 8K-token prompt (mlx-lm, text) |
40
- |---|---|---|---|---|
41
- | [netprojects/DaisyRose-27B-MLX-6bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-6bit) (recommended) | 23.1 GB | 98.7% | 0.0014 | 23.4 GB |
42
- | [netprojects/DaisyRose-27B-MLX-8bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-8bit) | 29.9 GB | 99.5% | 0.0004 | 29.7 GB |
43
- | [netprojects/DaisyRose-27B-MLX-4bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-4bit) | 16.3 GB | 95.8% | 0.0207 | 17.1 GB |
44
- | [netprojects/DaisyRose-27B-MLX-bf16](https://huggingface.co/netprojects/DaisyRose-27B-MLX-bf16) | 55.6 GB | reference | reference | 53.4 GB |
 
45
 
46
- The 6-bit build is the recommended balance of size and fidelity; 8-bit is closest to bf16, 4-bit is the smallest.
 
 
 
47
 
48
  ## Use with mlx-lm
49
 
@@ -87,7 +91,7 @@ mlx_vlm.generate --model netprojects/DaisyRose-27B-MLX-6bit --draft-model netpro
87
  --temperature 0 --max-tokens 512 --prompt "Write a Python function that checks whether a number is prime."
88
  ```
89
 
90
- Every drafted token is checked by the full model; in our greedy check the text was identical to plain decoding.
91
 
92
  ## What was checked
93
 
@@ -95,12 +99,12 @@ On a MacBook Pro M5 Max (128 GB), 19 November 2026, with mlx 0.32.4, mlx-lm 0.32
95
 
96
  - **Source.** Every file of RadixArk/DaisyRose-27B at revision `293a7a2` matches the Hub's SHA-256 hashes; its config and tensor list are identical to Qwen3.8-27B's.
97
  - **Quantization.** Each quantized build is checked to be exactly the affine (group 64) quantization of the bf16 build.
98
- - **Agreement with bf16** (12 short prompts, 9 with thinking off and 3 with thinking on, 953 teacher-forced positions): the 8-bit build picks the same next token as bf16 on 99.5% of positions, 6-bit on 98.7% and 4-bit on 95.8% (mean KL 0.0004 / 0.0014 / 0.0207).
99
- - **Tool calling and coding** (thinking off): one agent-style prompt with `bash` and `read` tools and three short Python tasks run against asserts: 8-bit passed, 6-bit passed, 4-bit passed.
100
- - **Thinking** (two short maths prompts, reasoning effort low): the reply closes `</think>` once and the answer is right: 8-bit 2/2, 6-bit 2/2, 4-bit 2/2.
101
- - **GSM8K, 50 fixed test questions** (thinking off, greedy, 512-token cap): 6-bit 44/50, 4-bit 45/50.
102
  - **Vision** (mlx-vlm): asked to name the shapes and colours in a test image, every build answered correctly and answered a text-only question in the same session.
103
- - **MTP layer.** The bundled MTP tensors equal DaisyRose-27B's `mtp.*` weights (norms in mlx-vlm's stored form), quantized exactly. The 8-bit, 6-bit and 4-bit builds were loaded in mlx-lm and mlx-vlm with the extra file present, and greedy MTP decoding gave the same text as plain decoding.
104
 
105
  ## Credits
106
 
 
36
  Publisher's figures for the original BF16/FP8 weights; see the [original model card](https://huggingface.co/RadixArk/DaisyRose-27B)
37
  for the harnesses, repetitions and other models.
38
 
39
+ | Build | Size | Bits per weight | Same next token as bf16 | KL vs bf16 | GSM8K, 50 questions | Peak memory, 8K-token prompt | Recommended Mac | Tool call / coding / thinking / vision |
40
+ |---|---|---|---|---|---|---|---|---|
41
+ | [6-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-6bit) (recommended) | 23.1 GB | 6.66 | 98.7% | 0.0014 | 44/50 | 23.4 GB | 36 GB+ | pass / 3/3 / 2/2 / pass |
42
+ | [8-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-8bit) | 29.9 GB | 8.63 | 99.5% | 0.0004 | – | 29.7 GB | 48 GB+ | pass / 3/3 / 2/2 / pass |
43
+ | [4-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-4bit) | 16.3 GB | 4.70 | 95.8% | 0.0207 | 45/50 | 17.1 GB | 32 GB+ | pass / 3/3 / 2/2 / pass |
44
+ | [3-bit](https://huggingface.co/netprojects/DaisyRose-27B-MLX-3bit) | 13.8 GB | 3.91 | 92.1% | 0.0814 | 43/50 | 14.6 GB | 24 GB+ | pass / 3/3 / 2/2 / pass |
45
+ | [bf16](https://huggingface.co/netprojects/DaisyRose-27B-MLX-bf16) | 55.6 GB | 16 | reference | – | – | 53.4 GB | 96 GB+ | – / – / – / pass |
46
 
47
+ Measured on our builds as described under "What was checked" (GSM8K: thinking off, greedy; peak memory: mlx-lm, text). "Recommended Mac" is the smallest Mac whose default GPU memory limit (about two thirds of RAM up to 36 GB, three quarters above) holds that peak; longer contexts need more.
48
+
49
+ On a smaller Mac you can raise the GPU memory limit until the next restart, for example `sudo sysctl iogpu.wired_limit_mb=27000` on a 32 GB Mac
50
+ (leave several GB for macOS and your apps). With a raised limit and short contexts the next size down usually works: 6-bit on 32 GB, 4-bit on 24 GB. The 6-bit build is the recommended balance of size and fidelity; 8-bit is closest to bf16; 4-bit and 3-bit are for Macs with less memory (the 3-bit build fits 24 GB Macs).
51
 
52
  ## Use with mlx-lm
53
 
 
91
  --temperature 0 --max-tokens 512 --prompt "Write a Python function that checks whether a number is prime."
92
  ```
93
 
94
+ Every drafted token is checked by the full model. With greedy decoding the text matches plain decoding until a near-tie, where the batched check can round the other way, so long answers can diverge slightly.
95
 
96
  ## What was checked
97
 
 
99
 
100
  - **Source.** Every file of RadixArk/DaisyRose-27B at revision `293a7a2` matches the Hub's SHA-256 hashes; its config and tensor list are identical to Qwen3.8-27B's.
101
  - **Quantization.** Each quantized build is checked to be exactly the affine (group 64) quantization of the bf16 build.
102
+ - **Agreement with bf16** (12 short prompts, 9 with thinking off and 3 with thinking on, 953 teacher-forced positions): the 8-bit build picks the same next token as bf16 on 99.5% of positions, 6-bit on 98.7%, 4-bit on 95.8% and 3-bit on 92.1% (mean KL 0.0004 / 0.0014 / 0.0207 / 0.0814).
103
+ - **Tool calling and coding** (thinking off): one agent-style prompt with `bash` and `read` tools and three short Python tasks run against asserts: 8-bit passed, 6-bit passed, 4-bit passed, 3-bit passed.
104
+ - **Thinking** (two short maths prompts, reasoning effort low): the reply closes `</think>` once and the answer is right: 8-bit 2/2, 6-bit 2/2, 4-bit 2/2, 3-bit 2/2.
105
+ - **GSM8K, 50 fixed test questions** (thinking off, greedy, 512-token cap): 6-bit 44/50, 4-bit 45/50, 3-bit 43/50.
106
  - **Vision** (mlx-vlm): asked to name the shapes and colours in a test image, every build answered correctly and answered a text-only question in the same session.
107
+ - **MTP layer.** The bundled MTP tensors equal DaisyRose-27B's `mtp.*` weights (norms in mlx-vlm's stored form), quantized exactly. The 8-bit, 6-bit, 4-bit and 3-bit builds were loaded in mlx-lm and mlx-vlm with the extra file present and decoded with their bundled MTP head (the 3-bit build keeps an 8-bit head, declared in `config.json` as `mtplx_mtp_quantization`).
108
 
109
  ## Credits
110