hletrd commited on
Commit
f945cb1
·
verified ·
1 Parent(s): e0dc371

Card: add the KL row, drop the greedy-equality sentence

Browse files
Files changed (1) hide show
  1. README.md +8 -1
README.md CHANGED
@@ -55,6 +55,14 @@ tags:
55
  | Engram tables | as in the base model, in memory-mapped files |
56
  | DFly2 drafter | bf16 |
57
 
 
 
 
 
 
 
 
 
58
  ## How to run
59
 
60
  Download the repository, install `mlx-lm`, and run the chat command from the model folder:
@@ -70,7 +78,6 @@ python -m hy4_mlx.chat --model . # interactive chat
70
 
71
  - Keep the folder on a local SSD: the Engram files are read at random while generating.
72
  - The reasoning effort is `no_think`, `high` or `max`.
73
- - Decoding with the DFly2 drafter checks each draft greedily, so the output equals plain greedy decoding.
74
 
75
  ## How this was made
76
 
 
55
  | Engram tables | as in the base model, in memory-mapped files |
56
  | DFly2 drafter | bf16 |
57
 
58
+ ## Accuracy
59
+
60
+ Mean KL and same top-1 against the BF16 model (the BF16 GGUF trunk with its Engram file in llama.cpp), on 24 chunks of 512 tokens of wikitext-2, measured with MLX 0.32.4 on its CUDA backend and bf16 activations:
61
+
62
+ | Model | Mean KL vs BF16 | Same top-1 vs BF16 |
63
+ |---|---|---|
64
+ | Hy4-flash-MLX-4bit | 0.3066 | 82.8% |
65
+
66
  ## How to run
67
 
68
  Download the repository, install `mlx-lm`, and run the chat command from the model folder:
 
78
 
79
  - Keep the folder on a local SSD: the Engram files are read at random while generating.
80
  - The reasoning effort is `no_think`, `high` or `max`.
 
81
 
82
  ## How this was made
83