StillDeadcode commited on
Commit
8188c21
·
verified ·
1 Parent(s): b9c0cfc

Conversion recipe

Browse files
Files changed (2) hide show
  1. README.md +3 -2
  2. minicpm5-2b-bf16-dspark.recipe +18 -0
README.md CHANGED
@@ -58,8 +58,9 @@ docker run --rm --user 1000:1000 -v ~/models:/models --entrypoint rad-convert ra
58
  -o /models/rad/minicpm5-2b-bf16.rad
59
  ```
60
 
61
- Running it twice gives byte-identical files. The recipe (`rad-info --recipe minicpm5-2b-bf16.rad`)
62
- quantises the drafter only:
 
63
 
64
  ```
65
  dspark.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
 
58
  -o /models/rad/minicpm5-2b-bf16.rad
59
  ```
60
 
61
+ Running it twice gives byte-identical files. The recipe file the command names is in this
62
+ repository as `minicpm5-2b-bf16-dspark.recipe`, with its comments. Its rules, as the container
63
+ records them (`rad-info --recipe minicpm5-2b-bf16.rad`), quantise the drafter only:
64
 
65
  ```
66
  dspark.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
minicpm5-2b-bf16-dspark.recipe ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # minicpm5-2b-bf16-dspark.recipe -- MiniCPM5-2B in bf16 as the checkpoint ships it, with the DSpark
2
+ # drafter merged in (--draft-model). Every weight of the model itself, the lm_head included, is the
3
+ # checkpoint's own; the drafter is quantised as in minicpm5-2b-dspark.recipe, because the DSpark
4
+ # block reads its weights in one form only: block fp8, and the vocabulary head as 2-bit codes (the
5
+ # head only proposes; the model's own bf16 lm_head scores every token it accepts).
6
+ #
7
+ # rad-convert <MiniCPM5-2B checkpoint> --draft-model <MiniCPM5-2B-DSpark checkpoint> \
8
+ # --tokenizer <MiniCPM5-2B checkpoint>/tokenizer.json \
9
+ # --recipe data/recipes/minicpm5-2b-bf16-dspark.recipe -o minicpm5-2b-bf16.rad
10
+
11
+ dspark.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
12
+ dspark.fc.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
13
+ dspark.blk.*.attn_q.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
14
+ dspark.blk.*.attn_k.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
15
+ dspark.blk.*.attn_v.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
16
+ dspark.blk.*.attn_output.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
17
+ dspark.blk.*.ffn_gate_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
18
+ dspark.blk.*.ffn_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16