Model card

README.md

@@ -0,0 +1,73 @@
+---
+license: apache-2.0
+base_model:
+- openbmb/MiniCPM5-2B
+- openbmb/MiniCPM5-2B-DSpark
+pipeline_tag: text-generation
+language:
+- en
+- zh
+tags:
+- radiance
+- rocm
+- amd
+- rdna4
+- bf16
+- speculative-decoding
+- dspark
+---
+
+# MiniCPM5-2B BF16 + DSpark — radiance container
+
+[openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) in bf16, exactly as the checkpoint
+ships it, as a single `.rad` container for the **radiance** inference engine (AMD RDNA4, ROCm), with
+the [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark) block drafter merged in
+for speculative decoding. An FP8 version is at
+[StillDeadcode/minicpm5-2b-fp8](https://huggingface.co/StillDeadcode/minicpm5-2b-fp8).
+
+| | |
+|---|---|
+| File | `minicpm5-2b-bf16.rad` — 5.12 GiB |
+| Weights | every weight of the model, the lm_head included, bf16 as the checkpoint ships it |
+| Speculator | DSpark (5 layers, block size 7). Its weights are FP8 E4M3 per 128×128 block and its vocabulary head 2-bit codes, the one form the engine's DSpark block reads; the drafter only proposes, and the bf16 model verifies every token it emits |
+| Context | 131,072 tokens (the checkpoint's trained length) |
+| Modalities | text |
+
+## Serve
+
+```sh
+radiance --model minicpm5-2b-bf16.rad --tp 2 --max-model-len 131072 --kv-cache-dtype fp8 \
+    --max-num-seqs 8 --host 0.0.0.0 --port 8000
+```
+
+`--tp 1` serves it on one card. The server speaks the OpenAI API (`/v1/chat/completions`, `/v1/completions`), with tool calls and
+structured output; the drafter's
+depth is chosen automatically (`--num-speculative-tokens N` states one, `0` turns speculation off).
+
+Built and tested on Radeon AI PRO R9700 (gfx1201).
+
+## How this file was made
+
+The exact command, with the radiance image built by `docker/build.sh` from radiance commit 93bbe22:
+
+```sh
+docker run --rm --user 1000:1000 -v ~/models:/models --entrypoint rad-convert radiance:93bbe224b392 \
+    /models/openbmb/MiniCPM5-2B --draft-model /models/openbmb/MiniCPM5-2B-DSpark \
+    --tokenizer /models/openbmb/MiniCPM5-2B/tokenizer.json \
+    --recipe /opt/radiance/share/radiance/recipes/minicpm5-2b-bf16-dspark.recipe \
+    -o /models/rad/minicpm5-2b-bf16.rad
+```
+
+Running it twice gives byte-identical files. The recipe (`rad-info --recipe minicpm5-2b-bf16.rad`)
+quantises the drafter only:
+
+```
+dspark.draft_head.weight rtn codes=u2 zero=u8 group=128 scale=f16
+dspark.fc.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.attn_q.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.attn_k.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.attn_v.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.attn_output.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.ffn_gate_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+dspark.blk.*.ffn_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
+```