Model card

README.md

@@ -0,0 +1,46 @@
+---
+license: apache-2.0
+base_model:
+- openbmb/MiniCPM5-2B
+- openbmb/MiniCPM5-2B-DSpark
+pipeline_tag: text-generation
+language:
+- en
+- zh
+tags:
+- radiance
+- rocm
+- amd
+- rdna4
+- fp8
+- speculative-decoding
+- dspark
+---
+
+# MiniCPM5-2B FP8 + DSpark — radiance container
+
+[openbmb/MiniCPM5-2B](https://huggingface.co/openbmb/MiniCPM5-2B) as a single `.rad` container for
+the **radiance** inference engine (AMD RDNA4, ROCm), with the
+[MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark) block drafter merged in for
+speculative decoding.
+
+| | |
+|---|---|
+| File | `minicpm5-2b-fp8.rad` — 3.03 GiB |
+| Weights | every linear FP8 E4M3 with a bf16 scale per 128×128 block (round to nearest); embeddings and norms bf16 |
+| Speculator | DSpark (5 layers, block size 7), FP8; its vocabulary head at 2 bits |
+| Context | 131,072 tokens (the checkpoint's trained length) |
+| Modalities | text |
+
+## Serve
+
+```sh
+radiance --model minicpm5-2b-fp8.rad --tp 2 --max-model-len 131072 --kv-cache-dtype fp8 \
+    --max-num-seqs 8 --host 0.0.0.0 --port 8000
+```
+
+`--tp 1` serves it on one card. The server speaks the OpenAI API (`/v1/chat/completions`,
+`/v1/completions`) with tool calls and structured output; the drafter's depth is chosen
+automatically (`--num-speculative-tokens N` states one, `0` turns speculation off).
+
+Built and tested on Radeon AI PRO R9700 (gfx1201).