---
base_model: openbmb/MiniCPM5-2B
base_model_relation: quantized
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
tags:
  - minicpm
  - compressed-tensors
  - vllm
  - quantized
  - fp8
---
# MiniCPM5-2B · FP8 (dynamic)

8-bit quantization of [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) for **vLLM serving**. Quantized by **axteam**.

| | |
|---|---|
| **Base model** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) |
| **Method** | FP8 (dynamic) |
| **Format** | compressed-tensors `FP8_DYNAMIC` |
| **Weights / activations** | W8A8 — FP8 E4M3 weights (per-channel), dynamic per-token FP8 activations |
| **Architecture** | Dense (Llama-architecture) · text-only |
| **Size on disk** | ~2.8 GB |
| **Calibration** | None (data-free) |
| **License** | Apache-2.0 |

## Quantization details
- **Scheme** — W8A8 — FP8 E4M3 weights (per-channel), dynamic per-token FP8 activations, stored as compressed-tensors `FP8_DYNAMIC`.
- **Kept in BF16** — token embeddings, the LM head, and all RMSNorms (a small fraction of parameters).
- Post-training quantization (no retraining). Tokenizer / `chat_template` identical to the original.

## Evaluation (measured)

Fidelity to the BF16 original on an internal mixed EN/KO set (100 prompts, BF16 greedy answers scored under each build).

| Metric | This model | BF16 original |
|---|---|---|
| Perplexity (lower = closer) | 1.6494 | 1.5988 (**+3.2%**) |
| Top-1 next-token agreement | 92.9% | 98.9% |


## Usage (vLLM)
```bash
vllm serve axteam/MiniCPM5-2B-FP8-dynamic
```

## License
Quantized derivative of [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) (Apache-2.0). Original weights & tokenizer belong to OpenBMB.

---
## 한국어 요약
[`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) 를 **FP8 (dynamic)** 로 양자화한 서빙용(8-bit) 모델입니다. 작성: **axteam**.
- **방식** — FP8 E4M3, 가중치 채널별 스케일(정적·보정불필요), 활성화 동적 FP8. 형식 compressed-tensors `FP8_DYNAMIC`.
- **BF16 유지** — 토큰 임베딩·LM 헤드·모든 norm.
- PTQ(재학습 없음), 토크나이저·chat_template 원본 동일.
- **품질(실측)** — BF16 대비 perplexity 1.6494(원본 1.5988, **+3.2%**), top-1 92.9%(원본 98.9%). 내부 영/한 100문항.
- **사용** — `vllm serve axteam/MiniCPM5-2B-FP8-dynamic`. 라이선스 Apache-2.0(원본 OpenBMB).