File size: 10,224 Bytes

---
license: apache-2.0
language:
  - en
  - zh
base_model:
  - openbmb/MiniCPM5-1B
base_model_relation: quantized
library_name: litert-lm
pipeline_tag: text-generation
tags:
  - litert-lm
  - minicpm
  - minicpm5
  - on-device
  - edge-ai
---

# MiniCPM5-1B — LiteRT-LM (on-device)

Main Model Card: [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)

MiniCPM5-1B packaged in the `.litertlm` format so it runs fully **on-device** — Android, iOS, Desktop and IoT — with the [LiteRT-LM](https://ai.google.dev/edge/litert-lm) framework. No internet connection required at inference time.

## 👉 Not sure which file to pick? Download `MiniCPM5-1B.litertlm`

That's the one and only file here, and it's meant to "just work". You don't need to understand quantization to use it.

| File | What it is | Who it's for |
| :--- | :--- | :--- |
| **`MiniCPM5-1B.litertlm`** | The ready-to-run model (INT8, ~1.1 GB) | **Everyone — this is the default** |
| `chat_template.jinja` | The chat format (auto-used by the runtime) | Advanced / reference |

## Try it in one command (Desktop)

```bash
uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=luispoveda93/MiniCPM5-1B MiniCPM5-1B.litertlm --prompt="What is the capital of France?"
```

## Run it on your phone (Android)

1. Install the [Google AI Edge Gallery](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery) app.
2. Open **Model manager → “+” → Import from HF**, paste this repo's link, and download `MiniCPM5-1B.litertlm`.

See the [Edge Gallery wiki](https://github.com/google-ai-edge/gallery/wiki) for details.

## About this model

MiniCPM5-1B is a compact 1B decoder-only model from OpenBMB (standard `LlamaForCausalLM`), built for local assistants, coding agents, and tool use, with a hybrid `<think>` chat mode and long-context support.

| Item | Value |
| :--- | :--- |
| Parameters | 1,080,632,832 (~1B) |
| Layers | 24 |
| Attention (GQA) | 16 query heads / 2 KV heads |
| Context length | 131,072 |
| Quantization | INT8 dynamic-range (weights), FP32 activations |
| Signatures | `prefill_128`, `prefill_512`, `decode` (KV-cache 1280) |

### Recommended sampling (from the source model card)

| Mode | Params | Enable |
| :--- | :--- | :--- |
| **Think** | `temperature=0.9, top_p=0.95` | `enable_thinking=True` |
| **No Think** | `temperature=0.7, top_p=0.95` | `enable_thinking=False` |

## How this was built (transparency)

Converted from the BF16 source ([openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B)) with Google AI Edge tooling:

- **Re-authoring + TFLite conversion**: `ai-edge-torch` Generative API. MiniCPM5-1B maps onto the standard decoder-only building blocks (hidden 1536, 24 layers, GQA 16/2, head_dim 128, RMSNorm 1e-6, RoPE θ=5e6, untied `lm_head`, vocab 130560).
- **Quantization**: INT8 dynamic-range weight-only.
- **Packaging**: `litert-lm-builder`, native `minicpm5` runtime type, source Hugging Face tokenizer, start token `<s>`, stop tokens `[1, 130073]` (`</s>`, `<|im_end|>`), `thought` reasoning channel, and an `ExecutorMetadata` section declaring all 48 KV-cache state buffers (required for on-device prefill).
- **Verification**: the re-authored model was checked against the original `transformers` implementation — last-token logits matched with **cosine 1.0000**, identical argmax, 5/5 top-5 overlap; parameter count matches exactly. The packaged bundle was also run end-to-end with `litert-lm run` and generates correctly (reasoning channel parses as expected).

> This is a personal community contribution, not an official OpenBMB or Google release. An INT4 (`wi4b32`) variant with a smaller footprint is available from the official [litert-community/MiniCPM5-1B](https://huggingface.co/litert-community/MiniCPM5-1B) repo.

## Community-Contributed Evaluation: Spanish, Catalan, Basque, Galician (0-shot)

> **⚠️ Note:** This section is a **community addition** — it does **not** appear in the original [openbmb/MiniCPM5-1B](https://huggingface.co/openbmb/MiniCPM5-1B) model card. It was benchmarked independently by this repo's maintainer and is provided as supplementary information, not an official OpenBMB result.

To understand how this compact 1B EN/ZH-pretrained model performs on Spain's co-official languages, we ran the exact task selection published on the [Salamandra-2b-instruct model card](https://huggingface.co/BSC-LT/salamandra-2b-instruct) — a 2.25B model whose pretraining specifically oversampled these languages 2x — using the same [LM Evaluation Harness](https://github.com/EleutherAI/lm-evaluation-harness) task suite (SpanishBench, CatalanBench, BasqueBench, GalicianBench), 0-shot, chat template applied. Evaluation was run on the BF16 source model (`openbmb/MiniCPM5-1B`), not the on-device `.litertlm` build in this repo. The Salamandra-2B column is transcribed verbatim from its published card and was not re-run.

### Spanish

<table><thead>
  <tr>
    <th>Category</th>
    <th>Task</th>
    <th>Metric</th>
    <th>MiniCPM5-1B</th>
    <th>Salamandra-2B</th>
  </tr></thead>
<tbody>
  <tr>
    <td>Commonsense Reasoning</td>
    <td>xstorycloze_es</td>
    <td>acc</td>
    <td>51.69</td>
    <td>61.95</td>
  </tr>
  <tr>
    <td rowspan="2">NLI</td>
    <td>wnli_es</td>
    <td>acc</td>
    <td>43.66</td>
    <td>49.30</td>
  </tr>
  <tr>
    <td>xnli_es</td>
    <td>acc</td>
    <td>42.49</td>
    <td>48.52</td>
  </tr>
  <tr>
    <td>Paraphrasing</td>
    <td>paws_es</td>
    <td>acc</td>
    <td><strong>57.30</strong></td>
    <td>57.10</td>
  </tr>
  <tr>
    <td>QA</td>
    <td>xquad_es</td>
    <td>f1<sup>†</sup></td>
    <td>0.00</td>
    <td>29.60</td>
  </tr>
  <tr>
    <td>Translation</td>
    <td>flores_es</td>
    <td>bleu</td>
    <td>0.00</td>
    <td>15.92</td>
  </tr>
</tbody>
</table>

<sup>†</sup> Salamandra's card lists this task under metric "acc"; the installed `xquad_es` task only computes `f1`/`exact_match`, so `f1` is reported here for MiniCPM5-1B.

### Catalan

<table><thead>
  <tr>
    <th>Category</th>
    <th>Task</th>
    <th>Metric</th>
    <th>MiniCPM5-1B</th>
    <th>Salamandra-2B</th>
  </tr></thead>
<tbody>
  <tr>
    <td rowspan="2">Commonsense Reasoning</td>
    <td>copa_ca</td>
    <td>acc</td>
    <td>50.80</td>
    <td>71.00</td>
  </tr>
  <tr>
    <td>xstorycloze_ca</td>
    <td>acc</td>
    <td>48.64</td>
    <td>61.61</td>
  </tr>
  <tr>
    <td rowspan="2">NLI</td>
    <td>wnli_ca</td>
    <td>acc</td>
    <td>54.93</td>
    <td>56.34</td>
  </tr>
  <tr>
    <td>xnli_ca</td>
    <td>acc</td>
    <td>37.83</td>
    <td>52.54</td>
  </tr>
  <tr>
    <td rowspan="2">Paraphrasing</td>
    <td>parafraseja</td>
    <td>acc</td>
    <td>56.75</td>
    <td>64.48</td>
  </tr>
  <tr>
    <td>paws_ca</td>
    <td>acc</td>
    <td>56.45</td>
    <td>60.30</td>
  </tr>
  <tr>
    <td rowspan="5">QA</td>
    <td>arc_ca_easy</td>
    <td>acc</td>
    <td>28.87</td>
    <td>52.15</td>
  </tr>
  <tr>
    <td>arc_ca_challenge</td>
    <td>acc</td>
    <td>20.22</td>
    <td>30.03</td>
  </tr>
  <tr>
    <td>openbookqa_ca</td>
    <td>acc</td>
    <td>21.80</td>
    <td>29.20</td>
  </tr>
  <tr>
    <td>piqa_ca</td>
    <td>acc</td>
    <td>53.32</td>
    <td>63.93</td>
  </tr>
  <tr>
    <td>siqa_ca</td>
    <td>acc</td>
    <td>33.76</td>
    <td>42.32</td>
  </tr>
  <tr>
    <td>Translation</td>
    <td>flores_ca</td>
    <td>bleu</td>
    <td>0.00</td>
    <td>20.04</td>
  </tr>
</tbody></table>

### Basque

<table><thead>
  <tr>
    <th>Category</th>
    <th>Task</th>
    <th>Metric</th>
    <th>MiniCPM5-1B</th>
    <th>Salamandra-2B</th>
  </tr></thead>
<tbody>
  <tr>
    <td rowspan="2">Commonsense Reasoning</td>
    <td>xcopa_eu</td>
    <td>acc</td>
    <td>48.20</td>
    <td>55.20</td>
  </tr>
  <tr>
    <td>xstorycloze_eu</td>
    <td>acc</td>
    <td>49.77</td>
    <td>55.26</td>
  </tr>
  <tr>
    <td rowspan="2">NLI</td>
    <td>wnli_eu</td>
    <td>acc</td>
    <td><strong>52.11</strong></td>
    <td>52.11</td>
  </tr>
  <tr>
    <td>xnli_eu</td>
    <td>acc</td>
    <td>33.91</td>
    <td>45.89</td>
  </tr>
  <tr>
    <td rowspan="3">QA</td>
    <td>eus_exams<sup>‡</sup></td>
    <td>acc</td>
    <td>24.40</td>
    <td>24.94</td>
  </tr>
  <tr>
    <td>eus_proficiency</td>
    <td>acc</td>
    <td>24.28</td>
    <td>25.15</td>
  </tr>
  <tr>
    <td>eus_trivia</td>
    <td>acc</td>
    <td><strong>30.44</strong></td>
    <td>27.35</td>
  </tr>
  <tr>
    <td>Reading Comprehension</td>
    <td>eus_reading</td>
    <td>acc</td>
    <td>25.85</td>
    <td>28.69</td>
  </tr>
  <tr>
    <td>Translation</td>
    <td>flores_eu</td>
    <td>bleu</td>
    <td>0.00</td>
    <td>7.93</td>
  </tr>
</tbody></table>

<sup>‡</sup> `eus_exams` is a group of 29 exam sub-tasks; MiniCPM5-1B's score is the sample-count-weighted average across all sub-tasks.

### Galician

<table><thead>
  <tr>
    <th>Category</th>
    <th>Task</th>
    <th>Metric</th>
    <th>MiniCPM5-1B</th>
    <th>Salamandra-2B</th>
  </tr></thead>
<tbody>
  <tr>
    <td rowspan="2">Paraphrasing</td>
    <td>parafrases_gl</td>
    <td>acc</td>
    <td>52.72</td>
    <td>53.40</td>
  </tr>
  <tr>
    <td>paws_gl</td>
    <td>acc</td>
    <td>53.95</td>
    <td>56.85</td>
  </tr>
  <tr>
    <td>QA</td>
    <td>openbookqa_gl</td>
    <td>acc</td>
    <td>21.60</td>
    <td>26.20</td>
  </tr>
  <tr>
    <td>Translation</td>
    <td>flores_gl</td>
    <td>bleu</td>
    <td>0.00</td>
    <td>17.53</td>
  </tr>
</tbody>
</table>

**Bold** values indicate MiniCPM5-1B matches or exceeds the Salamandra-2B-instruct published baseline despite the parameter and pretraining-data gap. MiniCPM5-1B has no targeted Spanish/Catalan/Basque/Galician pretraining data; the gaps on Basque and Galician (and all FLORES/BLEU translation tasks, which scored ≈0 across every language) are the expected result of that mismatch, not a defect in the on-device conversion. This comparison is offered as context for anyone considering MiniCPM5-1B for these languages — it is **not** a claim of official parity with Salamandra.

## License

[Apache-2.0](https://github.com/OpenBMB/MiniCPM/blob/main/LICENSE), following the source model.