---
license: apache-2.0
base_model:
- Qwen/Qwen2.5-0.5B-Instruct
base_model_relation: quantized
library_name: executorch
pipeline_tag: text-generation
inference: false
tags:
- executorch
- webgpu
- on-device
- browser
---
# Qwen2.5 0.5B Instruct for ExecuTorch WebGPU

This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
[Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).

- **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
- **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
- **Quantization:** 4-bit groupwise weights (q4gsw), fp16 KV cache
- **Total size:** 296.5 MB

## About the original model

The 0.5B instruction-tuned model of the Qwen2.5 family.

- **Developed by:** Qwen Team, Alibaba Cloud
- **Paper:** [Qwen2.5 Technical Report](https://arxiv.org/abs/2412.15115)
- 0.49B parameters, 24 layers
- Context length 32,768 tokens; generation up to 8,192 tokens
- Supports over 29 languages

For training data, evaluation results and full details, see the original model card: [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).

## Intended use and limitations

Chat and instruction following in a small footprint.

Small language models can state false information confidently, repeat themselves, and reflect biases in their training data. Check important outputs.

## What changed in this conversion

- Weights quantized to 4-bit groupwise (q4gsw); the KV cache is fp16
- Context capped at 8,192 tokens (upstream: 32,768)

## Inputs and outputs

- `forward(tokens int64 [1, 1..512], input_pos int64 [1]) -> logits float32 [1, 151936]` (last position only)
- The KV cache lives inside the program; call it with up to 512 new tokens at a time, then one token per step
- Tokenize with the files in `tokenizer/` (the upstream Qwen2.5 tokenizer) and the ChatML template

## Files

| File | Size | SHA-256 |
|---|---|---|
| `qwen2_5_0_5b_instruct_q4gsw.pte` | 285.6 MB | `474274b934978f6c…` |
| `tokenizer/config.json` | 659 B | `18e18afcaccafade…` |
| `tokenizer/generation_config.json` | 242 B | `e558847a8b440261…` |
| `tokenizer/merges.txt` | 1.6 MB | `599bab5407508877…` |
| `tokenizer/tokenizer.json` | 6.7 MB | `c0382117ea329cdf…` |
| `tokenizer/tokenizer_config.json` | 7.1 KB | `5b5d4f65d0acd3b2…` |
| `tokenizer/vocab.json` | 2.6 MB | `ca10d7e9fb3ed185…` |

Full checksums are in `manifest.json`. The playground verifies them after download.

## Running it

These programs were exported with the ExecuTorch WebGPU backend
([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
the playground is the supported way to run them. It downloads these files once, caches them in the browser,
and runs everything locally.

Requirements: a browser with WebGPU enabled. The language models use the `shader-f16` GPU feature when available and fall back to fp32 math without it.

## Provenance

- **Upstream:** `Qwen/Qwen2.5-0.5B-Instruct` at revision `7ae557604adf67be50417f59c2c2f167def9a775` (official)
- **Exported with:** executorch 1.5.1+cpu, torch 2.14.1+cpu, torchao 0.18.0+cpu, transformers 5.19.0 (executorch git `3b6068392324`)
- **Reproduce:** `python recipes/qwen2.5-0.5b-instruct.py --out out/qwen2.5-0.5b-instruct` (recipe in `export/`)
- Re-running the recipe produces byte-identical files.

## Citation

If you use this model, please cite the original work:

```bibtex
@misc{qwen2.5,
  title={Qwen2.5: A Party of Foundation Models},
  url={https://qwenlm.github.io/blog/qwen2.5/},
  author={Qwen Team},
  month={September},
  year={2024}
}
```

## License

This is a derivative of the upstream model and is distributed under the same terms
(`apache-2.0`); see `LICENSE`.
Modifications (conversion to the ExecuTorch format and quantization) by Meta Platforms, Inc.