Add ExecuTorch WebGPU program for qwen2.5-0.5b-instruct

README.md

@@ -5,6 +5,7 @@ base_model:
 base_model_relation: quantized
 library_name: executorch
 pipeline_tag: text-generation
+inference: false
 tags:
 - executorch
 - webgpu
@@ -15,14 +16,42 @@ tags:
 
 This repository contains an [ExecuTorch](https://github.com/pytorch/executorch) conversion of
 [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct) for the ExecuTorch **WebGPU** backend, so the model runs entirely in a web browser on
-the visitor's GPU. It powers the corresponding demo in the
-ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
+the visitor's GPU. It powers the corresponding demo in the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)).
 
 - **Base model:** [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct)
 - **Format:** ExecuTorch `.pte` programs delegated to the WebGPU backend
 - **Quantization:** 4-bit groupwise weights (q4gsw), fp16 KV cache
 - **Total size:** 296.5 MB
 
+## About the original model
+
+The 0.5B instruction-tuned model of the Qwen2.5 family.
+
+- **Developed by:** Qwen Team, Alibaba Cloud
+- **Paper:** [Qwen2.5 Technical Report](https://arxiv.org/abs/2412.15115)
+- 0.49B parameters, 24 layers
+- Context length 32,768 tokens; generation up to 8,192 tokens
+- Supports over 29 languages
+
+For training data, evaluation results and full details, see the original model card: [Qwen/Qwen2.5-0.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct).
+
+## Intended use and limitations
+
+Chat and instruction following in a small footprint.
+
+Small language models can state false information confidently, repeat themselves, and reflect biases in their training data. Check important outputs.
+
+## What changed in this conversion
+
+- Weights quantized to 4-bit groupwise (q4gsw); the KV cache is fp16
+- Context capped at 8,192 tokens (upstream: 32,768)
+
+## Inputs and outputs
+
+- `forward(tokens int64 [1, 1..512], input_pos int64 [1]) -> logits float32 [1, 151936]` (last position only)
+- The KV cache lives inside the program; call it with up to 512 new tokens at a time, then one token per step
+- Tokenize with the files in `tokenizer/` (the upstream Qwen2.5 tokenizer) and the ChatML template
+
 ## Files
 
 | File | Size | SHA-256 |
@@ -39,10 +68,13 @@ Full checksums are in `manifest.json`. The playground verifies them after downlo
 
 ## Running it
 
-These programs need the ExecuTorch WebGPU runtime compiled to WebAssembly
-([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)). The easiest way to
-try the model is the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)),
-which downloads these files once, caches them in the browser, and runs everything locally.
+These programs were exported with the ExecuTorch WebGPU backend
+([backends/webgpu](https://github.com/pytorch/executorch/tree/main/backends/webgpu)) and validated with the
+WebAssembly runtime build used by the ExecuTorch WebGPU Playground (launching soon on [executorch.ai](https://executorch.ai)). That runtime build is not published yet; until it is,
+the playground is the supported way to run them. It downloads these files once, caches them in the browser,
+and runs everything locally.
+
+Requirements: a browser with WebGPU enabled. The language models use the `shader-f16` GPU feature when available and fall back to fp32 math without it.
 
 ## Provenance
 
@@ -51,6 +83,20 @@ which downloads these files once, caches them in the browser, and runs everythin
 - **Reproduce:** `python recipes/qwen2.5-0.5b-instruct.py --out out/qwen2.5-0.5b-instruct` (recipe in `export/`)
 - Re-running the recipe produces byte-identical files.
 
+## Citation
+
+If you use this model, please cite the original work:
+
+```bibtex
+@misc{qwen2.5,
+  title={Qwen2.5: A Party of Foundation Models},
+  url={https://qwenlm.github.io/blog/qwen2.5/},
+  author={Qwen Team},
+  month={September},
+  year={2024}
+}
+```
+
 ## License
 
 This is a derivative of the upstream model and is distributed under the same terms