Instructions to use litert-community/Qwen2.5-Coder-1.5B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT-LM
How to use litert-community/Qwen2.5-Coder-1.5B-Instruct with LiteRT-LM:
# LiteRT-LM runs on various platforms (Android, iOS, Windows, Linux, macOS, IoT, Web/WASM) # and supports many APIs (C++, Python, Kotlin, Swift, JavaScript, Flutter). # For platform-specific integration guides, please refer to the official developer website: # https://ai.google.dev/edge/litert-lm # To try LiteRT-LM, the easiest way is to use our CLI tool. # 1. Install the LiteRT-LM CLI tool: pip install -U litert-lm # 2. Download and run this model locally: # See: https://ai.google.dev/edge/litert-lm/cli # A single .litertlm file in the repo is picked automatically; otherwise the CLI asks which one to run # (or pass its name right after the repo id). litert-lm run \ --from-huggingface-repo=litert-community/Qwen2.5-Coder-1.5B-Instruct \ --prompt="Write me a poem"
- LiteRT
How to use litert-community/Qwen2.5-Coder-1.5B-Instruct with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
card: Galaxy S26 same-handset GPU/CPU speed pair (2026-09-05 CPU control) replaces 'no speed rows'; Android recommendation is cpu, as in litertlm_manifest.json
Browse files
README.md
CHANGED
|
@@ -62,7 +62,7 @@ Honest notes:
|
|
| 62 |
- **A Mac's GPU-vs-CPU ratio does not transfer to a phone.** Here the GPU decodes 2.9× the CPU; on a Pixel 8a a 3B model's GPU and CPU decode land within 1% of each other, because a phone's CPU and GPU share the same LPDDR. On phone hardware the GPU's win is prefill and time-to-first-token.
|
| 63 |
- The embedding table is stored in its own bundle section. That is what keeps this file at 1.12 GB — with the table quantized inline it duplicates per prefill signature and the same weights come out at 2.53 GB.
|
| 64 |
|
| 65 |
-
## Galaxy S26 — GPU
|
| 66 |
|
| 67 |
The published bundle runs on the Android GPU backend and generates.
|
| 68 |
|
|
@@ -74,7 +74,14 @@ Measured on a **Samsung Galaxy S26** (SM-S942Q / SM8850, Android 16) with `liter
|
|
| 74 |
|
| 75 |
The op counts above are the `LiteRT GPU` partitions. XNNPACK additionally takes 1 of the 4 nodes in `decode_embedder` and 1 of the 4 nodes in `prefill_embedder_1024`; the runtime accepts that split.
|
| 76 |
|
| 77 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 78 |
|
| 79 |
GPU wiring, including the Gallery import toggle: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).
|
| 80 |
|
|
|
|
| 62 |
- **A Mac's GPU-vs-CPU ratio does not transfer to a phone.** Here the GPU decodes 2.9× the CPU; on a Pixel 8a a 3B model's GPU and CPU decode land within 1% of each other, because a phone's CPU and GPU share the same LPDDR. On phone hardware the GPU's win is prefill and time-to-first-token.
|
| 63 |
- The embedding table is stored in its own bundle section. That is what keeps this file at 1.12 GB — with the table quantized inline it duplicates per prefill signature and the same weights come out at 2.53 GB.
|
| 64 |
|
| 65 |
+
## Galaxy S26 — GPU and CPU
|
| 66 |
|
| 67 |
The published bundle runs on the Android GPU backend and generates.
|
| 68 |
|
|
|
|
| 74 |
|
| 75 |
The op counts above are the `LiteRT GPU` partitions. XNNPACK additionally takes 1 of the 4 nodes in `decode_embedder` and 1 of the 4 nodes in `prefill_embedder_1024`; the runtime accepts that split.
|
| 76 |
|
| 77 |
+
Speed on the same handset with one binary and one 205-token prompt file; only `--backend` changes (`litert_lm_advanced_main` from litert-lm 0.16.0, `--benchmark`, `--sampler_backend=cpu`). The GPU row is the gate run of 2026-08-24 (one run); the CPU rows were taken 2026-09-05, two cold runs (compile and weight caches deleted between runs) with the phone cooled below 42 °C before each run.
|
| 78 |
+
|
| 79 |
+
| Backend | Prefill | Decode | TTFT | Engine init | Peak (`VmHWM`) |
|
| 80 |
+
|---|---|---|---|---|---:|
|
| 81 |
+
| GPU (`LiteRT GPU`, full delegation) | 234.9 tok/s | 8.6 tok/s | 1.07 s | 8.3 s | 885 MB |
|
| 82 |
+
| CPU (XNNPACK) | 57–106 tok/s | **27.0–29.7 tok/s** | 2.14–3.94 s | 2.7 s | 1566 MB |
|
| 83 |
+
|
| 84 |
+
The CPU decodes 3.3× faster; the GPU wins prefill and peaks lower. A code answer is mostly decode (the bench replies ran 360–409 tokens), so this repo's `litertlm_manifest.json` recommends `cpu` for flagship-class Android on that pair. Pick `gpu` for long prompts with short answers.
|
| 85 |
|
| 86 |
GPU wiring, including the Gallery import toggle: [GPU guide](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/android-gpu.md).
|
| 87 |
|