litert-community/Qwen2.5-1.5B-Instruct

This model provides a few variants of Qwen/Qwen2.5-1.5B-Instruct that are ready for deployment on Android using the LiteRT (fka TFLite) stack, MediaPipe LLM Inference API (in maintenance-only mode) and LiteRT-LM.

Use the models

Colab

Disclaimer: The target deployment surface for the LiteRT models is Android/iOS/Web and the stack has been optimized for performance on these targets. Trying out the system in Colab is an easier way to familiarize yourself with the LiteRT stack, with the caveat that the performance (memory and latency) on Colab could be much worse than on a local device.

Android

Edge Gallery App

• Download or build the app from GitHub.

• Install the app from Google Play.

• Follow the instructions in the app.

LiteRT-LM in your own app

Add com.google.ai.edge.litertlm:litertlm-android to your app as shown in the LiteRT-LM Kotlin guide, and load Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv4096.litertlm from this repository. This file needs LiteRT-LM 0.18.0 or newer.

iOS

• See the LLM Inference guide for iOS (MediaPipe, in maintenance-only mode). The iOS sample app this section linked was removed from mediapipe-samples.

Performance

Android

Note that all benchmark stats are from a Samsung S25 Ultra and multiple prefill signatures enabled.

Backend	Quantization scheme	Context length	Prefill (tokens/sec)	Decode (tokens/sec)	Time-to-first-token (sec)	Model size (MB)	Peak RSS Memory (MB)	GPU Memory (RSS in MB)
CPU	fp32 (baseline)	1280	49.50	10 tk/s	21.25 s	6182 MB	6254 MB	N/A	🔗
CPU	dynamic_int8	1280	297.58	34.25 tk/s	3.71 s	1598 MB	1997 MB	N/A	🔗
CPU	dynamic_int8	4096	162.72 tk/s	26.06 tk/s	6.57 s	1598 MB	2216 MB	N/A	🔗
GPU	dynamic_int8	1280	1667.75 tk/s	30.88 tk/s	3.63 s	1598 MB	1846 MB	1505 MB	🔗
GPU	dynamic_int8	4096	933.45 tk/s	27.30 tk/s	4.77 s	1598 MB	1869 MB	1505 MB	🔗

• For the list of supported quantization schemes see supported-schemes. For these models, we are using prefill signature lengths of 32, 128, 512 and 1280.
• Model Size: measured by the size of the .tflite flatbuffer (serialization format for LiteRT models)
• Memory: indicator of peak RAM usage
• The inference on CPU is accelerated via the LiteRT XNNPACK delegate with 4 threads
• Benchmark is run with cache enabled and initialized. During the first run, the time to first token may differ.

Known runtime issues

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders the Qwen2.5-1.5B-Instruct_multi-prefill-seq_q8_ekv4096.litertlm file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it. Qwen2.5-1.5B-Instruct_multi-prefill-seq_f32_ekv4096.litertlm carries no Jinja chat template and is not affected: the template the runtime builds for it only loops over the messages.