Measured on device on an earlier file (edge-compat): Galaxy S26 · LiteRT-LM 0.16.0 · GPU · decode 85.4 tok/s · prefill 1456 tok/s · TTFT 150 ms · all 1063 ops delegated (2026-08-24); Galaxy S26 · LiteRT-LM 0.16.0 · CPU · decode 81.7 tok/s · prefill 359 tok/s · TTFT 680 ms (2026-09-05). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/internvl3-1b/CARD.md

InternVL3-1B — LiteRT-LM (on-device Vision-Language Model)

OpenGVLab/InternVL3-1B converted to the LiteRT-LM (.litertlm) format for on-device image+text inference with Google's LiteRT-LM runtime. On 2026-11-15 InternVL3-1B.litertlm was replaced with a file whose prefill/decode graph is rewritten for the GPU, with every weight unchanged. Outside the dated entries under Conversion notes, "this file" means that file and "the previous file" the one it replaced. On a Galaxy S26 GPU this file decodes 111.9 tokens/s and prefills 1,024 tokens at 1,941 tokens/s (LiteRT-LM v0.19.0).

InternVL3-1B is the smallest InternVL3 vision-language model: an InternViT vision encoder + pixel-shuffle + MLP projector feeding a Qwen2.5-0.5B language decoder. At 739 MB it is a tiny, fast on-device VLM — give it an image and a question, get a grounded answer, fully offline. (See InternVL3-2B-LiteRT for the larger model.)

File InternVL3-1B.litertlm (~739 MB)
Vision InternViT-300M encoder (attention written with 4D tensors only) + pixel-shuffle + MLP projector, int8 — single 448×448 image → 256 image tokens
Decoder Qwen2.5-0.5B (896-dim, 24 layers), int4 weights (symmetric, blockwise-32 + OCTAV); input embedding INT8 (externalized)
Context (KV cache) 2048
Image input resized to 448×448 (ImageNet normalization baked into the vision encoder)
Base model OpenGVLab/InternVL3-1B

How to use

1. Install the runtime

pip install litert-lm

2. Run it in one command — this downloads the file, encodes your image and answers:

litert-lm run --from-huggingface-repo litert-community/InternVL3-1B InternVL3-1B.litertlm \
  --attachment photo.jpg \
  --prompt "Describe this picture in one sentence." --top-k 1

On the COCO photo 000000039769.jpg (640 × 480, two tabby cats on a pink blanket), this file gives this answer when --backend cpu --vision-backend cpu --cpu-thread-count 1 --cache no --max-num-tokens 2048 is added to the command:

The image shows two cats lying on a pink bed with two remote controls placed nearby.

This answer came from litert-lm 0.19.0, 0.19.1 and 0.19.2 on a Mac, with both files (2026-11-14). The command was also run with only its own flags (--attachment, --prompt, --top-k 1; litert-lm 0.19.2 on a Mac, 2026-11-15). Each file was given by its local path: this file as a copy with the same sha256 as the file in this repo, the previous file as downloaded from this repo. The wording then changed from run to run: two runs of this file answered "The image shows two cats lying on a pink bedspread." and "The image shows two cats lying on a pink bedspread, with a remote control visible nearby."; one run of the previous file answered the second sentence. Why the wording changes between runs with --top-k 1 was not checked.

The command above leaves --vision-backend out. This file, like the previous one, sets no default backend, so when --backend and --vision-backend are left out, litert-lm 0.19.0 to 0.19.2 run the decoder and the vision encoder on the CPU (unless a backend is set for the model in the CLI's config). With --backend gpu added and --vision-backend still left out, this file answered the picture in one run (on a Mac with litert-lm 0.19.2: "The image shows two cats lying on a pink bedspread, with a remote control placed nearby."); the run also printed one error line, TopK requires src tensor C dimension to be divisible by 4., and still answered. With --backend gpu --vision-backend gpu, neither this file nor the previous file answers a picture on a Mac (litert-lm 0.19.0 to 0.19.2; the reply is An error occurred); with litert-lm 0.19.2 the error log names two op types of the vision encoder that the GPU delegate does not take, BROADCAST_TO and GATHER_ND, and the runtime does not run the encoder split between the GPU and the CPU.

If run stops right after the download with no error message from the CLI (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the downloaded file may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/litert-community/InternVL3-1B/ and uses it again on later runs without checking its size. Delete InternVL3-1B.litertlm in that directory and run the command again.

If you downloaded InternVL3-1B.litertlm with --from-huggingface-repo before it was replaced on 2026-11-15 (05:12 UTC), delete ~/.litert-lm/cache/huggingface/litert-community/InternVL3-1B/InternVL3-1B.litertlm, then run the command again. Without deleting it, the litert-lm CLI (0.14.0 to 0.19.2) keeps running the previous file: it uses a file it finds there without checking it against this repo.

Drop --prompt for an interactive chat, and pass --attachment more than once for several images (but see the Known limitation below). litert-lm serve exposes this model as a local OpenAI-compatible API. None of these was tried with this file, and on desktop this file was run on macOS only.

Performance

This file's speed was measured on two phones (the Galaxy S26 and Pixel 11 Pro sections below), not on a Mac. Measured on an earlier file: litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine. These figures cover the text path; the vision encoder runs once per image and is not included.

Device Backend Prefill (256) Decode TTFT
Apple M4 Max (macOS) CPU 454 tok/s 94.0 tok/s 0.62 s

Every desktop backend listed above was checked by actually generating a caption on it, not just by reading the benchmark tool's output.

The macOS GPU backend was not usable for an earlier file of this repo on litert-lm 0.15.0. Asked to describe an image with --backend gpu, it returns a run of ! characters instead of a caption. litert-lm benchmark still reports GPU numbers for it — 5,583 tok/s prefill and 342 tok/s decode — but they measure a path that produces no usable text, so no GPU row is quoted here. With that version, use the CPU backend on the desktop. This is a desktop-runtime observation and says nothing about the iPhone or Android GPU paths described elsewhere on this card. With litert-lm 0.19.0 to 0.19.2, this file and the previous one answer a text question on the Mac GPU with the same reply, byte for byte. No Mac speed was measured with these versions.

Accuracy note

This paragraph was written for an earlier file and was not re-checked with this one. Output is coherent and image-grounded (CPU-verified; the vision tower converts faithfully to the reference, float CPU output corr ≈ 1.0). It was converted with the same recipe as the larger InternVL3-2B build; its on-device behavior was not compared with that build. With its 0.5B decoder it is the smallest of the family.

Galaxy S26 — GPU and CPU

Each run prefills 1,024 tokens and generates 256 tokens, three times (three iterations): the runtime's benchmark mode puts the prompt file's one sentence ("Explain what on-device AI means in simple terms.") in the chat template as a user message and pads the tokens to 1,024 with token id 0. Iteration 1 is the first pass after the engine starts. The GPU row is the median of iterations 2–3 over two runs, so it leaves iteration 1 out (this file's iteration 1 on the GPU: prefill 1,899–1,920 tok/s, decode 103.6–104.4 tok/s). On the CPU the median prefill and decode speeds drop from each iteration to the next, so each iteration has its own row: the median of that iteration over two runs.

Backend Prefill, 1,024 tokens Decode Time to first token Peak memory
GPU (LiteRT GPU, every node of the prefill and decode graphs), iterations 2–3 1,941 tok/s 111.9 tok/s 0.54 s 0.45 GB + 0.72 GB GPU driver
CPU (XNNPACK for most nodes), iteration 1 655 tok/s 116.8 tok/s 1.57 s 0.88 GB
CPU (XNNPACK for most nodes), iteration 2 625 tok/s 94.4 tok/s 1.65 s 0.88 GB
CPU (XNNPACK for most nodes), iteration 3 443 tok/s 83.0 tok/s 2.33 s 0.88 GB

Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16), litert_lm_advanced_main built from the LiteRT-LM v0.19.0 source (commit 771b06e1; the v0.19.0 tag is on the next commit, which changes only Package.swift and README.md), --benchmark with 1,024 prefill tokens and 256 decode tokens, context 2,048, 2026-11-15. Each run is a new process. The rows cover the text path. On the CPU the runtime used its default of 4 threads. The GPU runs started and ended at Android's thermal status 0 (none); one run of each file ended with the GPU driver's thermal limit at level 6 (this file) or 5 (the previous file), each level lowering the GPU's highest allowed clock by one step (0 = no limit), and the other two ended at 0. Of the four counted CPU runs, one run of each file started at thermal status 1 (light) and the other at 0; none reached status 2.

In each of the four counted CPU runs (two per file, after one warm-up run that is not counted), the phone lowers the clock caps of its two CPU clock domains from 2–5 s in, to 2.00 / 1.98 GHz by 11–14 s in (2.11 / 2.11 GHz in one run), and the runs took 14–15 s. The caps come down through all three iterations, so each iteration runs at lower caps than the one before.

Peak memory is per run, so the three CPU rows share one value. It is the process's VmHWM: on the CPU 0.88 GB in each of this file's two counted runs and in the warm-up run, and 0.75 GB in each of the previous file's two. VmHWM does not count memory the GPU driver holds, so the GPU row adds the rise of the phone's GPU allocation counter (kgsl_page_alloc) during each of this file's GPU runs (0.72 GB; 0.55 GB in each of the previous file's). In the CPU runs that counter did not rise.

With this file, the vision encoder on the CPU (--vision_backend=cpu) and the decoder on the GPU, this phone answered "Describe this picture in one sentence." about the photo under How to use with "Two cats are lying on a pink blanket, seemingly relaxed." (one run). The vision encoder was not run on this phone's GPU.

On the GPU this file prefills about 11 % faster than the previous file, and it decoded faster in both pairs of runs, by 6 % and 20 % (the 6 % is less than the 6.2 % between this file's own two runs). The two changes, the graph form and the 1,024-token signature, were not measured apart. Four runs went previous, this, this, previous (each run's mean of iterations 2–3): prefill 1,753, 1,939, 1,941 and 1,751 tok/s, decode 99.7, 105.8, 112.3 and 93.5 tok/s; from the previous file to this one, prefill ×1.106 and decode ×1.061 in the first two runs, prefill ×1.108 and decode ×1.202 in the last two. Two runs of the same file differ by 0.1 % in prefill and by 6.2 % (this file) and 6.7 % (the previous file) in decode. On the CPU, this file's medians were below the previous file's in every iteration, by 1.1 %, 1.1 % and 2.6 % in prefill and by 2.2 %, 0.7 % and 1.2 % in decode (iterations 1, 2 and 3); two counted runs of the same file differed by up to 1.4 %, 3.6 % and 10.5 % in iterations 1, 2 and 3. The runs behind each number, and the S26 section this card carried for an earlier file: MEASUREMENTS.md.

On the Galaxy S26 the GPU prefills 3.0× as fast as the CPU's iteration 1 and 3.1× and 4.4× as fast as its iterations 2 and 3. In decode the CPU's iteration 1 (116.8 tok/s) is faster: 4 % faster than the GPU row and 12–13 % faster than the GPU's own iteration 1. The GPU decodes 1.2× and 1.3× as fast as the CPU's iterations 2 and 3. This repo's litertlm_manifest.json recommends gpu for the Android device class it calls flagship, for its faster prefill and its faster decode from iteration 2 on. That recommendation rests on the Galaxy S26 alone.

GPU wiring, including the Gallery import toggle: GPU guide.

Pixel 11 Pro — GPU

On a Pixel 11 Pro GPU (the same build and workload, 2026-11-14) this file decodes 26.7 tok/s, 10 % faster than the previous file, and prefills 1,391 tok/s, 5 % faster (the median of iterations 2–3 over two runs per file). With the vision encoder on the CPU (--vision_backend=cpu) and the decoder on the GPU, both files answered the photo question with "Two cats are lying on a pink bedspread." (one run each). The Pixel 11 Pro CPU was not measured, so whether the GPU or the CPU is faster on that phone is not known. The runs: MEASUREMENTS.md.

⚠️ Known limitation — one image per conversation on the GPU (Metal) backend

Single-image VQA — the primary use case — worked in each setup tried with the decoder on the GPU and the vision encoder not on the GPU (one run each: on a Mac with litert-lm 0.19.2, --backend gpu with --vision-backend left out, a run that printed none of the GPU-delegate errors that --vision-backend gpu prints; on a Galaxy S26 and a Pixel 11 Pro, --vision_backend=cpu; see How to use and the two phone sections). But on the GPU (Metal) backend of Apple devices, a second image in the same conversation truncates the answer — ask about one image per chat (start a new conversation for a different image). On the CPU backend, a second image in the same conversation was answered, and the same truncation on the GPU was seen with other fast_vlm models. For multi-image, run on the CPU backend. The truncation, the CPU result and the other models' runs were all seen with an earlier file; none was tried with this file or on Android.

Run on iPhone / macOS

Use swift-litert-lm, a community Swift package over the LiteRT-LM runtime, and its LiteRTDemo sample. Load InternVL3-1B.litertlm with the image (vision) tower enabled (modalities Modality.textImage / [.vision] — a vision-only file, no audio tower), attach a photo, and ask a question.

These steps were written for an earlier file and were not run with this one. With the litert-lm CLI on a Mac, this file does not answer a picture with --vision-backend gpu (see How to use); whether swift-litert-lm puts the vision encoder on the GPU was not checked.

Run on Android — Google AI Edge Gallery

Update (July 2026): Google AI Edge Gallery v1.0.16+ can import litert-lm models directly from Hugging Face inside the app (tap +) — no computer or adb needed. The manual steps below are only required on older builds or for sideloading a local file.

Run this model with image input in the official Google AI Edge Gallery app — no custom app needed (the file carries the tokenizer, chat template, and image preprocessing config):

  1. Push the file onto the phone (or download it there directly from this repo): adb push InternVL3-1B.litertlm /sdcard/Download/
  2. Open the Gallery app, tap the + icon (bottom-right) and pick InternVL3-1B.litertlm in the file picker.
  3. In the Import Model dialog, check "Support image" (required for image input), pick GPU or CPU, then tap Import.
  4. Open the Ask Image task, choose the imported model, attach a photo, and ask.

Tip: the one-image-per-conversation limit above was seen on the GPU (Metal) backend of Apple devices and was not tried on Android; pick CPU if you want multiple images in one chat.

These steps were written for an earlier file and were not repeated with this one, and which backend Gallery gives the vision encoder was not checked. On a Galaxy S26 and a Pixel 11 Pro (LiteRT-LM v0.19.0), litert_lm_advanced_main answered the picture with the vision encoder on the CPU (see the two phone sections); the vision encoder was not run on a phone GPU.

Run on desktop (LiteRT-LM CLI)

The same .litertlm file runs with the official LiteRT-LM CLI, which is published for macOS, Linux and Windows — including as a local OpenAI-compatible API server (these commands were written for an earlier file and were not run with this one; on desktop this file was run on macOS only):

pip install litert-lm
litert-lm import --from-huggingface-repo litert-community/InternVL3-1B InternVL3-1B.litertlm internvl3-1b
litert-lm run internvl3-1b     # interactive chat in the terminal
litert-lm serve           # local OpenAI-compatible API server

If the run command above stops with no error message from the CLI (a bus error; exit status 138 on a Mac) or with an error log that contains too large for file size, the file that import downloaded may be incomplete. The litert-lm CLI (0.14.0 to 0.19.2, and litert-lm-nightly 0.20.0.dev20261112) can keep an incomplete download in ~/.litert-lm/cache/huggingface/litert-community/InternVL3-1B/ and imports it without checking its size. Delete InternVL3-1B.litertlm in that directory and run the import command again: it downloads a new copy and replaces the imported one.

If you imported InternVL3-1B.litertlm with --from-huggingface-repo and your first download of it (by run or by import) was before it was replaced on 2026-11-15 (05:12 UTC), the imported copy is the previous file: delete ~/.litert-lm/cache/huggingface/litert-community/InternVL3-1B/InternVL3-1B.litertlm (unless you already deleted it for the run --from-huggingface-repo command under How to use), then run the import command again. Without deleting it, import takes the previous file again: the litert-lm CLI (0.14.0 to 0.19.2) uses a file it finds there without checking it against this repo.

Conversion notes

  • LiteRT-LM fast_vlm file: VISION_ENCODER ([1,448,448,3]→[1,256,4096]) + VISION_ADAPTER ([1,256,4096]→[1,256,896]) + single-token EMBEDDER + PREFILL_DECODE (embeddings-input).
  • The vision encoder bakes ImageNet normalization + the NCHW transpose into the graph, and the InternViT attention is rewritten 4D-clean (qkv split before the head reshape — no GPU-rejected 5D reshape), numerically identical (corr ≈ 1.0).
  • Decoder exported with externalized embedder; InternVL's dynamic-NTK rope_scaling is stripped to base RoPE (valid since the export cache ≤ the base context window).

Each dated entry below was written on its date: in it, "this file" is the file published that day, "the previous file" is the one it replaced, and "this card" is the card as it was then.

2026-08-28 — start_token fix (weights unchanged)

The file's LlmMetadata start_token held the literal string "None". This tokenizer has no BOS, and LiteRT-LM resolved that string to a real vocabulary token — so every prompt began with the word None, which the model was never trained on. The start token has been removed.

Metadata-only change: every section of the file except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file — only the file's own sha256 differs. What changed is the input: the token stream the model reads for a given conversation can differ from the previous file's, and it now matches this model's own reference chat stream. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file in either direction. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-28, re-download.

2026-08-29 — default system prompt restored (weights unchanged)

The upstream chat template emits a default system turn whenever the caller sends no system message — for this model: 你是书生·万象,英文名是InternVL,是由上海人工智能实验室、清华大学及多家合作单位联合开发的多模态大语言模型。. The converter's template test renders the template with a system message already present, so that block was never seen and never reached the file: with no system message the model was running without the default system turn it was tuned with. The chat template in InternVL3-1B.litertlm now emits the block exactly once when no system message is given. In InternVL3-1B.litertlm, the block is not emitted when you pass a system message. The restored block adds 37 prefill tokens to a conversation that sends no system message, so time-to-first-token grows by that much; per-token speed is unchanged.

Metadata-only change: every section of the file except the LlmMetadata block is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the tokenizer are unchanged and the speed and memory numbers on this card still describe exactly this file per token — only the file's own sha256 differs. What changed is the input: with no system message, the prompt now renders byte-identical to the upstream chat template's output, verified on the LiteRT-LM runtime. A system message you pass yourself renders as before. Greedy decoding can turn on a single token, so an individual answer can differ from the previous file. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file and have not been re-measured on this one. If you downloaded before 2026-08-29, re-download.

2026-08-30 — tokenizer section replaced (weights unchanged)

The tokenizer in InternVL3-1B.litertlm was a SentencePiece conversion of the model's BPE tokenizer, and the conversion lost the byte-level semantics: a standalone accented letter or symbol (é, ñ, ü, °, ·, …) was encoded to the id of a single-byte token instead of the token the upstream tokenizer uses, and 36 of the 223 standalone characters U+00A1–U+017F in the checks (¡, Ā to ġ, and Ų) became the token the conversion had reused as UNK — the end-of-text token for this vocabulary. InternVL3-1B.litertlm now embeds the upstream tokenizer.json (the same HF tokenizer path most files in this collection use).

Tokenizer-only change: every section of the file except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: one user message ("What is the capital of France?") as the chat template renders it, 7 test strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer. A conversation without a system message of your own reaches the model differently from before, because the · in the default system prompt above now gets the upstream tokenizer's token; text with accented letters, symbols or non-Latin scripts and a short Python snippet did too in the checks. So answers can change: of the four ASCII-only test questions, one ("Name three colors of the rainbow.") answered byte-identically to the previous file and three did not. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device check has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.

2026-08-31 — chat template fix: history turns with role 'assistant' (metadata-only, weights unchanged)

The chat template in InternVL3-1B.litertlm matched history turns only on role 'model', so a conversation created through the Conversation API with the standard role 'assistant' (for example create_conversation(messages=[...]) restoring a chat) silently dropped every assistant turn from the rendered prompt — the model saw consecutive user turns with its own replies gone. The template's role conditions now also accept 'assistant'. The engine's own send_message flow was never affected.

Template-only change: every section of the file except the metadata is byte-identical to the previous file (verified by per-section sha256), and single-turn renders, token ids, prefill counts and greedy answers are unchanged (verified on the LiteRT-LM runtime), so the numbers on this card still describe this file — only the file's own sha256 differs. Re-download only matters if you inject conversation history with role 'assistant'. Details: google-ai-edge/LiteRT-LM#3417 (https://github.com/google-ai-edge/LiteRT-LM/issues/3417).

2026-11-15 — GPU graph and a 1,024-token prefill (weights unchanged)

The prefill/decode graph now has the GPU form litert-torch writes with --apply_gpu_composites, plus a 1,024-token prefill signature, prefill_1024: a copy of prefill_512 at a length of 1,024 tokens that shares its weight buffers. Every weight, and every section outside that graph, is byte-identical to the previous file. The whole file went from sha256 18650b2d… (737,378,304 B) to 6ab80286… (738,902,016 B). On a Mac the two files give the same replies byte for byte: 4 prompts and a picture on the CPU (litert-lm 0.19.2, 1 thread, greedy decoding with top-k 1), and one prompt and the picture on the CPU and the prompt on the GPU with litert-lm 0.19.0 to 0.19.2. 8 short questions (each ends with "Answer briefly."): 7 of 8 with both files on the Mac CPU and GPU (both answer "pink" to the rhyme question, "Roses are red, violets are ___", where "blue" is expected); on a Pixel 11 Pro GPU 6 of 8 with the previous file, whose step-by-step answer to "What is 17 + 25? Answer briefly." reached the 256-token cap before it reached 42, and 7 of 8 with this file; 7 of 8 with this file on a Galaxy S26 GPU (the rhyme question; the previous file was not asked there). Galaxy S26 GPU (LiteRT-LM v0.19.0, four runs in the order previous, this, this, previous; each run's mean of iterations 2–3): prefill 1,753, 1,939, 1,941 and 1,751 tokens/s, decode 99.7, 105.8, 112.3 and 93.5 tokens/s, so from the previous file to this one prefill ×1.106 and decode ×1.061 in the first pair, prefill ×1.108 and decode ×1.202 in the second; two runs of the same file differ by 0.1 % in prefill and by 6.2–6.7 % in decode. On the Galaxy S26 CPU (four runs in the same order, after one warm-up run), this file's medians were 0.7–2.6 % below the previous file's in each iteration, while two runs of the same file differed by up to 10.5 %; its peak memory (VmHWM) went from 0.75 to 0.88 GB on the CPU, and the rise of the GPU driver's allocation during a GPU run from 0.55 to 0.72 GB. Procedure: REPRODUCE.md. Measured on this file: the last sentence of the paragraph under the title, the file size, the example answer under How to use and the two paragraphs after it, the last two sentences of the macOS GPU paragraph, the Galaxy S26 and Pixel 11 Pro sections, the first sentence of the Known limitation section, the last paragraph of the iPhone / macOS section and of the Gallery section, and this entry. The Known runtime issues below come from earlier runtimes or other files (each says which) and were not re-checked with this file. The rest of this card was written for earlier files; the weights, tokenizer and chat template it describes are the same in this file.

Known runtime issues

External weights (since 2026-11-09). In InternVL3-1B.litertlm, the weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights) instead of inside the model graph. On a Pixel 8a, the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on a Qwen3-0.6B file with external weights (LiteRT-LM#3986). The last file with the weights inside the graph stays at revision 875247a.

From the second turn of a conversation on, LiteRT-LM puts the previous answer's stop token into the KV cache with a zero embedding, because this file holds the embedding table in its own section and the decoder takes embeddings, so later answers in a conversation can differ from the answer the model gives to the same messages in one prompt (read in the source of LiteRT-LM 0.16.0 to 0.18.1; measured with 0.18.1 on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file). One question per conversation avoids it, as does a new conversation that starts with the earlier messages and the new question.

From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads messages[0] without checking that the list is non-empty, so the render fails with Failed to apply template: undefined value. This happens in two places: when the preface is prefilled at creation (prefillPrefaceOnInit = true in Kotlin, prefillPrefaceOnInit: true on the web, SetPrefillPrefaceOnInit(true) in C++; the default is false), where creating the conversation fails with that error, and in the preface render (renderPrefaceIntoString() in Kotlin and Swift, RenderPrefaceIntoString() in C++, litert_lm_conversation_render_preface_to_string in the C API). The same happens with extra context (such as enable_thinking) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.

On LiteRT-LM 0.18.1 and 0.19.1, once a message fails because it does not fit in the context window, every later message in that conversation fails as well (the next one with Task failed with state: 7). Start a new conversation after such an error. This was measured with other models' files, not with this one.

Downloads last month
450
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for litert-community/InternVL3-1B

Quantized
(6)
this model

Collection including litert-community/InternVL3-1B