mlboydaisuke commited on
Commit
fcd2700
·
verified ·
1 Parent(s): 3835eff

Android GPU note: the measured crash instead of a version floor

Browse files

README.md and litertlm_manifest.json: the note on Android GPU builds now says what was measured (on a Pixel 8a, the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes on a Qwen3-0.6B file with external weights) instead of a LiteRT-LM version floor.

Files changed (2) hide show
  1. README.md +1 -1
  2. litertlm_manifest.json +1 -1
README.md CHANGED
@@ -188,7 +188,7 @@ Apart from the parts that name the file of 2026-11-17, this card's measurements
188
 
189
  ## Known runtime issues
190
 
191
- **External weights (since 2026-11-09).** In `VibeVoice-ASR-BitNet.litertlm`, the weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights) instead of inside the model graph. On Android, the GPU path needs LiteRT-LM 0.19.0 or later: the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on such a file ([LiteRT-LM#3986](https://github.com/google-ai-edge/LiteRT-LM/issues/3986)). The earlier file with the weights inside the graph stays at revision [`193b2d5`](https://huggingface.co/litert-community/VibeVoice-ASR-BitNet/tree/193b2d579a786206286ac36e74b6ea513562bdf5).
192
 
193
  From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads `messages[0]` without checking that the list is non-empty, so the render fails with `Failed to apply template: undefined value`. This happens in two places: when the preface is prefilled at creation (`prefillPrefaceOnInit = true` in Kotlin, `prefillPrefaceOnInit: true` on the web, `SetPrefillPrefaceOnInit(true)` in C++; the default is false), where creating the conversation fails with that error, and in the preface render (`renderPrefaceIntoString()` in Kotlin and Swift, `RenderPrefaceIntoString()` in C++, `litert_lm_conversation_render_preface_to_string` in the C API). The same happens with extra context (such as `enable_thinking`) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.
194
 
 
188
 
189
  ## Known runtime issues
190
 
191
+ **External weights (since 2026-11-09).** In `VibeVoice-ASR-BitNet.litertlm`, the weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights) instead of inside the model graph. On a Pixel 8a, the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on a Qwen3-0.6B file with external weights ([LiteRT-LM#3986](https://github.com/google-ai-edge/LiteRT-LM/issues/3986)). The earlier file with the weights inside the graph stays at revision [`193b2d5`](https://huggingface.co/litert-community/VibeVoice-ASR-BitNet/tree/193b2d579a786206286ac36e74b6ea513562bdf5).
192
 
193
  From a conversation preface (the system message, initial messages, tools and extra context a conversation is created with) that has tools and no messages, LiteRT-LM renders this file's chat template with an empty message list, and the template reads `messages[0]` without checking that the list is non-empty, so the render fails with `Failed to apply template: undefined value`. This happens in two places: when the preface is prefilled at creation (`prefillPrefaceOnInit = true` in Kotlin, `prefillPrefaceOnInit: true` on the web, `SetPrefillPrefaceOnInit(true)` in C++; the default is false), where creating the conversation fails with that error, and in the preface render (`renderPrefaceIntoString()` in Kotlin and Swift, `RenderPrefaceIntoString()` in C++, `litert_lm_conversation_render_preface_to_string` in the C API). The same happens with extra context (such as `enable_thinking`) and no messages, and in the preface render with an empty preface (run in the runtime on one Qwen3 template; the other templates rendered with minijinja 2.14 and jinja2). Read in the source of LiteRT-LM 0.17.0, 0.18.1 and main; measured with LiteRT-LM 0.18.1 (the litert-lm package from PyPI) through its C API on this file's template text. The default path, where the first message is rendered together with the preface, is not affected: the Python API and the CLI of 0.18.1 expose neither the flag nor the preface render, and Google AI Edge Gallery sets neither (its source as read on 2026-10-25). A system message or any initial message in the conversation config avoids it.
194
 
litertlm_manifest.json CHANGED
@@ -219,7 +219,7 @@
219
  }
220
  ],
221
  "known_issues": [
222
- "Android GPU needs LiteRT-LM 0.19.0 or later: since 2026-11-09 this file's weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights), and the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on such a file (https://github.com/google-ai-edge/LiteRT-LM/issues/3986); the earlier file with the weights inside the graph stays at revision 193b2d579a786206286ac36e74b6ea513562bdf5 (https://huggingface.co/litert-community/VibeVoice-ASR-BitNet/tree/193b2d579a786206286ac36e74b6ea513562bdf5)",
223
  "Audio encoder does not run on the GPU delegates: on Metal/WebGPU the 30 s window dispatches 90001+ workgroups per stem kernel (limit 65535); on Adreno (OpenCL, all 1857 nodes delegated) the fp16 path returns an empty transcript; keep audio_backend=cpu",
224
  "Audio placeholder tokens <|object_ref_start|>/<|object_ref_end|>/<|box_start|> are added to the tokenizer.json inside the file (ids 151646/151647/151648, the ids the model was trained with); the upstream tokenizer.json has no string for them"
225
  ],
 
219
  }
220
  ],
221
  "known_issues": [
222
+ "Since 2026-11-09 this file's weights sit in TFLiteWeights sections at the end of the file (LiteRT's external weights). On a Pixel 8a, the GPU engine of the v0.18.0 and v0.18.1 Android builds crashes (SIGSEGV) on a Qwen3-0.6B file with external weights (https://github.com/google-ai-edge/LiteRT-LM/issues/3986). The earlier file with the weights inside the graph stays at revision 193b2d579a786206286ac36e74b6ea513562bdf5 (https://huggingface.co/litert-community/VibeVoice-ASR-BitNet/tree/193b2d579a786206286ac36e74b6ea513562bdf5).",
223
  "Audio encoder does not run on the GPU delegates: on Metal/WebGPU the 30 s window dispatches 90001+ workgroups per stem kernel (limit 65535); on Adreno (OpenCL, all 1857 nodes delegated) the fp16 path returns an empty transcript; keep audio_backend=cpu",
224
  "Audio placeholder tokens <|object_ref_start|>/<|object_ref_end|>/<|box_start|> are added to the tokenizer.json inside the file (ids 151646/151647/151648, the ids the model was trained with); the upstream tokenizer.json has no string for them"
225
  ],