Add WeVisOne-VL-4B_int8.litertlm, the file for a CPU, with its card rows, measurements and manifest entry

.gitattributes

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 WeVisOne-VL-4B_int4.litertlm filter=lfs diff=lfs merge=lfs -text
+WeVisOne-VL-4B_int8.litertlm filter=lfs diff=lfs merge=lfs -text

MEASUREMENTS.md

@@ -1,6 +1,6 @@
 # WeVisOne-VL-4B for LiteRT-LM: measurements
 
-The [model card](https://huggingface.co/litert-community/WeVisOne-VL-4B) gives the numbers. This page keeps how they were measured, the raw values and the earlier runs. "This file" below is `WeVisOne-VL-4B_int4.litertlm`, as on the card.
+The [model card](https://huggingface.co/litert-community/WeVisOne-VL-4B) gives the numbers. This page keeps how they were measured, the raw values and the earlier runs. "This file" below is `WeVisOne-VL-4B_int4.litertlm`, as on the card, and "the int8 file" is `WeVisOne-VL-4B_int8.litertlm`, the file for a CPU.
 
 ## Performance
 
@@ -47,6 +47,25 @@ The variant (5,358,955,008 bytes, not uploaded) has the decoder and the token em
 
 Peak memory is the process's peak resident memory (VmHWM). In each of the three pairs of both orders, the variant reached the first token sooner and this file decoded faster. The variant's answers are under Correctness.
 
+### The int8 file on the CPU (2026-11-10)
+
+The int8 file has the variant's weights; the variant is the int8 file without its 1,024-token prefill graph. It ran with build A on the same Galaxy S26's CPU, with the card's command, its own `--model_path`, and `--backend=cpu --vision_backend=cpu --num_cpu_threads=4`, in one session (23:41–23:55): the picture question once, the text benchmark three times, then the picture question three more times. "(first)" marks the process that built the CPU caches; the card's values are the medians of the other processes (picture 3, text 3). Each process started at thermal status 1 with no CPU clock cap, after a rest of 78–174 s. During each one the phone capped the CPU clocks in most of the samples taken every 0.5 s (15–66 of 22–71 in the six counted processes; the lowest cap 32–47 % of a cluster's top frequency), so these are sustained speeds. Engine creation took 4.9 s in the first process and 0.28–0.41 s after it; opening the picture conversation took another 1.26 s and 0.48–1.10 s.
+
+| Start | File | Picture: first token | of it, vision encoder | Picture: decode | Peak memory |
+|---|---|---:|---:|---:|---:|
+| 23:41 | the int8 file (first) | 4.38 s | 1.10 s | 8.41 tok/s | 7.18 GB |
+| 23:51 | the int8 file | 4.16 s | 2.00 s | 10.60 tok/s | 7.38 GB |
+| 23:52 | the int8 file | 4.49 s | 1.24 s | 10.25 tok/s | 7.26 GB |
+| 23:54 | the int8 file | 4.95 s | 2.07 s | 9.84 tok/s | 7.48 GB |
+
+| Start | File | Text: prefill | Text: decode | Text: first token | Peak memory |
+|---|---|---:|---:|---:|---:|
+| 23:42 | the int8 file | 213.62 tok/s | 8.16 tok/s | 4.92 s | 5.89 GB |
+| 23:45 | the int8 file | 213.12 tok/s | 8.52 tok/s | 4.92 s | 5.89 GB |
+| 23:48 | the int8 file | 210.24 tok/s | 8.24 tok/s | 4.99 s | 5.89 GB |
+
+Peak memory is VmHWM, as above.
+
 ## Correctness
 
 ### Questions with a known answer
@@ -62,14 +81,17 @@ A run passes when, in each of four groups (boxes, counts, points, the no-target
 | Mac GPU (Apple M4 Max), LiteRT-LM 0.18.1 from PyPI, 2026-10-27 | 11 | 3 | 2 | 1 | 0.000 | 10 |
 | The original model in float32, on the CPU | 11 | 3 | 2 | 1 | 0.000 | 10 |
 | The int8 variant made for the GPU (not uploaded): Galaxy S26 GPU, LiteRT-LM 0.19.1 (build A), 2026-11-10, and 0.18.1, 2026-10-28 | 11 | 2 | 2 | 1 | 0.000 | 10 |
+| The int8 file: Galaxy S26 CPU, LiteRT-LM 0.19.1 (build A), 2026-11-10 and 11, and 0.18.1, 2026-10-27 | 11 | 3 | 2 | 1 | 0.000 | 10 |
 
 The three runs of this file gave the same answers to every question of both tests (29 picture questions token for token, 10 text questions word for word). The one miss is the original's too: asked "How many cups on the left in the image?" about a crop of the source package's counting figure, whose tag says 4, each answers 1. Asked "How many days are in a week? Answer briefly." without a picture, the original and this file both answer with seven points and `Total number: 7.`; the answer key looks for the 7.
 
 The int8 variant made for the GPU (see Performance) gave the same answers on LiteRT-LM 0.19.1 and on 0.18.1, token for token on the 29 picture questions and word for word on the 10 text questions, so it passes the test above against its own 0.18.1 run. Besides the cups, it counts 9 green balls where the figure's tag and the original say 10.
 
+The int8 file gave the same answers on the CPU with LiteRT-LM 0.19.1 and 0.18.1, token for token on the 29 picture questions and word for word on the 10 text questions, so it passes the test against its own 0.18.1 run; its one miss is the cups, as the original's.
+
 ### Agreement with the original model
 
-These numbers say how far the file drifts from the original model's own answers, not whether an answer is right. The reference is the source package at `40f735b8` in float32 on the CPU, greedy, given the same 512 × 512 pictures. On 29 questions over 17 pictures (2 synthetic test pictures, 4 NASA pictures, 10 crops of the source package's example figures and its demo photo) across 9 tasks (the first seven rows of the card's template table, plus segmentation and matting, whose boxes are scored), a run is scored by how many of the reference's boxes it reproduces (IoU ≥ 0.75), its counts, its points (within 2 % of the side) and its OCR character error rate against the reference's text. The first row is the original model itself in bfloat16.
+These numbers say how far the file drifts from the original model's own answers, not whether an answer is right. The reference is the source package at `40f735b8` in float32 on the CPU, greedy, given the same 512 × 512 pictures. On 29 questions over 17 pictures (2 synthetic test pictures, 4 NASA pictures, 10 crops of the source package's example figures and its demo photo) across 9 tasks (the first seven rows of the card's template table, plus segmentation and matting, whose boxes are scored), a run is scored by how many of the reference's boxes it reproduces (IoU ≥ 0.75), its counts, its points (within 2 % of the side) and its OCR character error rate against the reference's text. The first row is the original model itself in bfloat16. The pass rule, fixed on 2026-10-27 before any 4B file was scored: at least 90 % of the reference's boxes, counts and points, and an OCR character error rate of at most 0.05; where the bfloat16 original itself misses more than 10 % of a column, that column's line is 1 − 2 × its miss rate (counts: 2/3). This file passes in all three runs; the variant and the int8 file pass every column but OCR.
 
 | Run | Boxes (IoU ≥ 0.75) | Counts | Points (within 2 % of the side) | OCR character error rate |
 |---|---:|---:|---:|---:|
@@ -78,31 +100,35 @@ These numbers say how far the file drifts from the original model's own answers,
 | This file, Galaxy S26 GPU, LiteRT-LM 0.18.1, 2026-10-28 | 57/58 | 6/6 | 13/14 | 0.016 |
 | This file, Mac GPU, LiteRT-LM 0.18.1, 2026-10-27 | 57/58 | 6/6 | 13/14 | 0.016 |
 | The int8 variant made for the GPU (not uploaded), Galaxy S26 GPU, LiteRT-LM 0.19.1 (build A), 2026-11-10, and 0.18.1, 2026-10-28 | 54/58 | 5/6 | 13/14 | 0.098 |
+| The int8 file, Galaxy S26 CPU, LiteRT-LM 0.19.1 (build A), 2026-11-10 and 11, and 0.18.1, 2026-10-27 | 57/58 | 6/6 | 14/14 | 0.098 |
 
 The OCR difference comes from one picture, a crop of a sign from the source package's example figures that reads "HAMBURGERS": this file reads "HAMBURGERS" and the original "HAMBURGER". The variant's OCR difference is the same picture: the original reads "Image", "THOMAS", "World", "Famous", "Chili", "HAMBURGER", and the variant leaves out "Image", a small word in the picture's top right corner, and reads the other five as the original does.
 
+The int8 file's OCR difference on the CPU is the variant's: it leaves out "Image" and reads the other five words as the original does. Three fresh processes on the phone's CPU gave this answer token for token: on LiteRT-LM 0.18.1 (2026-10-27 22:19, the phone at thermal status 2 with its CPU clocks capped) and on 0.19.1 (2026-11-11 00:02, thermal status 3, capped; and 01:21, thermal status 0, in the first process of a session, which built the CPU caches). The original is not near a tie there: in float32 it puts "Image" 6.09 logits above its second choice, "Imagine" (probability 0.996), and the int8 file's next token, "TH" of "THOMAS", is the float32 original's fourth choice, 8.05 logits below "Image" (probability 0.0003); in bfloat16 the original puts "Image" 4.25 logits above "Imagine", and "TH" is its seventh choice, 8.25 logits below "Image". On the Mac CPU (LiteRT-LM 0.18.1 from PyPI, 2026-10-27) the int8 file reads "Image" as the original does (OCR error 0.000).
+
 Every score was computed through the source package's `parse_prediction` (`evaluation/utils/parser.py`). On the referring question whose figure answer is "No target", the file answered `<box>none</box>`, as the original does; asked to count airplanes with boxes in a radar picture that holds none, it answered one small box and `Total number: 1.`, as the original does. The answers in the card's "On-device answers" table are from 2026-11-10; the model got each original NASA file (KSC-08pd3809, KSC-20170424-PH_SWW01_0007, S67-34267) resized to 512 × 512.
 
 ## Limits in detail
 
 - **Text-only conversations past 271 tokens.** In a conversation without a picture, every token after the 271st of the conversation, prompt and answer together (the system turn and the chat template count), gets a rotary position 240 lower than the original gives it, because the decoder computes picture and text positions with one formula. On a 353-token summary request, the original model run in PyTorch with these positions followed its own answer for 11 tokens and then wrote "The bicycle evolved into the same time, …". Conversations with a picture get the original's positions.
 - **The second message of a conversation.** From the second message on, LiteRT-LM passes the previous answer's stop token to the model with a zero embedding, so the second answer starts from a context the original never sees. LiteRT-LM 0.19.1 has the same code as 0.18.1 (`runtime/executor/llm_litert_compiled_model_executor.cc`, L473–509, L661–672 and L1488–1504 at v0.19.1); by that source it affects every file with a separate embedder section, as this one has. The cause and a replay that reproduces it on WeVisOne-VL-2B are in the section "A runtime defect that every file with an embedder section meets" of [`docs/deepstack-wide-embedding.md`](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/deepstack-wide-embedding.md). On this file one two-message test was run: after "My name is Ada.", the second answer to "What is my name?" was "Your name is Ada.", as the original answers.
-- **Memory.** On 2026-11-10 the 12 GB Galaxy S26 ran 57 processes of this file through `litert_lm_advanced_main` and none was stopped. The highest peak resident memory was 4.02 GB (VmHWM 3,922,820 kB, on a picture question), and the phone's free memory (MemAvailable) fell to 0.92 GB at the lowest.
-- **Apps and sampling.** Every run was greedy, on the phone through `litert_lm_advanced_main` and on the Mac through the PyPI `litert-lm` package. On 2026-11-10 one Android test app on the Maven library `com.google.ai.edge.litertlm:litertlm-android:0.19.1` (decoder and vision encoder on the GPU, `maxNumTokens = 2048`, one conversation per question, no sampler settings) asked the person-and-hat question about S67-34267, resized to 512 × 512 as on the card, in five processes on the Galaxy S26; each gave the card's answer byte for byte. No other app was tried.
+- **Memory.** On 2026-11-10 the 12 GB Galaxy S26 ran 57 processes of this file through `litert_lm_advanced_main` and none was stopped. The highest peak resident memory was 4.02 GB (VmHWM 3,922,820 kB, on a picture question), and the phone's free memory (MemAvailable) fell to 0.92 GB at the lowest. The int8 file ran 91 processes on the phone's CPU (44 on LiteRT-LM 0.18.1 on 2026-10-27, 47 on 0.19.1 on 2026-11-10 and 11) and none was stopped; its highest peak was 8.00 GB (VmHWM 7,815,316 kB, on a referring question, 0.18.1), and the free memory fell to 4.78 GB at the lowest. On the phone's GPU (LiteRT-LM 0.18.1, 2026-10-27) the test harness stopped a run of the int8 file when the free memory fell below 0.7 GB (MemAvailable 696,352 kB); the variant, without the 1,024-token prefill graph, is the int8 form that ran there.
+- **Apps and sampling.** Every run was greedy, on the phone through `litert_lm_advanced_main` and on the Mac through the PyPI `litert-lm` package. On 2026-11-10 one Android test app on the Maven library `com.google.ai.edge.litertlm:litertlm-android:0.19.1` (decoder and vision encoder on the GPU, `maxNumTokens = 2048`, one conversation per question, no sampler settings) asked the person-and-hat question about S67-34267, resized to 512 × 512 as on the card, in five processes on the Galaxy S26; each gave the card's answer byte for byte. No other app was tried. The int8 file ran through `litert_lm_advanced_main` only.
 - **Repetition penalty.** The source package generates with a repetition penalty of 1.05 (`wevisone_vl/engine.py`). No run used one: the file's sampler settings have none, and the `--repetition_penalty` option of `litert_lm_advanced_main` (default 1.0) was not tested with this file. The reference answers are plain greedy.
 
 ## The file and its use, in detail
 
-- **Activations.** The file declares float32 activations for the decoder, so on a GPU LiteRT-LM 0.19.1 runs the decoder in float32 (`MaybeOverrideActivationType` in `runtime/engine/engine_settings.cc`) and the vision encoder at its default, also float32.
+- **Activations.** The file declares float32 activations for the decoder, so on a GPU LiteRT-LM 0.19.1 runs the decoder in float32 (`MaybeOverrideActivationType` in `runtime/engine/engine_settings.cc`) and the vision encoder at its default, also float32. The int8 file declares no activation type.
 - **Sampling.** The file carries no sampler settings, so LiteRT-LM 0.19.1 decodes it greedily (top-k 1, `runtime/engine/engine_settings.cc`).
 - **The Android program.** The v0.19.1 release has no Android binary; the card's numbers come from `litert_lm_advanced_main` built from the v0.19.1 tag for `android_arm64`, with the tag's `prebuilt/android_arm64` libraries in the same folder and `LD_LIBRARY_PATH` set to it, one process per answer.
-- **Cache files.** The first process writes cache files next to the model, about 0.46 GB: 436,314,752 bytes for the vision adapter (XNNPACK) and 24,204,112 bytes of GPU programs.
+- **Cache files.** The first process writes cache files next to the model, about 0.46 GB: 436,314,752 bytes for the vision adapter (XNNPACK) and 24,204,112 bytes of GPU programs. The int8 file's first process on the CPU writes about 5.69 GB, all XNNPACK caches: 4,039,354,952 bytes for the decoder, 1,211,999,392 for the vision encoder and 436,314,752 for the vision adapter.
 - **Prompts and answers.** The source package's parsers read the answers (`parse_predictions` in `visualization.py`, and `parse_prediction` in `evaluation/utils/parser.py`). The fixed words of the card's templates are the package's, grammar included. The chat template places only the first picture of the first message; a second picture was not tried with this file. The system turn is fixed to "You are a helpful assistant.", as in the source package.
 
 ## Conversion numbers
 
 - **DeepStack.** Qwen3-VL adds three extra vision features (DeepStack) to the decoder after layers 0, 1 and 2, while LiteRT-LM's `fast_vlm` type gives the decoder one embedding per image token. The file widens that embedding: each image token carries 10,244 values, the main embedding, the three DeepStack features (3 × 2,560), an image flag and 3 zeros of padding. LiteRT-LM copies image embeddings at the decoder's input width, so it passes the wide rows through.
 - **Positions.** The runtime gives the decoder one position per token. The decoder computes Qwen3-VL's three-axis rotary positions from that position and the image flag; in conversations with a picture they equal the original's at all 38,131 positions of the 29 test questions at 512 × 512.
+- **The int8 file.** The same float32 export, quantized with ai-edge-quantizer 0.10.0's `dynamic_wi8_afp32` recipe (int8 weights, one scale per output channel) on the whole decoder graph, its output layer included, and on the token embedder's table; the vision encoder and adapter in float16. It has neither the float32 activation declaration nor the GPU graph step below.
 - **The float32 activation declaration.** A build with the same int4 weights and no declaration prefilled 441.87 tokens per second on the Galaxy S26 GPU against 91.70 with it (LiteRT-LM 0.18.1, 2026-10-27), but answered one of the ten text questions wrong on the Mac GPU, so this file keeps the declaration.
 - **GPU graph.** On the Galaxy S26 GPU (LiteRT-LM 0.18.1, one session on 2026-10-27) the card's GPU graph step (the composite ops `odml.cache_update` and `odml.runtime_bmm`) raised the text decode from 11.20 to 14.76 tokens per second and cut the picture's first token from 4.33 to 3.92 s. On the Mac GPU the answers to all 29 picture questions and 10 text questions stayed the same.
 - **Source revisions.** The conversion started from the source repository's float32 release ([revision `4a7bed92`](https://huggingface.co/tencent/WeVisOne-VL-4B/tree/4a7bed92750ac9d8b8a124cede14759036932dc2)). Its current release ([revision `6b44a458`](https://huggingface.co/tencent/WeVisOne-VL-4B/tree/6b44a45873b4bbc7dca383922504430dc60eca23), 2026-10-27) holds the same weights in bfloat16: each of its 4,830,581,248 values is the float32 value rounded to the nearest bfloat16, checked value by value.

README.md

@@ -28,8 +28,9 @@ Speed still being tuned (updates follow).
 | File | Bytes | sha256 | Weights |
 |---|---:|---|---|
 | `WeVisOne-VL-4B_int4.litertlm` | 3,507,874,304 | `1c14c17a7e8bd688d888300a68e2f7512b255a275a444a3f22bde395cad4b12f` | int4 in the 36-layer decoder and its output layer (blocks of 32 values, one scale per block), int8 in the token embedder, float16 in the vision encoder and adapter |
+| `WeVisOne-VL-4B_int8.litertlm` | 5,358,955,008 | `277492d63a16615152d3445e3748126f4c7b60adea224365da1505f90d211881` | for a CPU: int8 in the decoder, its output layer and the token embedder (one scale per output channel), float16 in the vision encoder and adapter |
 
-The file holds the vision encoder, the vision adapter, the token embedder, the decoder and the source repository's own `tokenizer.json`, byte for byte. LiteRT-LM runs it as a single-picture model (its `fast_vlm` type) with 2,048 tokens of context for the prompt and the answer. How each number here was measured, the raw values and the earlier runs are in [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
+Each file holds the vision encoder, the vision adapter, the token embedder, the decoder and the source repository's own `tokenizer.json`, byte for byte. LiteRT-LM runs each as a single-picture model (its `fast_vlm` type) with 2,048 tokens of context for the prompt and the answer. Use the int4 file on a GPU and the int8 file on a CPU; below, "this file" is the int4 file. How each number here was measured, the raw values and the earlier runs are in [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
 
 ## Use it
 
@@ -48,6 +49,8 @@ litert_lm_advanced_main --model_path=WeVisOne-VL-4B_int4.litertlm --backend=gpu
 
 The first run writes about 0.46 GB of cache files next to the model. The file carries no sampler settings, so LiteRT-LM 0.19.1 decodes it greedily.
 
+The int8 file ran on the CPU with the same command, its own `--model_path`, and `--backend=cpu --vision_backend=cpu --num_cpu_threads=4` in place of the two GPU flags; its first run writes about 5.69 GB of cache files next to the model.
+
 In an Android app, the Maven library `com.google.ai.edge.litertlm:litertlm-android:0.19.1` runs the file with `backend = Backend.GPU()`, `visionBackend = Backend.GPU()`, `maxNumTokens = 2048` and `maxNumImages = 1` in its `EngineConfig`, a new conversation per question, and the message `Contents.of(Content.ImageBytes(png), Content.Text(question))` with the picture resized as above. A test app built this way gave the person-and-hat answer under On-device answers byte for byte on the same Galaxy S26 (2026-11-10).
 
 Word each request with the source package's fixed template for its task ([Tencent/WeVisOne-VL](https://github.com/Tencent/WeVisOne-VL), built by its `build_prompt`); `{…}` is your text. Free-form questions about a picture were not tested.
@@ -89,21 +92,31 @@ In two later sessions on the same phone with build A, this file and an int8 vari
 
 In all three pairs the variant reached the first token sooner and this file decoded faster. This file stays the one here: it is 1.85 GB smaller and follows the original's answers more closely (boxes 57 of 58 against the variant's 54, OCR error 0.016 against 0.098). Each value and the variant's answers: [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
 
+On the same phone's CPU, with build A, 4 threads and the vision encoder on the CPU too, the int8 file ran in one session (2026-11-10, 23:41–23:55):
+
+| File | Engine creation | Picture: first token | Picture: decode | Text: prefill | Text: decode | Peak memory (picture) |
+|---|---:|---:|---:|---:|---:|---:|
+| `WeVisOne-VL-4B_int8.litertlm`, CPU | 0.39 s | 4.49 s | 10.25 tok/s | 213.12 tok/s | 8.24 tok/s | 7.26–7.48 GB |
+
+Each value is the median of three processes, run after a first one that built the CPU caches (4.9 s of engine creation); after engine creation, opening the picture conversation took another 0.61 s. Each process started with no CPU clock cap, and the phone lowered its CPU clocks during each run, so these are sustained speeds. The int8 variant above is this int8 file without its 1,024-token prefill graph.
+
 ## Correctness
 
 | Run | Boxes (of 11) | Counts (of 4) | Points (of 2) | No target (of 1) | OCR error | Text (of 10) |
 |---|---:|---:|---:|---:|---:|---:|
 | This file: Galaxy S26 GPU on LiteRT-LM 0.19.1 (build A) and 0.18.1, Mac GPU on 0.18.1 | 11 | 3 | 2 | 1 | 0.000 | 10 |
 | The original model in float32, on the CPU | 11 | 3 | 2 | 1 | 0.000 | 10 |
+| The int8 file: Galaxy S26 CPU on LiteRT-LM 0.19.1 (build A) and 0.18.1 | 11 | 3 | 2 | 1 | 0.000 | 10 |
 
-These are 21 questions with a known answer: 11 about two drawn test pictures and three crops of the source package's figures, and 10 text-only. The three runs of this file gave the same answers, and the one miss, a count of cups, is the original's too.
+These are 21 questions with a known answer: 11 about two drawn test pictures and three crops of the source package's figures, and 10 text-only. The three runs of this file gave the same answers, and the one miss, a count of cups, is the original's too. The int8 file's two runs gave the same answers as each other, with the same one miss.
 
 | Run | Boxes (IoU ≥ 0.75) | Counts | Points (within 2 % of the side) | OCR character error rate |
 |---|---:|---:|---:|---:|
 | The original in bfloat16 | 57/58 | 5/6 | 14/14 | 0.000 |
 | This file (the same three runs) | 57/58 | 6/6 | 13/14 | 0.016 |
+| The int8 file (the same two runs) | 57/58 | 6/6 | 14/14 | 0.098 |
 
-This second table shows how closely the answers to 29 picture questions follow the original model's own answers in float32, not whether they are right. The OCR difference is one sign that reads "HAMBURGERS": this file reads it right and the original drops the S. The questions, the pass rule and each run: [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
+This second table shows how closely the answers to 29 picture questions follow the original model's own answers in float32, not whether they are right. The OCR difference is one sign that reads "HAMBURGERS": this file reads it right and the original drops the S. The int8 file's OCR difference is on the same sign: on the CPU it leaves out "Image", a small word in the corner that the original reads first, in each of three fresh processes, and reads the other five words as the original does. On 1 of the 29 questions, this puts its OCR error at 0.098, above the 0.05 that the pass rule for this table allows. The questions, the pass rule and each run: [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
 
 ## On-device answers (Galaxy S26 GPU, LiteRT-LM 0.19.1)
 
@@ -120,8 +133,8 @@ The model got each original NASA file resized to 512 × 512; the pictures shown
 - **512 × 512 only.** The file always reads 512 × 512, and every comparison on this card is at that size; the source package chooses the resolution per picture, and how far the original's answers at its own resolution differ was not measured for the 4B. Resize your picture as in Use it.
 - **Keep text-only conversations within 271 tokens.** Without a picture, every token after the 271st of the conversation (prompt, answer and chat template together) gets a wrong position, and from there the answer can differ from the original's. Conversations with a picture are not affected.
 - **One question per conversation.** From the second message on, LiteRT-LM passes the previous answer's stop token to the model with a zero embedding, so the second answer starts from a context the original never sees (the cause, and a replay on WeVisOne-VL-2B: [`docs/deepstack-wide-embedding.md`](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/deepstack-wide-embedding.md#a-runtime-defect-that-every-file-with-an-embedder-section-meets)).
-- **Memory.** The 12 GB Galaxy S26 ran 57 processes of this file and stopped none; the highest peak memory of any process was 4.02 GB. Phones with less memory were not tried.
-- **Not tested:** running on a CPU, desktop runs on LiteRT-LM 0.19.1, apps other than the one test app in Use it, sampled answers, and the source package's repetition penalty of 1.05. There is no NPU build.
+- **Memory.** The 12 GB Galaxy S26 ran 57 processes of this file (highest peak memory 4.02 GB) and 91 of the int8 file on its CPU (8.00 GB) and stopped none. On that phone's GPU, the test harness stopped a run of the int8 file when the free memory fell below 0.7 GB (LiteRT-LM 0.18.1). Phones with less memory were not tried.
+- **Not tested:** this file on a CPU, the int8 file in an app (it ran through `litert_lm_advanced_main` only), desktop runs on LiteRT-LM 0.19.1, apps other than the one test app in Use it, sampled answers, and the source package's repetition penalty of 1.05. There is no NPU build.
 
 Details: [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4B/blob/main/MEASUREMENTS.md).
 
@@ -131,6 +144,7 @@ Details: [MEASUREMENTS.md](https://huggingface.co/litert-community/WeVisOne-VL-4
 - **Qwen3-VL's positions, computed in the graph** from the runtime's one position per token and an image flag. With a picture they equal the original's at every position of the test questions.
 - **The vision tower rewritten in plain PyTorch** for a fixed 512 × 512 input, without a gather op, and converted to [LiteRT](https://github.com/google-ai-edge/litert) graphs with float16 weights. On the Galaxy S26 the whole encoder runs on the GPU.
 - **int4 decoder.** ai-edge-quantizer 0.10.0 put the decoder's weights, its output layer included, in int4 blocks of 32 with one scale per block, every block scale kept at or above 2^-14, and the token embedder in int8 (one scale per output channel). The file declares float32 activations for the decoder: without the declaration the prefill on the Galaxy S26 GPU was faster, but one of the ten text questions was answered wrong on the Mac GPU.
+- **int8 file.** The same export with ai-edge-quantizer 0.10.0's dynamic int8 recipe (one scale per output channel) on the decoder, its output layer and the token embedder, without the activation declaration and the GPU graph step.
 - **GPU graph.** After the export, [`gpu_graph_retrofit.py`](https://github.com/john-rocky/hf-to-litertlm/blob/main/tools/gpu_graph/gpu_graph_retrofit.py) in hf-to-litertlm rewrote the decoder's KV-cache writes and attention products into composite ops that LiteRT-LM's GPU path recognizes (`odml.cache_update`, `odml.runtime_bmm`), with every weight byte the same. On the Galaxy S26 GPU the decode and the picture's first token got faster; on the Mac GPU the answers stayed the same.
 
 The conversion started from the source's float32 release (revision `4a7bed92`); its current bfloat16 release (`6b44a458`) holds the same weights. It follows the WeVisOne-VL-2B recipe in [hf-to-litertlm](https://github.com/john-rocky/hf-to-litertlm) ([REPRODUCE.md](https://github.com/john-rocky/hf-to-litertlm/blob/main/REPRODUCE.md), the WeVisOne-VL-2B entry, and [`docs/deepstack-wide-embedding.md`](https://github.com/john-rocky/hf-to-litertlm/blob/main/docs/deepstack-wide-embedding.md)) at the 4B's dimensions, with the int4 decoder and the GPU graph step above. REPRODUCE.md has no entry for the 4B itself yet.

WeVisOne-VL-4B_int8.litertlm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:277492d63a16615152d3445e3748126f4c7b60adea224365da1505f90d211881
+size 5358955008

litertlm_manifest.json

@@ -1,7 +1,7 @@
 {
   "manifest_schema": "0.1.3",
   "repo": "litert-community/WeVisOne-VL-4B",
-  "generated": "2026-11-10",
+  "generated": "2026-11-11",
   "generator": "make_manifest.py",
   "model": {
     "display_name": "WeVisOne-VL-4B (LiteRT-LM)",
@@ -22,8 +22,8 @@
   "variants": [
     {
       "file": "WeVisOne-VL-4B_int4.litertlm",
-      "size_bytes": 3507874304,
       "sha256": "1c14c17a7e8bd688d888300a68e2f7512b255a275a444a3f22bde395cad4b12f",
+      "size_bytes": 3507874304,
       "sections": [
         {
           "type": "LlmMetadataProto",
@@ -181,6 +181,134 @@
         }
       ],
       "context_length": 2048
+    },
+    {
+      "file": "WeVisOne-VL-4B_int8.litertlm",
+      "size_bytes": 5358955008,
+      "sha256": "277492d63a16615152d3445e3748126f4c7b60adea224365da1505f90d211881",
+      "sections": [
+        {
+          "type": "LlmMetadataProto",
+          "size_bytes": 1449
+        },
+        {
+          "type": "ExecutorMetadataProto",
+          "size_bytes": 4893
+        },
+        {
+          "type": "HF_Tokenizer_Zlib",
+          "size_bytes": 2148495
+        },
+        {
+          "type": "TFLiteModel",
+          "size_bytes": 392695648,
+          "model_type": "tf_lite_embedder"
+        },
+        {
+          "type": "TFLiteModel",
+          "size_bytes": 4133951184,
+          "model_type": "tf_lite_prefill_decode"
+        },
+        {
+          "type": "TFLiteModel",
+          "size_bytes": 611814736,
+          "model_type": "tf_lite_vision_encoder"
+        },
+        {
+          "type": "TFLiteModel",
+          "size_bytes": 218245632,
+          "model_type": "tf_lite_vision_adapter"
+        }
+      ],
+      "quantization": "For a CPU. int8 weights (ai-edge-quantizer 0.10.0 dynamic_wi8_afp32, one scale per output channel) in the decoder (36 layers + untied output head) and in the token embedder outside the decoder graph (EMBEDDER section, 10244-wide rows); fp16 vision encoder and adapter. No activation type declared; the decoder graph has no GPU composite ops (the export as written, not rewritten)",
+      "backends": [
+        "cpu"
+      ],
+      "default_backend": "cpu",
+      "min_runtime_version": "0.18.1",
+      "recommended": [
+        {
+          "platform": "android",
+          "device_class": "12 GB flagship (Galaxy S26, Qualcomm SM8850 CPU)",
+          "backend": "cpu",
+          "reason": "Galaxy S26 CPU (4 threads, vision on the CPU), litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag: a 512x512 picture + question to first token 4.49 s, decode 10.25 tok/s; text 1,024 / 256: 213.12 / 8.24 tok/s (medians of 3; sustained: the phone capped the CPU clocks during each run). Engine creation 0.39 s after the first process, then 0.61 s to open a picture conversation. Known-answer test: the same score as the float32 original (11 picture and 10 text questions); against the original's own answers it leaves out one word ('Image') on 1 of 29 picture questions (OCR character error rate 0.098)."
+        }
+      ],
+      "requirements": {
+        "peak_ram_mb": 7632,
+        "platform_notes": [
+          "Turn the picture upright from its EXIF orientation and stretch it to 512x512 with PIL BICUBIC before sending it: every accuracy and speed row was measured on pictures resized that way and sent as PNG files; other resizing was not tested.",
+          "Measured with --backend=cpu --vision_backend=cpu --num_cpu_threads=4. The first process writes 5.69 GB of XNNPACK caches next to the model (decoder 4,039,354,952 B, vision encoder 1,211,999,392 B, vision adapter 436,314,752 B); engine creation then takes 0.28-0.41 s on the Galaxy S26 CPU (4.9 s for the first process), and opening a picture conversation 0.48-1.10 s.",
+          "The file carries no sampler settings: LiteRT-LM 0.19.1 decodes it greedily (top-k 1, runtime/engine/engine_settings.cc), and every measured answer is greedy.",
+          "One picture per conversation, in the first message before the question; the chat template places only the first message's first picture. The system turn is fixed to 'You are a helpful assistant.' (the source package's); a passed system message is not rendered.",
+          "One question per conversation: from the second message on, LiteRT-LM (0.18.1 and 0.19.1, runtime/executor/llm_litert_compiled_model_executor.cc) passes the previous answer's stop token to the model with a zero embedding; measured on WeVisOne-VL-2B and Qwen2.5-Coder-1.5B-Instruct, not on this file. On this file the two-message text question was run on the Galaxy S26 CPU: after 'My name is Ada.', the second answer to 'What is my name?' was 'Your name is Ada.'.",
+          "Galaxy S26 CPU, 91 processes (44 on LiteRT-LM 0.18.1 on 2026-10-27, 47 on 0.19.1 on 2026-11-10 and 11): peak VmHWM up to 7,815,316 kB (a referring question, 0.18.1); the phone's free memory (MemAvailable) fell to 4,671,108 kB at the lowest; none was stopped. On the Galaxy S26 GPU (LiteRT-LM 0.18.1, 2026-10-27) the test harness stopped a run of this file when MemAvailable fell below 700,000 kB (696,352 kB): on a GPU use WeVisOne-VL-4B_int4.litertlm.",
+          "Known-answer test (11 picture questions, 10 text questions, fixed 2026-10-30): on the Galaxy S26 CPU with LiteRT-LM 0.19.1 (2026-11-10 and 11) boxes 11/11, counts 3/4, points 2/2, no target 1/1, OCR character error rate 0.000, text 10/10, the same score as the float32 original; all 29 picture answers equal the 2026-10-27 LiteRT-LM 0.18.1 run token for token and all 10 text answers word for word. Agreement with the float32 original on 29 picture questions: boxes 57/58 (IoU >= 0.75), counts 6/6, points 14/14 (within 2 % of the side), OCR character error rate 0.098 (the bfloat16 original: 57/58, 5/6, 14/14, 0.000). The OCR difference is one word on one picture: the file leaves out 'Image', which the original reads first and puts 6.09 logits above its second choice in float32; three fresh processes on the phone's CPU (0.18.1 once, 0.19.1 twice) gave the same tokens. On the Mac CPU (LiteRT-LM 0.18.1 from PyPI) the file reads 'Image'.",
+          "Text-only conversations: every token after the 271st of the conversation gets a rotary position 240 lower than the original's (one position formula for picture and text); on a 353-token summary request the original model given these positions in PyTorch followed its own answer for 11 tokens, then broke down. Conversations with a picture get the original's positions.",
+          "GPU: use WeVisOne-VL-4B_int4.litertlm. Apps: this file ran through litert_lm_advanced_main only. NPU: no NPU build. iPhone: not run."
+        ]
+      },
+      "measured": [
+        {
+          "device": "Galaxy S26 (SM-S942Q, Android 16, Qualcomm SM8850)",
+          "backend": "cpu",
+          "runtime": "litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (sha256 5d869be7..., GPU accelerator statically linked = the tag's default), XNNPACK, --num_cpu_threads=4; vision encoder on the CPU",
+          "prompt_tokens": 297,
+          "decode_tokens": 60,
+          "prefill_tps": "94.31-144.03",
+          "decode_tps": "9.84-10.60",
+          "ttft_s": "4.16-4.95",
+          "load_s": 0.39,
+          "peak_memory_mb": 7132,
+          "max_num_tokens": 2048,
+          "cache": "disk",
+          "runs": 3,
+          "date": "2026-11-10",
+          "source": "a 512x512 drawn test picture + a detection question; one session, the 3 picture processes after the first one (which built the XNNPACK caches: engine creation 4.9 s); ttft_s = the vision encoder's 1.24-2.07 s + the runtime's time to first token 2.16-3.25 s; opening the picture conversation took another 0.48-1.10 s (Init Conversation); each process started at thermal status 1 with no CPU clock cap, and the phone capped the CPU clocks in 15-21 of 22-25 samples (every 0.5 s) of each run, down to 45-47 % of a cluster's top frequency: sustained speeds; peak = the larger VmHWM / 1024"
+        },
+        {
+          "device": "Galaxy S26 (SM-S942Q, Android 16, Qualcomm SM8850)",
+          "backend": "cpu",
+          "runtime": "litert_lm_advanced_main built from the LiteRT-LM v0.19.1 tag (sha256 5d869be7..., GPU accelerator statically linked = the tag's default), XNNPACK, --num_cpu_threads=4",
+          "prompt_tokens": 1024,
+          "decode_tokens": 256,
+          "prefill_tps": "210.24-213.62",
+          "decode_tps": "8.16-8.52",
+          "ttft_s": "4.92-4.99",
+          "load_s": 0.28,
+          "peak_memory_mb": 5615,
+          "max_num_tokens": 2048,
+          "cache": "disk",
+          "runs": 3,
+          "date": "2026-11-10",
+          "source": "forced 1,024-token prefill and 256 decoded tokens (--benchmark_prefill_tokens=1024 --benchmark_decode_tokens=256); the same session, 3 processes after the picture question's first process built the caches; each started at thermal status 1 with no CPU clock cap, and the phone capped the CPU clocks in 62-66 of 68-71 samples of each run, down to 32 % of a cluster's top frequency: sustained speeds; peak = the larger VmHWM / 1024"
+        }
+      ],
+      "checked": [
+        {
+          "device": "Galaxy S26 (SM-S942Q, Android 16, Qualcomm SM8850)",
+          "os": "android",
+          "backend": "cpu",
+          "runtime": "0.19.1",
+          "top_k": 1,
+          "top_p": 0.95,
+          "temperature": 1.0,
+          "runs": 1,
+          "date": "2026-11-10"
+        },
+        {
+          "device": "Galaxy S26 (SM-S942Q, Android 16, Qualcomm SM8850)",
+          "os": "android",
+          "backend": "cpu",
+          "runtime": "0.18.1",
+          "top_k": 1,
+          "top_p": 0.95,
+          "temperature": 1.0,
+          "runs": 1,
+          "date": "2026-10-27"
+        }
+      ],
+      "context_length": 2048
     }
   ]
 }