WeVisOne-VL-4B for LiteRT-LM

This repository is a conversion of tencent/WeVisOne-VL-4B to LiteRT-LM, made with litert-torch 0.10.0: on a Galaxy S26 GPU, with LiteRT-LM built from its v0.19.1 tag, WeVisOne-VL-4B_int4.litertlm answers a question about a 512 × 512 picture with its first token after 4.86 s and then writes 15.2 tokens per second (2026-11-10). On 21 test questions with a known answer (11 about pictures, 10 text-only) it scores the same as the original model in float32.

Speed still being tuned (updates follow).

tencent/WeVisOne-VL-4B is a vision-language model by WeChat Vision, Tencent, built on Qwen3-VL. It answers detection, referring, pointing, counting and OCR requests as structured text: boxes and points are the coordinate tokens <0> … <1000>, thousandths of the picture's width and height. The files here do not do segmentation or matting: those need the source model's 9.6M-parameter pixel decoder, which they do not contain.

File	Bytes	sha256	Weights
WeVisOne-VL-4B_int4.litertlm	3,507,874,304	1c14c17a7e8bd688d888300a68e2f7512b255a275a444a3f22bde395cad4b12f	int4 in the 36-layer decoder and its output layer (blocks of 32 values, one scale per block), int8 in the token embedder, float16 in the vision encoder and adapter
WeVisOne-VL-4B_int8.litertlm	5,358,955,008	277492d63a16615152d3445e3748126f4c7b60adea224365da1505f90d211881	for a CPU: int8 in the decoder, its output layer and the token embedder (one scale per output channel), float16 in the vision encoder and adapter

Each file holds the vision encoder, the vision adapter, the token embedder, the decoder and the source repository's own tokenizer.json, byte for byte. LiteRT-LM runs each as a single-picture model (its fast_vlm type) with 2,048 tokens of context for the prompt and the answer. Use the int4 file on a GPU and the int8 file on a CPU; below, "this file" is the int4 file. How each number here was measured, the raw values and the earlier runs are in MEASUREMENTS.md.

Use it

Turn the picture upright from its EXIF orientation and stretch it to 512 × 512 with Pillow's BICUBIC filter, without padding or cropping; the coordinates then map back onto the upright picture by its width and height. Every result here used pictures resized this way and saved as PNG; other resizing was not tested.

pip install pillow
python3 -c "from PIL import Image, ImageOps; ImageOps.exif_transpose(Image.open('photo.jpg')).convert('RGB').resize((512, 512), Image.BICUBIC).save('photo_512.png')"

On Android, build litert_lm_advanced_main, a command-line program of LiteRT-LM, from its v0.19.1 tag for android_arm64 (the v0.19.1 release has no Android build of it), put the tag's prebuilt/android_arm64 libraries in the same folder and set LD_LIBRARY_PATH to it. Each answer on this card from this program is one process with these flags (the int8 file's CPU flags are below). The prompt file holds [image:<path to photo_512.png>] and then the question, with nothing between them and no newline at the end:

litert_lm_advanced_main --model_path=WeVisOne-VL-4B_int4.litertlm --backend=gpu --vision_backend=gpu --max_num_tokens=2048 --max_output_tokens=1024 --async=false --use_session=false --benchmark --benchmark_prefill_tokens=0 --benchmark_decode_tokens=0 --report_peak_memory_footprint=true --input_prompt_file=<prompt file>

The first run writes about 0.46 GB of cache files next to the model. The file carries no sampler settings, so LiteRT-LM 0.19.1 decodes it greedily.

The int8 file ran on the CPU with the same command, its own --model_path, and --backend=cpu --vision_backend=cpu --num_cpu_threads=4 in place of the two GPU flags; its first run writes about 5.69 GB of cache files next to the model.

In an Android app, the Maven library com.google.ai.edge.litertlm:litertlm-android:0.19.1 runs the int4 file with backend = Backend.GPU(), visionBackend = Backend.GPU(), maxNumTokens = 2048 and maxNumImages = 1 in its EngineConfig, a new conversation per question, and the message Contents.of(Content.ImageBytes(png), Content.Text(question)) with the picture resized as above. A test app built this way reproduced, byte for byte, the person-and-hat answer in the On-device answers table below, on the same Galaxy S26 (2026-11-10).

Word each request with the source package's fixed template for its task (Tencent/WeVisOne-VL, built by its build_prompt); {…} is your text. Free-form questions about a picture were not tested.

Task	Template	Example
Detection	Locate all the instances that match the following description: {label}. {label}. (labels joined by ". ", a final ".")	Locate all the instances that match the following description: person. hat.
Referring	Locate the instances that match the following description: {phrase}.	Locate the instances that match the following description: the airplane.
Pointing	point to: {label}. {label} (no final ".")	point to: airplane. windsock
Counting with boxes	How many {objects} in the image? Please count them with boxes.	How many airplanes in the image? Please count them with boxes.
Counting with points	How many {objects} in the image?	How many people in the image?
OCR	detect all the word in the box format	the same
Finding a text	Locate the following text in the image: {text}	Locate the following text in the image: OVERSIZE LOAD
Segmentation or matting (this file gives the box only)	Locate the instances that match the following description and provide the alpha mattes: {phrase}.	Locate the instances that match the following description and provide the alpha mattes: the man on the left.

Answers name each label in <ref>…</ref> and follow it with <box> (left, top, right, bottom) or <point> (x, y); several boxes or points of one label are separated by commas inside one tag. The counting answers in the tests ended with Total number: N. To a referring question with no matching object the file answered <box>none</box>, as the original does. Asked to count airplanes with boxes in a radar picture that holds none, it answered one small box and Total number: 1., as the original does. To the segmentation and matting template the file answers with a box that ends in an <OBJ> token, and no mask. The source package's parse_predictions (visualization.py) reads this format.

• Put one picture in the first message, before the question. The chat template places only the first picture of the first message; a second picture was not tried.
• The system turn is fixed to "You are a helpful assistant."; a system message you pass is not used.
• Ask one question per conversation; for the next question, start a new conversation and send the picture again (see Limits).

Performance

Measured on a Galaxy S26 (SM-S942Q, Adreno GPU, Android 16) on 2026-11-10 with LiteRT-LM 0.19.1; the first two tables ran the decoder and the vision encoder on the GPU. Picture: a 512 × 512 picture and a detection question (297 prompt tokens, 60 answer tokens). Text: the runtime's speed benchmark with 1,024 prompt tokens; it measures speed only (a text-only prompt that long gets wrong positions, see Limits).

LiteRT-LM 0.19.1 build	Engine creation	Picture: first token	Picture: decode	Text: prefill	Peak memory (picture)
A: built from the tag (GPU accelerator linked in, the tag's default)	11.5 s	4.86 s	15.2 tok/s	108.20 tok/s	3.78–3.79 GB
B: the same commit, loading the tag's prebuilt libLiteRtGpuAccelerator.so	11.9 s	4.22 s	15.1 tok/s	90.45 tok/s	3.68–3.80 GB

Each time and speed is the median of two processes, run after a first process built the GPU caches; memory is the range of the two. After engine creation, opening the picture conversation took another 4.9 s (A) and 5.4–5.7 s (B). On the Galaxy S26, in both pairs of processes (run in the order A, B, B, A), build A prefilled the text faster and build B, which loads the prebuilt libLiteRtGpuAccelerator.so, reached the picture's first token sooner; the answers were checked with build A only. Method, raw values and how build B is made: MEASUREMENTS.md.

In two later sessions on the same phone with build A, this file took turns with an int8 variant made for the GPU (the int8 file without its 1,024-token prefill graph; 5.36 GB, not uploaded): this file, the variant, the variant, this file, this file, the variant (2026-11-10). Each value is the median of three processes (the variant's picture values: four). This file's first token took 4.61 s in these sessions and 4.86 s in the table above.

File	Picture: first token	Picture: decode	Text: prefill	Text: decode
This file	4.61 s	15.2 tok/s	109.74 tok/s	14.83 tok/s
The int8 variant (not uploaded)	3.53 s	11.3 tok/s	167.05 tok/s	11.63 tok/s

In all three pairs the variant reached the first token sooner and this file decoded faster. This file stays this repository's file for a GPU: it is 1.85 GB smaller and follows the original's answers more closely (boxes 57 of 58 against the variant's 54, OCR error 0.016 against 0.098). Each value and how the variant's answers scored: MEASUREMENTS.md.

On the same phone's CPU, with build A, 4 threads and the vision encoder on the CPU too, the int8 file ran in one session (2026-11-10, 23:41–23:55):

File	Engine creation	Picture: first token	Picture: decode	Text: prefill	Text: decode	Peak memory (picture)
WeVisOne-VL-4B_int8.litertlm, CPU	0.39 s	4.49 s	10.25 tok/s	213.12 tok/s	8.24 tok/s	7.26–7.48 GB

Each time and speed is the median of three processes, run after a first one that built the CPU caches (4.9 s of engine creation), and memory is the range of the three; after engine creation, opening the picture conversation took another 0.61 s. Each process started with no limit on the CPU's frequency, and the phone set one during each run, so these speeds include that slowdown.

Correctness

Run	Boxes (of 11)	Counts (of 4)	Points (of 2)	No target (of 1)	OCR error	Text (of 10)
This file: Galaxy S26 GPU on LiteRT-LM 0.19.1 (build A) and 0.18.1, Mac GPU on 0.18.1	11	3	2	1	0.000	10
The original model in float32, on the CPU	11	3	2	1	0.000	10
The int8 file: Galaxy S26 CPU on LiteRT-LM 0.19.1 (build A) and 0.18.1	11	3	2	1	0.000	10

These are 21 questions with a known answer: 11 about two drawn test pictures and three crops of the source package's figures, and 10 text-only. The three runs of this file gave the same answers, and the one miss, a count of cups, is the original's too. The int8 file's two runs gave the same answers as each other, with the same one miss.

Run	Boxes (IoU ≥ 0.75)	Counts	Points (within 2 % of the side)	OCR character error rate
The original in bfloat16	57/58	5/6	14/14	0.000
This file (the same three runs)	57/58	6/6	13/14	0.016
The int8 file (the same two runs)	57/58	6/6	14/14	0.098

This second table shows how closely the answers to 29 picture questions follow the original model's own answers in float32, not whether they are right. The OCR difference is one sign that reads "HAMBURGERS": this file reads it right and the original drops the S. The int8 file's OCR difference is in the same picture: on the Galaxy S26's CPU it leaves out "Image", a small label in the picture's corner that the original reads first, and reads the other five words as the original does; three separate processes gave this answer. This one answer, 1 of the 29, puts the int8 file's OCR error at 0.098, above the 0.05 that this table's pass rule allows. The questions, the pass rule and each run: MEASUREMENTS.md.

On-device answers (Galaxy S26 GPU, LiteRT-LM 0.19.1)

Picture (NASA, public domain)	Question	WeVisOne-VL-4B_int4.litertlm, verbatim
	detect all the word in the box format	<ref>OVERSIZE</ref><box><119><790><263><859></box>, <ref>LOAD</ref><box><277><792><353><858></box>
	point to: airplane. windsock	<ref>airplane</ref><point><605><284></point>, <ref>windsock</ref><point><846><600></point>
	Locate all the instances that match the following description: person. hat.	<ref>person</ref><box><116><246><408><999>,<478><276><583><484>,<670><222><824><892></box>, <ref>hat</ref><box><478><276><550><356>,<737><221><801><323></box>

The model got each original NASA file resized to 512 × 512; the pictures shown are NASA's smaller copies.

Limits

• 512 × 512 only. The file always reads 512 × 512, and every comparison on this card is at that size; the source package chooses the resolution per picture, and how far the original's answers at its own resolution differ was not measured for the 4B. Resize your picture as in Use it.
• Keep text-only conversations within 271 tokens. Without a picture, every token after the 271st of the conversation (prompt, answer and chat template together) gets a wrong position, and from there the answer can differ from the original's. Conversations with a picture are not affected.
• One question per conversation. From the second message on, LiteRT-LM passes the previous answer's stop token to the model with a zero embedding, so the second answer starts from a context the original never sees (the cause, and a replay on WeVisOne-VL-2B: docs/deepstack-wide-embedding.md).
• Memory. On LiteRT-LM 0.19.1 the 12 GB Galaxy S26 ran 57 processes of this file (highest peak memory 4.02 GB); on 0.18.1 and 0.19.1 it ran 91 of the int8 file on its CPU (8.00 GB); none was stopped. On that phone's GPU, the test harness stopped a run of the int8 file when the free memory fell to 0.71 GB (LiteRT-LM 0.18.1). Phones with less memory were not tried.
• Not tested: this file on a CPU, the int8 file in an app (on the phone it ran through litert_lm_advanced_main only), desktop runs on LiteRT-LM 0.19.1, apps other than the one test app in Use it, sampled answers, and the source package's repetition penalty of 1.05. There is no NPU build.

Details: MEASUREMENTS.md.

How it was converted

• DeepStack carried in full. Each image token carries the main embedding and Qwen3-VL's three DeepStack features, and the decoder adds each DeepStack feature after layers 0, 1 and 2, as the original does.
• Qwen3-VL's positions, computed in the graph from the runtime's one position per token and an image flag. With a picture they equal the original's at every position of the test questions.
• The vision tower rewritten in plain PyTorch for a fixed 512 × 512 input, without a gather op, and converted to LiteRT graphs with float16 weights. On the Galaxy S26 the whole encoder runs on the GPU.
• int4 decoder. ai-edge-quantizer 0.10.0 put the decoder's weights, its output layer included, in int4 blocks of 32 with one scale per block, every block scale kept at or above 2^-14, and the token embedder in int8 (one scale per output channel). The file declares float32 activations for the decoder: without the declaration the prefill on the Galaxy S26 GPU was faster, but one of the ten text questions was answered wrong on the Mac GPU.
• int8 file. The same export with ai-edge-quantizer 0.10.0's dynamic int8 recipe (one scale per output channel) on the decoder, its output layer and the token embedder, without the activation declaration and the GPU graph step below.
• GPU graph. After the export, gpu_graph_retrofit.py in hf-to-litertlm rewrote the decoder's KV-cache writes and attention products into composite ops that LiteRT-LM's GPU path recognizes (odml.cache_update, odml.runtime_bmm), with every weight byte the same. On the Galaxy S26 GPU the decode and the picture's first token got faster; on the Mac GPU the answers stayed the same.

The conversion started from the source's float32 release (revision 4a7bed92); its current bfloat16 release (6b44a458) holds the same weights. It follows the WeVisOne-VL-2B recipe in hf-to-litertlm (REPRODUCE.md, the WeVisOne-VL-2B entry, and docs/deepstack-wide-embedding.md) at the 4B's dimensions, with the int4 decoder and the GPU graph step above. REPRODUCE.md has no entry for the 4B itself yet.

Known runtime issues

• Two builds of LiteRT-LM 0.19.1 for Android. The v0.19.1 tag build links the GPU accelerator in from source (its log: Statically linked GPU accelerator registered.). On the Galaxy S26, with this file, it prefilled the 1,024-token text benchmark faster but reached the picture question's first token later than the same commit loading the prebuilt libLiteRtGpuAccelerator.so (Performance); of the two, only the tag's default build had its answers checked.

License

Apache-2.0, as the source model tencent/WeVisOne-VL-4B. The source model card states its intended use: research in visual grounding, referring localization, pointing, counting, OCR localization, referring segmentation and image matting; it is "not intended to make autonomous high-impact decisions or to identify individuals".