WeVisOne-VL-2B for LiteRT-LM

This repository is a conversion of tencent/WeVisOne-VL-2B to LiteRT-LM, made with litert-torch 0.10.0 for LiteRT-LM 0.18.1: on a Galaxy S26 GPU, the file answers a question about a 512 × 512 picture with its first token after 1.53 s and then writes 12.97 tokens per second (2026-10-25).

tencent/WeVisOne-VL-2B is a vision-language model by WeChat Vision, Tencent, built on Qwen3-VL. It answers detection, referring, pointing, counting and OCR requests as structured text: boxes and points are the coordinate tokens <0> … <1000>, thousandths of the picture's width and height. This file does not do segmentation or matting: those need the source model's 9.5M-parameter pixel decoder, which it does not contain.

WeVisOne-VL-2B_f16.litertlm (4,899,179,008 bytes, sha256 97f29a6a1b672ad09afe33ee8c41589f62bac19a46092458bc4c96ef3f26adcb) holds the vision encoder, the vision adapter, the token embedder and the 28-layer decoder, all with float16 weights, and the source repository's own tokenizer.json. The file tells LiteRT-LM to run it as a single-picture vision-language model (its fast_vlm model type), with 2,048 tokens of context for the prompt and the answer.

Use it

Download WeVisOne-VL-2B_f16.litertlm from this repository. Before you send a picture, turn it upright from its EXIF orientation and stretch it to 512 × 512 with Pillow's BICUBIC filter, without padding or cropping, so that the coordinates map back onto the upright picture by its width and height. Neither Pillow's Image.open nor LiteRT-LM applies the EXIF orientation by itself. Every result for this file on this card was measured on pictures resized that way and sent as PNG files (none of the test pictures is stored rotated; the upright step was checked on a copy tagged as rotated). Do the resize yourself: LiteRT-LM 0.18.1 stretches a picture of another size to 512 × 512 with its own filter (stb's Catmull-Rom), and an int8 build of this model, given the original pictures to resize that way, gave the identical answer text as with Pillow's pictures to only 9 of 29 questions on the Mac GPU and to 1 of 29 on the Mac CPU. This file was not run that way. Other resizing (Android's bitmap scaling, for example) and JPEG-compressed 512 × 512 pictures were not tested.

pip install "litert-lm==0.18.1" pillow
python3 -c "from PIL import Image, ImageOps; ImageOps.exif_transpose(Image.open('photo.jpg')).convert('RGB').resize((512, 512), Image.BICUBIC).save('photo_512.png')"
litert-lm run WeVisOne-VL-2B_f16.litertlm --backend gpu --vision-backend gpu --attachment photo_512.png --prompt "Locate all the instances that match the following description: truck. crane."

On a Mac M4 Max, with NASA's photo KSC-08pd3809 as photo.jpg, this prints <ref>truck</ref><box><60><359><601><951></box>, <ref>crane</ref><box><939><152><1000><682></box>: the truck spans 6.0–60.1 % of the width and 35.9–95.1 % of the height. --backend cpu --vision-backend cpu runs it on the CPU. The two runs wrote 8.55 GB of cache files next to the model, 6.89 GB of it the CPU's float32 copy of the decoder weights. With that cache the CPU run peaked at 19.3 GB of memory. --cache no writes no files but needs more memory: the Mac CPU runs below, made that way, peaked at about 50 GB.

The source package (Tencent/WeVisOne-VL) asks each task with a fixed prompt, built by its build_prompt. Its templates, with {…} for your words, and an example of each; the fixed words are the package's, grammar included. Every picture question on this card used one of them, and free-form questions about a picture were not tested:

Task	Template	Example
Detection	Locate all the instances that match the following description: {label}. {label}. (labels joined by ". ", a final ".")	Locate all the instances that match the following description: person. hat.
Referring	Locate the instances that match the following description: {phrase}. (the package adds the final "." unless the phrase ends with ".", "?" or "!")	Locate the instances that match the following description: the airplane.
Pointing	point to: {label}. {label} (no final ".")	point to: airplane. windsock
Counting with boxes	How many {objects} in the image? Please count them with boxes.	How many airplanes in the image? Please count them with boxes.
Counting with points	How many {objects} in the image?	How many people in the image?
OCR	detect all the word in the box format	the same
Finding a text	Locate the following text in the image: {text}	Locate the following text in the image: OVERSIZE LOAD

Answers name each label in <ref>…</ref> and follow it with <box> or <point>: four tokens per box (left, top, right, bottom) and two per point (x, y), several boxes or points of one label separated by commas inside one tag, labels separated by ", ". The counting answers in the tests ended with Total number: N., also for "Counting with points", which answers with points. The source package's parsers read this format (parse_predictions in visualization.py, and parse_prediction in evaluation/utils/parser.py, through which every score on this card was computed). On the one test picture without the asked object (a radar image asked "How many airplanes in the image? Please count them with boxes."), the original at its own resolution and at 512 × 512, and this file on every backend, answered with one box over nearly the whole picture and Total number: 1.

Three things to know when you call this file (only a second picture gives an error; a system message is ignored and a second question gets a changed answer, both without one):

• One picture per conversation, in the same first message as the question and before it (--attachment places it there). With a second picture LiteRT-LM 0.18.1 stops with INVALID_ARGUMENT: Provided more images than expected in the prompt.
• The system turn is fixed to "You are a helpful assistant.", as in the source package; a system message you pass is not used.
• One question per conversation: start a new conversation, with the picture again, for the next question. From the second message on, LiteRT-LM 0.18.1 changes what the model sees (see Limits).

The file carries no sampler settings, so LiteRT-LM 0.18.1 decodes it greedily (top-k 1), and every answer on this card is greedy; sampled answers were not tested. Creating the engine takes longer than an answer (22.2 s on the Galaxy S26 GPU, see Performance), so keep the engine and open a new conversation per question: on the Mac GPU, five conversations in a row on one engine gave the same answers as one new process per question. Every other measurement here, and every phone run, started one process per question.

On Android the numbers below come from litert_lm_advanced_main, the command-line program built from the LiteRT-LM v0.18.1 tag: --backend=gpu --vision_backend=gpu --max_num_tokens=2048 --input_prompt_file=<file>, the file holding [image:<path>] and then the question. An Android app integration was not tested, and this file's answers were checked on LiteRT-LM 0.18.1 only.

Performance

Measured on this file. Picture columns: a synthetic 512 × 512 test picture (shapes and the text "EXIT 12", drawn by a script) and a detection question, 297 prompt tokens and 60 answer tokens. Text columns: the runtime's speed benchmark, 1,024 prompt tokens and 256 generated tokens in a 2,048-token context; it measures speed only (a text-only prompt that long gets wrong positions, see Limits).

Device	Runtime, backend	Engine creation	Picture: first token	Picture: decode	Text: prefill	Text: decode	Text: first token	Peak memory
Galaxy S26 (SM-S942Q: Qualcomm SM8850, Adreno GPU; Android 16), 2026-10-25	LiteRT-LM 0.18.1, GPU (OpenCL), vision on the GPU	22.2 s	1.53 s	12.97 tok/s	1,125.93 tok/s	13.87 tok/s	0.99 s	6.56–6.58 GB (picture), 6.51 GB (text)
Mac M4 Max (macOS 27.0), 2026-10-24	litert-lm 0.18.1, GPU (WebGPU on Metal)	4.54 s	0.30 s	81.4 tok/s	3,675 tok/s	83.0 tok/s	0.29 s	8.0 GB (picture), 6.6 GB (text)
Mac M4 Max (macOS 27.0), 2026-10-24	litert-lm 0.18.1, CPU	5.70 s	4.85 s	12.6 tok/s	506 tok/s	14.9 tok/s	2.09 s	51.5 GB (picture), 49.7 GB (text), both with --cache no

• Galaxy S26: the picture's first token is the vision encoder's time (0.90 s) plus the runtime's time to first token (0.63 s). The picture numbers are the median of the second and third process (the first process built the GPU caches); the text numbers are the median of three processes. The first process after the file was copied to the phone took 1.49 s to its first token, decoded 12.41 tokens per second, spent 29.0 s creating the engine and peaked at 6.32 GB (VmHWM 6,169,416 kB); the table's 22.2 s is the median of the later picture processes. By the end of each of the two timed picture runs the phone had started to throttle the GPU (its top power level two steps lower; six steps after the first, cache-building run). Peak memory is the process's peak resident memory (Linux's VmHWM counter, read from /proc: 6,403,164 and 6,427,320 kB; text 6,354,436 kB); during the picture runs the phone's free memory (MemAvailable) fell to 881,808–952,620 kB, under 1 GB. The decoder and the vision encoder ran on the GPU in full; the runtime runs the embedder and the adapter on the CPU.
• Mac: text with litert-lm benchmark -p 1024 -d 256 --max-num-tokens 2048 --cache no, pictures through the Python API without a cache; the picture's first token is the time to the first streamed text. Medians of 3 processes; engine creation is the picture processes' (Python API, no cache); peak memory is the process's maximum resident memory.

Accuracy: agreement with the original model

The reference is the source package at 40f735b8 in float32 on the CPU, greedy, given the same 512 × 512 pictures. Each cell below counts how many of that original's boxes, counts and points a run reproduces, and the last column is the character error rate against the original's OCR text: agreement with the original model, not accuracy against ground truth (what the 512 × 512 input itself costs is under Limits). The first row is the original model itself run in bfloat16 and scored the same way. The test set is 29 questions on 17 pictures (2 synthetic test pictures, 4 NASA photos, 11 crops of the model repository's example figures) across 9 tasks (the 7 above plus segmentation and matting, whose boxes are scored), and 10 text questions.

Run	Boxes (IoU ≥ 0.75)	Counts	Points (within 2 % of the side)	OCR character error rate
The original in bfloat16	55/55	5/6	38/44	0.000
This file, Mac CPU	55/55	6/6	42/44	0.000
This file, Mac GPU	55/55	6/6	39/44	0.000
This file, Galaxy S26 GPU	55/55	6/6	35/44	0.000

The thresholds we fixed before measuring: boxes 90 %, counts 66.7 %, points 72.7 %, character error at most 0.05. The count and point thresholds are below 90 % because the original itself differs on 1 of 6 counts and 6 of 44 points in bfloat16, and the thresholds allow twice that. Every point that differs is on two questions whose float32 answers at 512 × 512 hold 9 and 23 points (crops of cups and of a pig; see Limits). On the 10 text questions every backend answers the same questions right and wrong as the float32 original. A float32 build of the same graphs (not published) gives the original's tokens on all 29 picture questions on the Mac CPU and GPU, so this file's differences there come from float16.

On-device answers (Galaxy S26 GPU, this file)

Picture (NASA, public domain)	Question	Answer, verbatim
	detect all the word in the box format	<ref>OVERSIZE</ref><box><120><788><276><858></box>, <ref>LOAD</ref><box><274><792><351><858></box>
	point to: airplane. windsock	<ref>airplane</ref><point><620><285></point>, <ref>windsock</ref><point><848><600></point>
	Locate all the instances that match the following description: person. hat.	<ref>person</ref><box><117><247><407><994>,<480><277><583><486>,<672><226><826><893></box>, <ref>hat</ref><box><480><277><551><356>,<736><226><796><323></box>

The model got each original NASA file (KSC-08pd3809, KSC-20170424-PH_SWW01_0007, S67-34267) resized to 512 × 512; the pictures above are NASA's smaller copies for display.

Masks: an Android app with the source model's pixel decoder

This file cannot cut things out: the source model's pixel decoder reads the language model's hidden state at the <OBJ> token, and a LiteRT-LM conversation returns the answer's text, not that hidden state. The cutout/ folder of this repository holds the model as separate LiteRT graphs for an app that runs them itself: the vision encoder and adapter (float16 weights), the 28-layer language model with int8 weights (1,785,462,176 bytes) or float16 weights, the pixel decoder (wevisone_mask_decoder_gs16.tflite, float16 weights), the token embeddings as a float16 table, and tokenizer.json. wevisone/ in LiteRT-Models is an Android app that runs them with LiteRT 2.3.0 CompiledModel on the GPU: pick a photo and type what to cut out, and it draws the box and shows the cut-out; Save PNG writes it as a transparent PNG. On a Galaxy S26 it took 4.7 s from a 640 × 480 test picture of coloured shapes and "the green circle" to the cut-out, and 5.8 s for a 3000 × 2029 photo (2026-10-25, the median of four runs after the first; the first run after install took 35.7 s, 30.5 s of it creating the three GPU graphs, which the later runs reuse). Against the source model in float32 at the same 512 × 512 input, its mattes on six test requests differ by at most 0.0032 in mean absolute alpha.

Limits

• Fixed 512 × 512 input. The source package chooses the resolution per picture. At 512 × 512 the float32 original reproduces its own native-resolution answers on 47 of their 56 boxes, all 6 counts and 12 of their 18 points, with OCR character error 0.105; the differences are in large photos and in small crops stretched up (on the crops of cups and of a pig it answers with 9 and 23 points, where it gives 5 and 1 at its own resolution).
• Text-only conversations past 270 tokens. In a conversation without a picture, every token after the 270th of the conversation, prompt and answer together (the system turn and the chat template count), gets a rotary position (the index the decoder's attention uses) 240 lower than the original gives it, because the decoder computes picture and text positions with one formula. A 353-token summary request broke down after 7 tokens. Conversations with a picture get the original's positions.
• The second message of a conversation. From the second message on, LiteRT-LM 0.18.1 passes the previous answer's stop token to the model with a zero embedding, so the second answer is not the model's own: after "My name is Ada.", this file answers "What is my name?" with "You're asking about your name again? Hmm, maybe you want to know my name instead? I'm Qwen, …" where the original says "You're asking about your name, but you haven't provided it yet. …" (on the Mac CPU and the Galaxy S26 GPU; the Mac GPU's second answer differs in wording, not in kind). By the runtime's source it affects every LiteRT-LM 0.18.1 file with a separate embedder section (the graph that turns token ids into embeddings, which this file has) on the CPU and the GPU, not only this one; a file of litert-community/Qwen2.5-Coder-1.5B-Instruct showed it too. The cause in the runtime's source and a replay that reproduces it: the section "A runtime defect that every file with an embedder section meets" of docs/deepstack-wide-embedding.md in hf-to-litertlm. Ask one question per conversation.
• Memory. On the 12 GB Galaxy S26 the GPU process peaks at about 6.6 GB, with the GPU at its default float16 activations, and the phone's free memory falls under 1 GB during a picture question. Our test harness stops a run when the phone's free memory falls below a set limit, so that the phone does not run out. On a Pixel 8a (8 GB) it stopped the GPU engine creation of a smaller 2.91 GB int8 build of this model at 592,360 kB (limit 600,000 kB); this file was not started there, and whether it fits an 8 GB phone is not known. On an iPhone 18 Pro, in our iOS test app with the LiteRT-LM v0.16.0 iOS framework (Metal), this file never reached an answer: iOS ended the app at 6.45 GB in 3 of 3 launches, with the increased-memory-limit entitlement. LiteRT-LM 0.18.1 was not tried on an iPhone.
• Phone CPU (Galaxy S26). The CPU path expands the float16 weights to float32: on the Galaxy S26 CPU this file decoded 0.14 tokens per second while the phone swapped.
• NPU. This repository has no NPU build. On the Galaxy S26 the harness stopped the on-phone compile of the text decoder alone (int16 activations, int8 weights) 47 s in, with 889,412 kB of free memory left (its limit for that run was 1,000,000 kB) and the phone swapping; whether the compile would have finished was not tested. This file was not compiled for the NPU.
• Repetition penalty. The source package generates with a repetition penalty of 1.05, which LiteRT-LM 0.18.1's sampler does not offer. The reference above is plain greedy; on the 29 questions, at the package's own resolution, the two differ on 1 of 18 points and on no box or count.

How it was converted

• DeepStack carried in full. Qwen3-VL adds three extra vision features (DeepStack) to the decoder after layers 0, 1 and 2, while LiteRT-LM's fast_vlm type gives the decoder one embedding per image token. Adding the three features into that one embedding changed the answers: boxes 41/55, counts 4/6, points 17/44 and OCR error 0.472 against the original. This file widens the embedding instead. Each image token carries 8,196 values: the main embedding, the three DeepStack features (3 × 2,048), an image flag and 3 zeros of padding. The decoder adds each DeepStack part after layers 0, 1 and 2, as the original does. LiteRT-LM copies image embeddings at the decoder's input width, so it passes the wide rows through.
• Qwen3-VL's positions, computed in the graph. The runtime gives the decoder one position per token. The decoder computes Qwen3-VL's three-axis rotary positions from that position and the image flag; in conversations with a picture they equal the original's at every position checked (85,542 positions of 58 test prompts).
• The vision tower rewritten in plain PyTorch for a fixed 512 × 512 input, written without the gather op that mobile GPU runtimes do not accelerate, and converted to LiteRT graphs; on the Galaxy S26 the whole encoder runs on the GPU. The float16 encoder and adapter match the original's image features at a worst per-token cosine of 0.999995 (17 pictures).
• The tokenizer section is the source repository's tokenizer.json, byte for byte.
• Other builds measured and not published. A float32 build (9.77 GB) is faster than this file on the Mac CPU and reproduced all 44 points there (3.32 s to the first token of the picture question, 14.1 tokens per second; peak 50.9 GB without a cache); the conversion script builds it with FP32=1. Int8 decoders were faster (22.56 tokens per second of text decode on the Galaxy S26 GPU) but reproduced fewer of the original's answers on that GPU (points 28/44, OCR error 0.226) and on the phone CPUs we tried (19.82 tokens per second on the S26; points 29/44 on the S26 and the Pixel 8a, 32/44 on the iPhone 18 Pro with the v0.16.0 iOS framework). Int4 reproduced 20 and 25 of the 44 points on the Mac CPU and GPU; the harness stopped a build with float32 GPU activations on the Galaxy S26 at 479,412 kB of free memory (limit 700,000 kB); and a 768 × 768 build took 0.66 s to its first token on the Mac GPU against 0.30 s, while the original at 768 × 768 reproduces only 10 of the 18 points it gives at its own resolution.

Scripts and conversion notes: hf-to-litertlm, its REPRODUCE.md (the WeVisOne-VL-2B entry) and docs/deepstack-wide-embedding.md there.

License

Apache-2.0, as the source model tencent/WeVisOne-VL-2B. The source model card states its intended use: research in visual grounding, referring localization, pointing, counting, OCR localization, referring segmentation and image matting; it is "not intended to make autonomous high-impact decisions or to identify individuals".