UI-Venus-2-9B-mlx-8bit

Highest precision in this lineup. Pick it if you have the RAM and want the fewest surprises outside pure grounding.

MLX quantization of inclusionAI/UI-Venus-2-9B โ€” a GUI-agent vision-language model on the Qwen3.5 architecture (hybrid Gated-DeltaNet linear attention + full attention, SigLIP-style vision tower). Converted with mlx-vlm 0.6.17 / mlx 0.32.2.

The lineup

Build Size Effective bpw ScreenSpot-Pro (n=50) Notes
bf16 (original) 18 GB 16 60% (30/50) Unquantized reference, measured on the same harness.
โ†’ 8bit 9.8 GB 8.86 60% (30/50) Uniform 8-bit RTN. Highest precision here.
6bit 7.7 GB 6.96 60% (30/50) Uniform 6-bit RTN.
4bit-awq 5.6 GB 5.06 58% (29/50) AWQ, calibrated on 64 real GUI screenshots.
4bit 5.6 GB 5.06 60% (30/50) Uniform 4-bit RTN, no calibration.

Measured with Click on: <instruction> over 50 held-out ScreenSpot-Pro samples (images capped at 2048 px; a prediction counts as correct only if the point falls inside the ground-truth bbox). Every build ties the unquantized bf16 reference โ€” see below. This is not the paper's evaluation protocol and 50 samples is a small set, so read it as a relative check between quants, not as a benchmark score for the model.

Usage

pip install -U mlx-vlm
python -m mlx_vlm.generate \
  --model lethargicgeek/UI-Venus-2-9B-mlx-8bit \
  --image screenshot.png \
  --prompt "Click on: the Save button" \
  --max-tokens 64
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("lethargicgeek/UI-Venus-2-9B-mlx-8bit")
prompt = apply_chat_template(processor, model.config, "Click on: the Save button", num_images=1)
print(generate(model, processor, prompt, ["screenshot.png"], max_tokens=64))

Output format

Grounding prompts return a <point> tag. Coordinates are normalized to 0โ€“1000, not pixels:

px = x / 1000 * image_width
py = y / 1000 * image_height

Both <point>X Y</point> and <point x1="X" y1="Y" alt="..."> forms occur, so parse for either.

Quantization details

Method RTN (round-to-nearest), no calibration
Bits 8 uniform
Group size 64
Mode affine

The vision tower is not quantized. All 333 vision tensors stay at bfloat16 in every build here โ€” mlx-vlm's skip_multimodal_module() excludes them. This mirrors how GGUF conversions ship the vision encoder as a separate f16 mmproj. Only the language model is quantized, which is why a "4-bit" build measures 5.06 effective bits per weight rather than 4.

What I actually measured

Quantization is free on this task. The unquantized bf16 model scores 30/50 on this harness. So do the 8-bit, 6-bit and 4-bit builds โ€” not merely the same percentage, the same count. Going from 18 GB to 5.6 GB cost nothing measurable in grounding accuracy.

These 60% figures are lower than the model's real ability, and that is my harness, not the quantization. inclusionAI report 73.0% on ScreenSpot-Pro for this model. My setup differs from theirs in three ways that each cost accuracy: I prompt with a generic Click on: <instruction> rather than the official grounding prompt (their card specifies disabling reasoning and using temperature 0), I cap images at 2048 px on a benchmark built around ~30 px targets on 4K screens, and I sample 50 items from a single shard. Since bf16 lands at the same 60%, none of that gap is attributable to quantization โ€” use their 73% as the model's grounding ability and these numbers only to compare builds against each other.

AWQ did not measurably beat plain RTN. Worth stating plainly, since calibrated quants are usually assumed to win. Partly this is coverage โ€” only 8 of 32 layers received AWQ scaling (layers: 8, groups: 24), the other 24 being linear_attention (Gated DeltaNet) layers its scale search does not handle. But mostly there was simply nothing to recover: plain 4-bit already matches bf16 here.

Structured description degrades somewhat at 4 bits. Asked to list the buttons in a screenshot, plain 4bit sometimes emits coordinate tags (<ex>10 110</ex>...) instead of prose, where the higher-bit builds and 4bit-awq returned a proper list. That was a handful of prompts โ€” suggestive, not conclusive. Ordinary description ("what application is this?", "summarize this screen") worked on every build. This, rather than grounding, is the reason to consider a higher-bit build.

Picking one

  • Grounding/clicking only โ†’ any build, including 4bit. It matched bf16.
  • You also want description/reasoning about the screen โ†’ 8bit, or 6bit to save 2 GB.
  • 4bit-awq is included for completeness and reproducibility, not because it won.

License and attribution

The upstream repo inclusionAI/UI-Venus-2-9B declares no license โ€” no license tag, no cardData.license, no LICENSE file. None is asserted here. Check with inclusionAI before using these weights in a product. (Some third-party conversions label this Apache-2.0; I could not find a basis for that in the source repo.)

Credit for the model goes to inclusionAI (technical report, project page), derived in turn from Qwen/Qwen3.5-9B. This repo contains only a quantized conversion.

Downloads last month
49
Safetensors
Model size
9B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for lethargicgeek/UI-Venus-2-9B-mlx-8bit

Finetuned
Qwen/Qwen3.5-9B
Quantized
(8)
this model

Paper for lethargicgeek/UI-Venus-2-9B-mlx-8bit