Instructions to use lethargicgeek/UI-Venus-2-9B-mlx-8bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use lethargicgeek/UI-Venus-2-9B-mlx-8bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("lethargicgeek/UI-Venus-2-9B-mlx-8bit") config = load_config("lethargicgeek/UI-Venus-2-9B-mlx-8bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use lethargicgeek/UI-Venus-2-9B-mlx-8bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "lethargicgeek/UI-Venus-2-9B-mlx-8bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "lethargicgeek/UI-Venus-2-9B-mlx-8bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use lethargicgeek/UI-Venus-2-9B-mlx-8bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "lethargicgeek/UI-Venus-2-9B-mlx-8bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default lethargicgeek/UI-Venus-2-9B-mlx-8bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use lethargicgeek/UI-Venus-2-9B-mlx-8bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "lethargicgeek/UI-Venus-2-9B-mlx-8bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "lethargicgeek/UI-Venus-2-9B-mlx-8bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
UI-Venus-2-9B-mlx-8bit
Highest precision in this lineup. Pick it if you have the RAM and want the fewest surprises outside pure grounding.
MLX quantization of inclusionAI/UI-Venus-2-9B โ a GUI-agent vision-language model on the Qwen3.5 architecture (hybrid Gated-DeltaNet linear attention + full attention, SigLIP-style vision tower). Converted with mlx-vlm 0.6.17 / mlx 0.32.2.
The lineup
| Build | Size | Effective bpw | ScreenSpot-Pro (n=50) | Notes |
|---|---|---|---|---|
| bf16 (original) | 18 GB | 16 | 60% (30/50) | Unquantized reference, measured on the same harness. |
โ 8bit |
9.8 GB | 8.86 | 60% (30/50) | Uniform 8-bit RTN. Highest precision here. |
6bit |
7.7 GB | 6.96 | 60% (30/50) | Uniform 6-bit RTN. |
4bit-awq |
5.6 GB | 5.06 | 58% (29/50) | AWQ, calibrated on 64 real GUI screenshots. |
4bit |
5.6 GB | 5.06 | 60% (30/50) | Uniform 4-bit RTN, no calibration. |
Measured with Click on: <instruction> over 50 held-out
ScreenSpot-Pro samples (images capped at 2048 px;
a prediction counts as correct only if the point falls inside the ground-truth bbox). Every build ties the unquantized bf16 reference โ see below. This is not the paper's evaluation protocol and 50 samples is a small
set, so read it as a relative check between quants, not as a benchmark score for the model.
Usage
pip install -U mlx-vlm
python -m mlx_vlm.generate \
--model lethargicgeek/UI-Venus-2-9B-mlx-8bit \
--image screenshot.png \
--prompt "Click on: the Save button" \
--max-tokens 64
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template
model, processor = load("lethargicgeek/UI-Venus-2-9B-mlx-8bit")
prompt = apply_chat_template(processor, model.config, "Click on: the Save button", num_images=1)
print(generate(model, processor, prompt, ["screenshot.png"], max_tokens=64))
Output format
Grounding prompts return a <point> tag. Coordinates are normalized to 0โ1000, not pixels:
px = x / 1000 * image_width
py = y / 1000 * image_height
Both <point>X Y</point> and <point x1="X" y1="Y" alt="..."> forms occur, so parse for either.
Quantization details
| Method | RTN (round-to-nearest), no calibration |
| Bits | 8 uniform |
| Group size | 64 |
| Mode | affine |
The vision tower is not quantized. All 333 vision tensors stay at bfloat16 in every build here โ mlx-vlm's
skip_multimodal_module() excludes them. This mirrors how GGUF conversions ship the vision encoder as a
separate f16 mmproj. Only the language model is quantized, which is why a "4-bit" build measures 5.06
effective bits per weight rather than 4.
What I actually measured
Quantization is free on this task. The unquantized bf16 model scores 30/50 on this harness. So do the 8-bit, 6-bit and 4-bit builds โ not merely the same percentage, the same count. Going from 18 GB to 5.6 GB cost nothing measurable in grounding accuracy.
These 60% figures are lower than the model's real ability, and that is my harness, not the quantization.
inclusionAI report 73.0% on ScreenSpot-Pro for this model. My setup differs from theirs in three ways that
each cost accuracy: I prompt with a generic Click on: <instruction> rather than the official grounding prompt
(their card specifies disabling reasoning and using temperature 0), I cap images at 2048 px on a benchmark
built around ~30 px targets on 4K screens, and I sample 50 items from a single shard. Since bf16 lands at the
same 60%, none of that gap is attributable to quantization โ use their 73% as the model's grounding ability and
these numbers only to compare builds against each other.
AWQ did not measurably beat plain RTN. Worth stating plainly, since calibrated quants are usually assumed
to win. Partly this is coverage โ only 8 of 32 layers received AWQ scaling (layers: 8, groups: 24), the other
24 being linear_attention (Gated DeltaNet) layers its scale search does not handle. But mostly there was
simply nothing to recover: plain 4-bit already matches bf16 here.
Structured description degrades somewhat at 4 bits. Asked to list the buttons in a screenshot, plain 4bit
sometimes emits coordinate tags (<ex>10 110</ex>...) instead of prose, where the higher-bit builds and
4bit-awq returned a proper list. That was a handful of prompts โ suggestive, not conclusive. Ordinary
description ("what application is this?", "summarize this screen") worked on every build. This, rather than
grounding, is the reason to consider a higher-bit build.
Picking one
- Grounding/clicking only โ any build, including
4bit. It matched bf16. - You also want description/reasoning about the screen โ
8bit, or6bitto save 2 GB. 4bit-awqis included for completeness and reproducibility, not because it won.
License and attribution
The upstream repo inclusionAI/UI-Venus-2-9B declares no
license โ no license tag, no cardData.license, no LICENSE file. None is asserted here. Check with
inclusionAI before using these weights in a product. (Some third-party conversions label this Apache-2.0; I
could not find a basis for that in the source repo.)
Credit for the model goes to inclusionAI (technical report, project page), derived in turn from Qwen/Qwen3.5-9B. This repo contains only a quantized conversion.
- Downloads last month
- 49
8-bit