mlx-community/Fara1.5-9B-8bit

microsoft/Fara1.5-9B converted to MLX and quantized to 8-bit, for inference on Apple Silicon.

Fara1.5 is a computer-use / web-agent vision-language model built on Qwen3.5 โ€” it reads screenshots and acts on interfaces. The vision path is preserved in this conversion, because for this model class it is the point.

Quantization

Requested bits 8
Group size 64
Mode affine
Effective bits per weight 8.86
On-disk size 9.8 GB
Shards 2

Effective bits exceed the requested value because mlx-vlm quantizes only the language model and leaves the vision tower in bf16 by design โ€” 333 vision tensors, none of them quantized. The vision encoder is a small share of the weights but disproportionately sensitive to quantization error.

language_model : quantized (8-bit, group size 64)
vision_tower   : 333 tensors, bf16    <- unquantized

Measured results

All figures measured against the unquantized bf16 source, greedy decoding, on an M2 Pro / 32 GB. At 9B the bf16 control fits in memory, so this is a behavioural comparison, not a weight-level proxy.

Metric bf16 (source) 8-bit
Perplexity 3.2339 3.2150
Perplexity ratio 1.000 0.994
Top-1 agreement โ€” 1.000
KL(bf16 โ€– quant) 0 0.000593 nats/tok
Task accuracy 8/8 8/8
Decode tok/s 10.2 19.7
Peak RAM 18.97 GB 11.91 GB

Top-1 agreement is 1.000 over 204 teacher-forced tokens โ€” the 8-bit model picked the same argmax token as bf16 on every one. KL divergence is 5.9e-4 nats/token. At this fidelity, quantization is not the limiting factor in output quality.

At 8-bit the model runs 1.9x faster using 1.6x less memory than bf16.

Generation agreement (reported, but not a quality metric)

Metric vs bf16
BLEU 62.05
chrF 66.9
ROUGE-1 / ROUGE-L 0.736 / 0.731
Exact match 4/6

BLEU against bf16 is not a quality measure. It treats bf16 output as ground truth, so it penalises valid paraphrase. The divergences here are exactly that โ€” both models answer correctly, then differ in trailing filler:

prompt: "Name the capital of Japan in one word."
bf16 : "Tokyo ... I will answer the question directly without any extra content."
8-bit: "Tokyo ... The user asked for the capital of Japan in one word."

Same answer, different tail. That is why task accuracy is measured separately โ€” and it is 8/8 for both.

Vision path

Verified working, not merely present. Given a synthetic UI screenshot with two buttons and a total, the 8-bit model returned:

"two large rectangular buttons side by side: On the left: a "SubmitOrder" button. On the right: a "Cancel" button. Below them, the total amount displayed is 42.50 USD."

Both button labels and the amount read correctly.

What was not measured

No standard benchmarks โ€” no MMLU, GSM8K, ScreenSpot, WebArena, or any agentic evaluation. The accuracy layer is 8 short verifiable prompts, not a benchmark. No blind LLM-judge quality grading was run for this variant. The vision path was verified for correctness on a single synthetic screenshot, not scored on a dataset. If your use case is the full computer-use loop, evaluate on your own tasks.

Usage

pip install mlx-vlm
from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("mlx-community/Fara1.5-9B-8bit")

prompt = apply_chat_template(
    processor, model.config,
    "Describe this screenshot. What buttons do you see?",
    num_images=1,
)
out = generate(model, processor, prompt, image=["screenshot.png"], max_tokens=256)
print(out.text)

Text-only works too โ€” pass num_images=0 and omit image.

Note that stock mlx-lm loads the text path only; use mlx-vlm for image input.

Credits

All credit for the model belongs to Microsoft. This is a format conversion and quantization; no training or fine-tuning was performed. Licensed MIT, as the original. See the original card for intended use and limitations.

Downloads last month
57
Safetensors
Model size
9B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

8-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for mlx-community/Fara1.5-9B-8bit

Finetuned
Qwen/Qwen3.5-9B
Quantized
(11)
this model