Mage-VL-Cowork for Apple Silicon (MLX, 4-bit)

An MLX conversion of microsoft/Mage-VL-Cowork (revision 6fbe7f9), Microsoft's 4B hybrid computer-use model: a Mage-ViT vision encoder and a Qwen3-4B language model trained to operate a desktop through two tools in one policy, computer_use (mouse and keyboard on screenshots) and bash (a persistent shell). Microsoft reports 66.0% on OSWorld-Verified, the best result at the 4B scale. This build quantises the language model to 4-bit (affine, group 64) and keeps the vision tower in bf16. Size: 3.1 GB.

Published by NetProjects (Internet Projects Ltd). Not affiliated with Microsoft or Alibaba Cloud.

Build Size Same next token as the PyTorch fp32 reference
netprojects/Mage-VL-Cowork-MLX-6bit (recommended) 4.2 GB 96.5%
netprojects/Mage-VL-Cowork-MLX-8bit 5.3 GB 98.2%
netprojects/Mage-VL-Cowork-MLX-4bit 3.1 GB 89.3%
netprojects/Mage-VL-Cowork-MLX-bf16 9.5 GB 98.6%

The 6-bit build is the recommended balance of size and fidelity; the 8-bit and bf16 builds track Microsoft's PyTorch model most closely. The 4-bit build is the smallest; it agrees with the reference on 89.3% of tokens, so expect more drift in its reasoning and coordinates.

Run it

pip install -U "mlx-vlm>=0.7.11"

mlx-vlm 0.7.11 or newer has the mage_vl model built in, so no remote code is needed.

mlx_vlm.generate --model netprojects/Mage-VL-Cowork-MLX-4bit --max-tokens 1024 \
  --image screenshot.png --prompt "Turn on Dark Mode."

For agent use, send Microsoft's exact system prompt (MAGE_HYBRID_SYSTEM_PROMPT in the original model card, also in its osworld_eval/) and the first-step user turn format from Microsoft's evaluation harness: the screenshot, then Please generate the next move according to the UI screenshot, instruction and previous actions. with the instruction and previous actions. The model answers with a <think> block, an Action: line and one or more <tool_call> blocks. As a local endpoint for an agent harness:

mlx_vlm.server --model netprojects/Mage-VL-Cowork-MLX-4bit --port 8100

Coordinates. computer_use coordinates are normalised to [0, 999] on both axes, whatever the screen size. Map them to pixels as Microsoft's agent does: x_px = round(x / 999 * (width - 1)), and the same for y with the height. Screenshots up to 4,000,000 pixels are used at full resolution (the processor setting in this repo).

What was checked

On a MacBook Pro M5 Max (128 GB), 17 November 2026, with mlx 0.32.4 and mlx-vlm 0.7.11, against Microsoft's own PyTorch code (transformers 4.57.3, fp32 on CPU) at the same pinned revision.

  • Exact conversion. Only tensor names and config fields were mapped to mlx-vlm's mage_vl layout; every tensor is copied unchanged. On a small random-weight model built from Microsoft's code, MLX and PyTorch logits agree to within 1e-6 on CPU.

  • Parity with the PyTorch reference. 16 agent first steps with screenshots (synthetic desktop screens and ScreenSpot-v2 screenshots) and Microsoft's system prompts, 3,489 teacher-forced tokens on identical inputs:

    Build Same next token as reference Mean NLL (reference 0.413)
    bf16 98.6% 0.414
    8-bit 98.2% 0.414
    6-bit 96.5% 0.418
    4-bit 89.3% 0.469
  • Processor. The torch-free mlx-vlm processor produces the same input tokens as Microsoft's processor on all 16 prompts.

  • Tool calls. In every build the original chat template round-trips computer_use and bash calls and tool responses. On 8 first-step prompts with Microsoft's hybrid system prompt, 7 answers per build are well-formed <tool_call> JSON with valid actions and [0, 999] coordinates (the eighth was still reasoning at our 320-token test limit).

  • Source files. All weight files of the upstream revision match the Hub's SHA-256 hashes, and the chat template is the original one, byte for byte.

Use it in a sandbox

Mage-VL-Cowork is an autonomous agent that can run arbitrary shell commands and GUI actions. Follow Microsoft's guidance from the original model card:

  • Sandbox isolation. Run the agent inside an isolated container or VM with no access to host filesystems, local networks or personal environment variables.
  • Scoped authorization and human in the loop. Require explicit human confirmation before irreversible actions such as external network submissions, account authentication, financial transactions or privilege changes.
  • No credential exposure. Do not mount personal SSH keys, cloud tokens, browser cookies or password stores in the agent's workspace unless explicitly required and scoped to the sandbox.
  • Network allow-listing. Restrict outbound network access when the agent reads untrusted web content, to mitigate prompt injection hidden in web pages or documents.

License

Apache-2.0, as the original model and Qwen3-4B. LICENSE is the Apache-2.0 text and NOTICE describes the conversion and credits Microsoft and Alibaba Cloud (Qwen). Please cite Microsoft's paper if you use the model; the citation is in the original model card.

More MLX builds from NetProjects: huggingface.co/netprojects ยท netprojects.com

Downloads last month
41
Safetensors
Model size
5B params
Tensor type
U32
ยท
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for netprojects/Mage-VL-Cowork-MLX-4bit

Quantized
(12)
this model

Collection including netprojects/Mage-VL-Cowork-MLX-4bit