Mage-VL-Cowork for Apple Silicon (MLX, bf16)

An MLX conversion of microsoft/Mage-VL-Cowork (revision 6fbe7f9), Microsoft's 4B hybrid computer-use model: a Mage-ViT vision encoder and a Qwen3-4B language model trained to operate a desktop through two tools in one policy, computer_use (mouse and keyboard on screenshots) and bash (a persistent shell). Microsoft reports 66.0% on OSWorld-Verified, the best result at the 4B scale. This build keeps the full bf16 weights. Size: 9.5 GB.

Published by NetProjects (Internet Projects Ltd). Not affiliated with Microsoft or Alibaba Cloud.

Build Size Same next token as the PyTorch fp32 reference
netprojects/Mage-VL-Cowork-MLX-6bit (recommended) 4.2 GB 96.5%
netprojects/Mage-VL-Cowork-MLX-8bit 5.3 GB 98.2%
netprojects/Mage-VL-Cowork-MLX-4bit 3.1 GB 89.3%
netprojects/Mage-VL-Cowork-MLX-bf16 9.5 GB 98.6%

The 6-bit build is the recommended balance of size and fidelity; the 8-bit and bf16 builds track Microsoft's PyTorch model most closely. The 4-bit build is the smallest; it agrees with the reference on 89.3% of tokens, so expect more drift in its reasoning and coordinates.

Run it

pip install -U "mlx-vlm>=0.7.11"

mlx-vlm 0.7.11 or newer has the mage_vl model built in, so no remote code is needed.

mlx_vlm.generate --model netprojects/Mage-VL-Cowork-MLX-bf16 --max-tokens 1024 \
  --image screenshot.png --prompt "Turn on Dark Mode."

For agent use, send Microsoft's exact system prompt (MAGE_HYBRID_SYSTEM_PROMPT in the original model card, also in its osworld_eval/) and the first-step user turn format from Microsoft's evaluation harness: the screenshot, then Please generate the next move according to the UI screenshot, instruction and previous actions. with the instruction and previous actions. The model answers with a <think> block, an Action: line and one or more <tool_call> blocks. As a local endpoint for an agent harness:

mlx_vlm.server --model netprojects/Mage-VL-Cowork-MLX-bf16 --port 8100

Coordinates. computer_use coordinates are normalised to [0, 999] on both axes, whatever the screen size. Map them to pixels as Microsoft's agent does: x_px = round(x / 999 * (width - 1)), and the same for y with the height. Screenshots up to 4,000,000 pixels are used at full resolution (the processor setting in this repo).

What was checked

On a MacBook Pro M5 Max (128 GB), 17 November 2026, with mlx 0.32.4 and mlx-vlm 0.7.11, against Microsoft's own PyTorch code (transformers 4.57.3, fp32 on CPU) at the same pinned revision.

  • Exact conversion. Only tensor names and config fields were mapped to mlx-vlm's mage_vl layout; every tensor is copied unchanged. On a small random-weight model built from Microsoft's code, MLX and PyTorch logits agree to within 1e-6 on CPU.

  • Parity with the PyTorch reference. 16 agent first steps with screenshots (synthetic desktop screens and ScreenSpot-v2 screenshots) and Microsoft's system prompts, 3,489 teacher-forced tokens on identical inputs:

    Build Same next token as reference Mean NLL (reference 0.413)
    bf16 98.6% 0.414
    8-bit 98.2% 0.414
    6-bit 96.5% 0.418
    4-bit 89.3% 0.469
  • Processor. The torch-free mlx-vlm processor produces the same input tokens as Microsoft's processor on all 16 prompts.

  • Tool calls. In every build the original chat template round-trips computer_use and bash calls and tool responses. On 8 first-step prompts with Microsoft's hybrid system prompt, 7 answers per build are well-formed <tool_call> JSON with valid actions and [0, 999] coordinates (the eighth was still reasoning at our 320-token test limit).

  • Source files. All weight files of the upstream revision match the Hub's SHA-256 hashes, and the chat template is the original one, byte for byte.

Use it in a sandbox

Mage-VL-Cowork is an autonomous agent that can run arbitrary shell commands and GUI actions. Follow Microsoft's guidance from the original model card:

  • Sandbox isolation. Run the agent inside an isolated container or VM with no access to host filesystems, local networks or personal environment variables.
  • Scoped authorization and human in the loop. Require explicit human confirmation before irreversible actions such as external network submissions, account authentication, financial transactions or privilege changes.
  • No credential exposure. Do not mount personal SSH keys, cloud tokens, browser cookies or password stores in the agent's workspace unless explicitly required and scoped to the sandbox.
  • Network allow-listing. Restrict outbound network access when the agent reads untrusted web content, to mitigate prompt injection hidden in web pages or documents.

License

Apache-2.0, as the original model and Qwen3-4B. LICENSE is the Apache-2.0 text and NOTICE describes the conversion and credits Microsoft and Alibaba Cloud (Qwen). Please cite Microsoft's paper if you use the model; the citation is in the original model card.

More MLX builds from NetProjects: huggingface.co/netprojects ยท netprojects.com

Downloads last month
34
Safetensors
Model size
5B params
Tensor type
BF16
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for netprojects/Mage-VL-Cowork-MLX-bf16

Quantized
(12)
this model

Collection including netprojects/Mage-VL-Cowork-MLX-bf16