Instructions to use netprojects/Mage-VL-Cowork-MLX-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use netprojects/Mage-VL-Cowork-MLX-4bit with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("netprojects/Mage-VL-Cowork-MLX-4bit") config = load_config("netprojects/Mage-VL-Cowork-MLX-4bit") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use netprojects/Mage-VL-Cowork-MLX-4bit with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Mage-VL-Cowork-MLX-4bit"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "netprojects/Mage-VL-Cowork-MLX-4bit" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use netprojects/Mage-VL-Cowork-MLX-4bit with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Mage-VL-Cowork-MLX-4bit"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default netprojects/Mage-VL-Cowork-MLX-4bit
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use netprojects/Mage-VL-Cowork-MLX-4bit with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "netprojects/Mage-VL-Cowork-MLX-4bit"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "netprojects/Mage-VL-Cowork-MLX-4bit" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Mage-VL-Cowork for Apple Silicon (MLX, 4-bit)
An MLX conversion of microsoft/Mage-VL-Cowork (revision 6fbe7f9), Microsoft's 4B hybrid computer-use model:
a Mage-ViT vision encoder and a Qwen3-4B language model trained to operate a desktop through two tools in one policy,
computer_use (mouse and keyboard on screenshots) and bash (a persistent shell). Microsoft reports 66.0% on
OSWorld-Verified, the best result at the 4B scale. This build quantises the language model to 4-bit (affine, group 64) and keeps the vision tower in bf16. Size: 3.1 GB.
Published by NetProjects (Internet Projects Ltd). Not affiliated with Microsoft or Alibaba Cloud.
| Build | Size | Same next token as the PyTorch fp32 reference |
|---|---|---|
| netprojects/Mage-VL-Cowork-MLX-6bit (recommended) | 4.2 GB | 96.5% |
| netprojects/Mage-VL-Cowork-MLX-8bit | 5.3 GB | 98.2% |
| netprojects/Mage-VL-Cowork-MLX-4bit | 3.1 GB | 89.3% |
| netprojects/Mage-VL-Cowork-MLX-bf16 | 9.5 GB | 98.6% |
The 6-bit build is the recommended balance of size and fidelity; the 8-bit and bf16 builds track Microsoft's PyTorch model most closely. The 4-bit build is the smallest; it agrees with the reference on 89.3% of tokens, so expect more drift in its reasoning and coordinates.
Run it
pip install -U "mlx-vlm>=0.7.11"
mlx-vlm 0.7.11 or newer has the mage_vl model built in, so no remote code is needed.
mlx_vlm.generate --model netprojects/Mage-VL-Cowork-MLX-4bit --max-tokens 1024 \
--image screenshot.png --prompt "Turn on Dark Mode."
For agent use, send Microsoft's exact system prompt (MAGE_HYBRID_SYSTEM_PROMPT in the original model card,
also in its osworld_eval/) and the first-step user turn format from Microsoft's
evaluation harness: the screenshot, then
Please generate the next move according to the UI screenshot, instruction and previous actions. with the
instruction and previous actions. The model answers with a <think> block, an Action: line and one or more
<tool_call> blocks. As a local endpoint for an agent harness:
mlx_vlm.server --model netprojects/Mage-VL-Cowork-MLX-4bit --port 8100
Coordinates. computer_use coordinates are normalised to [0, 999] on both axes, whatever the screen size.
Map them to pixels as Microsoft's agent does: x_px = round(x / 999 * (width - 1)), and the same for y with the
height. Screenshots up to 4,000,000 pixels are used at full resolution (the processor setting in this repo).
What was checked
On a MacBook Pro M5 Max (128 GB), 17 November 2026, with mlx 0.32.4 and mlx-vlm 0.7.11, against Microsoft's own PyTorch code (transformers 4.57.3, fp32 on CPU) at the same pinned revision.
Exact conversion. Only tensor names and config fields were mapped to mlx-vlm's
mage_vllayout; every tensor is copied unchanged. On a small random-weight model built from Microsoft's code, MLX and PyTorch logits agree to within 1e-6 on CPU.Parity with the PyTorch reference. 16 agent first steps with screenshots (synthetic desktop screens and ScreenSpot-v2 screenshots) and Microsoft's system prompts, 3,489 teacher-forced tokens on identical inputs:
Build Same next token as reference Mean NLL (reference 0.413) bf16 98.6% 0.414 8-bit 98.2% 0.414 6-bit 96.5% 0.418 4-bit 89.3% 0.469 Processor. The torch-free mlx-vlm processor produces the same input tokens as Microsoft's processor on all 16 prompts.
Tool calls. In every build the original chat template round-trips
computer_useandbashcalls and tool responses. On 8 first-step prompts with Microsoft's hybrid system prompt, 7 answers per build are well-formed<tool_call>JSON with valid actions and[0, 999]coordinates (the eighth was still reasoning at our 320-token test limit).Source files. All weight files of the upstream revision match the Hub's SHA-256 hashes, and the chat template is the original one, byte for byte.
Use it in a sandbox
Mage-VL-Cowork is an autonomous agent that can run arbitrary shell commands and GUI actions. Follow Microsoft's guidance from the original model card:
- Sandbox isolation. Run the agent inside an isolated container or VM with no access to host filesystems, local networks or personal environment variables.
- Scoped authorization and human in the loop. Require explicit human confirmation before irreversible actions such as external network submissions, account authentication, financial transactions or privilege changes.
- No credential exposure. Do not mount personal SSH keys, cloud tokens, browser cookies or password stores in the agent's workspace unless explicitly required and scoped to the sandbox.
- Network allow-listing. Restrict outbound network access when the agent reads untrusted web content, to mitigate prompt injection hidden in web pages or documents.
License
Apache-2.0, as the original model and Qwen3-4B. LICENSE is the Apache-2.0 text and NOTICE describes the
conversion and credits Microsoft and Alibaba Cloud (Qwen). Please cite Microsoft's paper if you use the model; the
citation is in the original model card.
More MLX builds from NetProjects: huggingface.co/netprojects ยท netprojects.com
- Downloads last month
- 41
4-bit