Paris99's picture
Qwen3.8-27B-opencode-adapted MLX 3-bit: weights incl. MTP, card, Apache-2.0 LICENSE and NOTICE
b208b32 verified
|
Raw History Blame Contribute Delete
8.08 kB
---
license: apache-2.0
base_model: harness-hub/Qwen3.8-27B-opencode-adapted
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: mlx
tags:
- mlx
- apple-silicon
- agents
- coding-agent
- opencode
- lora
---
# Qwen3.8-27B-opencode-adapted for Apple Silicon (MLX, 3-bit)
[harness-hub/Qwen3.8-27B-opencode-adapted](https://huggingface.co/harness-hub/Qwen3.8-27B-opencode-adapted) (revision `b4171c0`) merged into
[Qwen/Qwen3.8-27B](https://huggingface.co/Qwen/Qwen3.8-27B) (revision `3756181`) and converted to MLX with mlx-vlm, vision
encoder and Qwen's MTP layer included. In this build most linear layers are quantized to 3-bit (affine, group 64); the token embedding, the output layer and the more sensitive v_proj / down_proj layers (the first and last eight layers and every third layer in between) are 4-bit, about 3.9 bits per weight overall. Size: 13.8 GB.
harness-hub trained this LoRA with reinforcement learning (AsyncGRPO in TRL) to be the model behind the
[OpenCode](https://opencode.ai) coding agent on software-engineering and terminal tasks. Their published results, under
OpenCode with thinking off: **SWE-bench Pro (hard 51) 35.3% → 51.0%** and **Terminal-Bench 2.1 46.4% → 52.4%** against
the base model, and with thinking off it matches the base model thinking at its highest effort while generating about
a sixth of the tokens. See the [original adapter card](https://huggingface.co/harness-hub/Qwen3.8-27B-opencode-adapted)
for the method, data and full results.
| Build | Size | Same next token as bf16 | KL vs bf16 |
|---|---|---|---|
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-6bit](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-6bit) (recommended) | 23.1 GB | 98.7% | 0.0014 |
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-8bit](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-8bit) | 30.0 GB | 99.2% | 0.0004 |
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-4bit](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-4bit) | 16.3 GB | 96.7% | 0.0203 |
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit) | 13.8 GB | 93.4% | 0.0812 |
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-bf16](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-bf16) | 55.6 GB | reference | reference |
| [netprojects/Qwen3.8-27B-opencode-adapted-MLX-adapter](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-adapter) | 638 MB | LoRA only, for mlx-lm | on any Qwen3.8-27B MLX build |
The 6-bit build is the recommended balance of size and fidelity; 8-bit is closest to bf16; 4-bit and 3-bit are for Macs with less memory (3-bit fits 24 GB Macs).
## Use with OpenCode
The adapter was trained and evaluated with **thinking off**, so keep it off. Serve the model with mlx-lm's
OpenAI-compatible server:
```bash
pip install -U "mlx-lm>=0.32.2"
mlx_lm.server --model netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit --port 8000 \
--chat-template-args '{"enable_thinking": false}'
```
Add it to `~/.config/opencode/opencode.json`:
```json
{
"$schema": "https://opencode.ai/config.json",
"provider": {
"local": {
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "http://localhost:8000/v1", "apiKey": "dummy" },
"models": { "netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit": {} }
}
}
}
```
```bash
opencode run --model local/netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit "Find out why the tests in tests/test_parser.py fail and fix the bug."
```
## Use from Python
Text only, with mlx-lm:
```python
from mlx_lm import load, generate
model, tokenizer = load("netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit")
messages = [{"role": "user", "content": "Write a Python function that parses '1.5M' into bytes."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True, enable_thinking=False, tokenize=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=512))
```
With images, with mlx-vlm:
```bash
pip install -U "mlx-vlm>=0.7.11"
mlx_vlm.generate --model netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit --image screenshot.png \
--prompt "What error does this terminal show?" --max-tokens 300 # thinking stays off unless --enable-thinking
```
The tokenizer and chat template are harness-hub's files, unchanged (same vocabulary and tool-call format as Qwen3.8-27B).
Tool calls use Qwen's XML format (`<tool_call><function=...><parameter=...>`), which mlx-lm's server returns as
OpenAI `tool_calls`.
## Faster decoding with MTP
This build keeps Qwen3.8-27B's own multi-token-prediction (MTP) layer in `model-mtp.safetensors`, quantized to 8-bit and
declared in `config.json` as `mtplx_mtp_quantization`. The adapter does not change that layer. mlx-vlm uses it as a built-in
draft model when you pass the same repository as `--draft-model`:
```bash
mlx_vlm.generate --model netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit --draft-model netprojects/Qwen3.8-27B-opencode-adapted-MLX-3bit \
--draft-kind mtp --temperature 0 --max-tokens 512 --prompt "Write a Python function that checks whether a number is prime."
```
Every drafted token is checked by the full model. With greedy decoding the text matches plain decoding until a near-tie,
where the batched check can round the other way, so long answers can diverge slightly.
## What was checked
On a MacBook Pro M5 Max (128 GB), 19 November 2026, with mlx 0.32.4, mlx-lm 0.32.2 and mlx-vlm 0.7.11. The build was made from
[netprojects/Qwen3.8-27B-opencode-adapted-MLX-bf16](https://huggingface.co/netprojects/Qwen3.8-27B-opencode-adapted-MLX-bf16) (every file checked against the Hub's SHA-256 hashes).
- **Quantization.** The build is checked to be exactly the affine (group 64) quantization of the bf16 build, with the bit widths described above.
- **Agreement with bf16** (12 short prompts, 9 with thinking off and 3 with thinking on, 949 teacher-forced positions): same next token as bf16 on 93.4% of positions (mean KL 0.0812).
- **Tool calling and coding** (thinking off, the included chat template): one agent-style prompt with `bash` and `read` tools produced a well-formed tool call; three short Python tasks run against asserts: 3/3.
- **GSM8K, 50 fixed test questions** (thinking off, greedy, 512-token cap): 45/50, against 47/50 for the bf16 build.
- **Thinking** (two short maths prompts): the reply closes `</think>` once and the answer is right: 2/2.
- **Memory.** Peak 14.6 GB with an 8K-token prompt (mlx-lm, text), so a Mac with 24 GB+ of memory runs it with the default GPU memory limit (about two thirds of RAM up to 36 GB). On a Mac just below that you can raise the limit until the next restart (`sudo sysctl iogpu.wired_limit_mb=...`), leaving several GB for macOS, and use short contexts.
- **Vision** (mlx-vlm): asked to name the shapes and colours in a test image, the build answered correctly and answered a text-only question in the same session.
- **MTP layer.** The bundled MTP tensors equal Qwen3.8-27B's `mtp.*` weights quantized to 8-bit; the build loads in mlx-lm and mlx-vlm with the extra file present and decodes with its bundled MTP head.
## Credits
The adapter is by harness-hub; the base model is Qwen3.8-27B by the Qwen Team, Alibaba Cloud. If you use the model,
please cite their work (see the [adapter card](https://huggingface.co/harness-hub/Qwen3.8-27B-opencode-adapted) and the
[Qwen3.8-27B card](https://huggingface.co/Qwen/Qwen3.8-27B)).
## License
Apache-2.0, as Qwen3.8-27B and the harness-hub adapter. `LICENSE` is the Apache-2.0 text from the Qwen3.8-27B repository
and `NOTICE` describes the conversion and credits harness-hub and the Qwen Team (Alibaba Cloud).
Published by [NetProjects](https://netprojects.com) (Internet Projects Ltd). Not affiliated with Hugging Face, harness-hub or Alibaba.
More MLX builds from NetProjects: [huggingface.co/netprojects](https://huggingface.co/netprojects) · [netprojects.com](https://netprojects.com)