mlx-community/Arete-27B-OptiQ-4bit

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs

OptiQ mixed-precision quant of NousResearch/Arete-27B, Nous Research's supervised fine-tune of Qwen3.8-27B. Nous trained it to answer politically sensitive questions factually instead of refusing or deflecting. It is a Qwen3.8-family vision-language model. 20 GB on disk.

What it is

Property	Value
Base	NousResearch/Arete-27B (Qwen3.8, 27B)
Method	OptiQ mixed-precision, per-layer 4/8-bit
Bit allocation	Reused from the Qwen3.8-27B OptiQ recipe: the architecture is identical, so the per-layer sensitivity ranking transfers directly and no per-model sweep is needed
Layer split	237 components at 4-bit, 261 at 8-bit
Group size	64
On disk	20 GB
Vision	bf16 vision tower kept in optiq_vision.safetensors for image input
KV cache	Mixed-precision KV config in kv_config.json
MTP	None: the Arete release does not include the speculation head

Following the naming llama.cpp uses for its mixed quants, the "4bit" label denotes the family, not the weighted average.

Run it

pip install mlx-optiq

import optiq  # registers the arch + vision sidecar
from mlx_lm import load, generate

model, tok = load("mlx-community/Arete-27B-OptiQ-4bit")
prompt = tok.apply_chat_template(
    [{"role": "user", "content": "Explain mixed-precision quantization in two sentences."}],
    tokenize=False, add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=400))

For image input plus an OpenAI- and Anthropic-compatible endpoint with mixed-precision KV cache:

optiq serve --model mlx-community/Arete-27B-OptiQ-4bit

This is a reasoning model, so give it a generous token budget.

Links

• Project website: mlx-optiq.com
• All OptiQ quants: mlx-optiq.com/models
• Base model: NousResearch/Arete-27B