Instructions to use nace-ai/Qwen3.8-Flash-Next-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use nace-ai/Qwen3.8-Flash-Next-MLX with MLX:
# Make sure mlx-vlm is installed # pip install --upgrade mlx-vlm from mlx_vlm import load, generate from mlx_vlm.prompt_utils import apply_chat_template from mlx_vlm.utils import load_config # Load the model model, processor = load("nace-ai/Qwen3.8-Flash-Next-MLX") config = load_config("nace-ai/Qwen3.8-Flash-Next-MLX") # Prepare input image = ["http://images.cocodataset.org/val2017/000000039769.jpg"] prompt = "Describe this image." # Apply chat template formatted_prompt = apply_chat_template( processor, config, prompt, num_images=1 ) # Generate output output = generate(model, processor, formatted_prompt, image) print(output) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use nace-ai/Qwen3.8-Flash-Next-MLX with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Qwen3.8-Flash-Next-MLX"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "nace-ai/Qwen3.8-Flash-Next-MLX" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Hermes Agent
How to use nace-ai/Qwen3.8-Flash-Next-MLX with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Qwen3.8-Flash-Next-MLX"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default nace-ai/Qwen3.8-Flash-Next-MLX
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use nace-ai/Qwen3.8-Flash-Next-MLX with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "nace-ai/Qwen3.8-Flash-Next-MLX"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "nace-ai/Qwen3.8-Flash-Next-MLX" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Qwen3.8-Flash-Next-MLX
Q4: KL 0.067 to the original BF16 model, against 0.249 for mlx-community 4-bit, with 5.4 GB less language-model weight (105.20 vs 110.62 GB).
Orange diamonds: Nace Q4, in this repo. Grey markers: public MLX packs. Lower = closer to BF16.
Checkpoints
| Bits | Download | KL to BF16 lower is closer | Fits | Kapitsa |
|---|---|---|---|---|
4-bit ยท Q4/ | 107.6 GB | 0.067 | 192 GB+ | not yet |
- Q4: 4-bit in groups of 64, with 696 modules at 8-bit (groups of 64 or 128).
Fits is a rule of thumb, not a measurement: the smallest RAM size whose default GPU share on macOS holds the download plus 2 GB for a conversation. Kapitsa skips parts it doesn't use and may fit one size smaller; kapitsa doctor checks a given Mac.
The repo root holds the same files as Q4/, so links to this repo and its earlier names keep working. To download Q4/:
hf download nace-ai/Qwen3.8-Flash-Next-MLX --include "Q4/*" \
--local-dir Qwen3.8-Flash-Next-MLX
Quick start
This is an MLX pack for Apple silicon Macs, 107.6 GB of files. mlx-lm 0.32.2 cannot load this architecture yet, and other MLX apps were not tested with it. The model thinks before it answers by default; each block below shows how to turn that off.
mlx-vlm
The release check loaded Q4/ with mlx-vlm 0.7.8 and MLX 0.32.4 on CUDA (NVIDIA H200) and ran this code, which answered "Hello!". It has not been run on a Mac. Download Q4/ with the command above first.
pip install mlx-vlm==0.7.8
from mlx_vlm import load, generate
model, processor = load("Qwen3.8-Flash-Next-MLX/Q4")
tok = getattr(processor, "tokenizer", processor)
messages = [{"role": "user",
"content": "Say hello in one short sentence."}]
prompt = tok.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True,
enable_thinking=False, # thinking off
)
out = generate(model, processor, prompt=prompt,
max_tokens=80, temperature=0.0)
print(out.text)
Kapitsa
Kapitsa is Nace's engine for Qwen models on the Mac. It serves a chat page and OpenAI, Anthropic and Ollama APIs.
Kapitsa 0.2.36 serves this repo's earlier revision, 8fda1047, not the files in Q4/. Kapitsa support for Q4/ comes in a later release.
brew install nace-ai/tap/kapitsa
kapitsa pull flash:q4
kapitsa serve --model flash:q4 --private
Open http://127.0.0.1:8000 to chat, or call the API. Thinking is on by default and comes back in reasoning_content:
curl -s http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [{"role": "user", "content": "What is 17 * 23?"}],
"temperature": 1.0, "top_p": 0.95, "top_k": 20, "presence_penalty": 0.0
}'
Thinking off: add "reasoning_effort": "none" (or "chat_template_kwargs": {"enable_thinking": false}) and use the non-thinking settings:
curl -s http://127.0.0.1:8000/v1/chat/completions -H "Content-Type: application/json" -d '{
"messages": [{"role": "user", "content": "What is 17 * 23?"}],
"reasoning_effort": "none",
"temperature": 0.7, "top_p": 0.8, "top_k": 20, "presence_penalty": 1.5
}'
Thinking depth: "reasoning_effort" takes low, medium, xhigh.
Tool calling
Kapitsa takes OpenAI tool definitions. A full round trip with the OpenAI SDK against kapitsa serve:
import json
from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="none")
MODEL = "nace-ai/Qwen3.8-Flash-Next-MLX"
tools = [{"type": "function", "function": {
"name": "get_weather",
"description": "Current weather for a city",
"parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]},
}}]
messages = [{"role": "user", "content": "What is the weather in Almaty right now?"}]
reply = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
call = reply.choices[0].message.tool_calls[0]
print("call:", call.function.name, call.function.arguments)
messages.append(reply.choices[0].message)
messages.append({"role": "tool", "tool_call_id": call.id,
"content": json.dumps({"city": "Almaty", "temp_c": 4, "sky": "clear"})})
final = client.chat.completions.create(model=MODEL, messages=messages, tools=tools)
print("answer:", final.choices[0].message.content)
Recommended settings
From the Qwen/Qwen3.8-Flash-Next card:
| Mode | temperature | top_p | top_k | min_p | presence_penalty |
|---|---|---|---|---|---|
| Thinking | 1.0 | 0.95 | 20 | 0.0 | 0.0 |
| Thinking off | 0.7 | 0.8 | 20 | 0.0 | 1.5 |
Keep repetition_penalty at 1.0. Give answers room: up to 32,768 output tokens for most tasks with thinking on. A presence_penalty between 0 and 2 curbs endless repetition; high values can mix languages.
About this pack
Qwen3.8-Flash-Next in MLX Q4 from Nace.AI, closer to the original than nace-ai plain 4-bit (Kapitsa default) (KL 0.067 vs 0.141).
Measurements
Original: WikiText-2, GSM8K chats and HumanEval code; Fresh: frozen UltraChat test_sft and Schema-Guided Dialogue test dialogues. Each corpus scores 32,704 next-token positions offline with mlx-vlm 0.7.8 against bf16; KL is not task accuracy or speed.
Decimal GB; base-LM payload only; separate MTP, drafter and vision payloads excluded. Means are shown without uncertainty bars.
Raw measurements and runtime checks: results.json.
Exact values, payload bytes and revisions
| Pack | Base-LM GB | Original KL | Fresh KL | Pin |
|---|---|---|---|---|
| Nace [1] | 105.204 | 0.06660683 | 0.07132136 | this revision |
| nace-ai plain 4-bit (Kapitsa default) [2] | 103.424 | 0.14110035 | 0.14112859 | 5d423f9f |
| mlx-community 4-bit [3] | 110.621 | 0.24928604 | 0.22910747 | 07b5dc6c |
| TensorFold 4-bit [4] | 110.680 | 0.24921756 | 0.22900716 | 4fc403bc |
| TensorFold oQ4-MTP [5] | 110.794 | 0.22957656 | 0.21622438 | 069090c3 |
Decimal GB; base-LM payload only; separate MTP, drafter and vision payloads excluded. Exact bytes by row: 1: 105,203,519,000 B; 2: 103,424,216,600 B; 3: 110,621,031,960 B; 4: 110,679,960,600 B; 5: 110,793,555,480 B.
The reference is Qwen/Qwen3.8-Flash-Next at de4b8e4d. Vision was not evaluated.
Speed has not been measured for this revision.
Download and memory
The download is 100.2 GiB (107.6 GB). On a 48 GB Mac, kapitsa doctor (0.2.36) plans 76.0 GB of GPU memory for Q4/ with an 8K-token conversation, 73.2 GB of it weights, against a 38.6 GB budget, so it does not fit.
Tool calls
With thinking off, 20 of 20 requests made a valid first tool call and 0 of 20 made an extra tool call instead of a final answer (mlx-vlm 0.7.8 with MLX 0.32.4 on CUDA, NVIDIA H200).
Earlier revision
The previous revision of this repo, 8fda1047, is the one Kapitsa 0.2.36 serves (the Kapitsa block above). Its speed and tool-call measurements stay in the repo history.
Credits and license
The Qwen Team at Alibaba Group designed and trained the original model. Nace AI quantized its published weights. This is not an official Qwen release.
The Qwen Community License 1.0 applies unchanged. Model-as-a-Service or AI Work Assistant businesses need a separate Qwen license for commercial use under condition 2.
- Downloads last month
- 1,576
4-bit
Model tree for nace-ai/Qwen3.8-Flash-Next-MLX
Base model
Qwen/Qwen3.8-Flash-Next