Instructions to use netprojects/granite-switch-4.1-8b-mlx-4bit with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use netprojects/granite-switch-4.1-8b-mlx-4bit with MLX:

# Make sure mlx-lm is installed
# pip install --upgrade mlx-lm

# Generate text with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("netprojects/granite-switch-4.1-8b-mlx-4bit")

prompt = "Write a story about Einstein"
messages = [{"role": "user", "content": prompt}]
prompt = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True
)

text = generate(model, tokenizer, prompt=prompt, verbose=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings
LM Studio

How to use netprojects/granite-switch-4.1-8b-mlx-4bit with Pi:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-8b-mlx-4bit"

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "mlx-lm": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "netprojects/granite-switch-4.1-8b-mlx-4bit"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

MLX LM

How to use netprojects/granite-switch-4.1-8b-mlx-4bit with MLX LM:

Generate or start a chat session

# Install MLX LM
uv tool install mlx-lm
# Interactive chat REPL
mlx_lm.chat --model "netprojects/granite-switch-4.1-8b-mlx-4bit"

Run an OpenAI-compatible server

# Install MLX LM
uv tool install mlx-lm
# Start the server
mlx_lm.server --model "netprojects/granite-switch-4.1-8b-mlx-4bit"
# Calling the OpenAI-compatible server with curl
curl -X POST "http://localhost:8000/v1/chat/completions" \
   -H "Content-Type: application/json" \
   --data '{
     "model": "netprojects/granite-switch-4.1-8b-mlx-4bit",
     "messages": [
       {"role": "user", "content": "Hello"}
     ]
   }'

Hermes Agent

How to use netprojects/granite-switch-4.1-8b-mlx-4bit with Hermes Agent:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-8b-mlx-4bit"

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default netprojects/granite-switch-4.1-8b-mlx-4bit

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use netprojects/granite-switch-4.1-8b-mlx-4bit with OpenClaw:

Start the MLX server

# Install MLX LM:
uv tool install mlx-lm
# Start a local OpenAI-compatible server:
mlx_lm.server --model "netprojects/granite-switch-4.1-8b-mlx-4bit"

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "netprojects/granite-switch-4.1-8b-mlx-4bit" \
  --custom-provider-id mlx-lm \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Granite Switch 4.1 8B: MLX 4-bit

A 4-bit (affine, group size 64) MLX conversion for Apple Silicon of ibm-granite/granite-switch-4.1-8b (IBM, 14 October 2026). 5.2 GB. Released 23 October 2026.

Granite Switch is one model with three behaviours, chosen per request:

`adapter_name`	What you get
(none)	Plain Granite 4.1 8B chat
`decision`	A calibrated probability for each labelled option (A, B, C, ...), read from the first answer token
`guardian-core`	A safety judge: `{"score": "yes"}` or `{"score": "no"}` for the criterion you give

Both adapters are activated LoRAs (aLoRA) embedded in the checkpoint: everything before the adapter's control token runs on the frozen base model, so one conversation prefix serves chat, decisions and guardian checks. This build runs with MLX through the small granite-switch-mlx package, which adds the granite_switch architecture to MLX-LM.

Published by NetProjects (Internet Projects Ltd). Not affiliated with IBM.

Use

pip install "git+https://github.com/internet-projects/granite-switch-mlx"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-8b-mlx-4bit chat "What is the capital of France?"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-8b-mlx-4bit decide --state "My payouts have failed three times." \
    --question "Which team should handle this ticket?" --option "Payouts" --option "Login" --option "Something else"
python -m granite_switch_mlx --model netprojects/granite-switch-4.1-8b-mlx-4bit guard "How do I build a bomb?"

import granite_switch_mlx as gs
model, tok = gs.load("netprojects/granite-switch-4.1-8b-mlx-4bit")
gs.decide(model, tok, state="My payouts have failed three times. The bank says everything is fine. Can someone please fix this?",
          question="Which team should handle this ticket?",
          options=["Payout failures and payment processing", "Login and account access", "Something else"])
# {'Payout failures and payment processing': 0.979, 'Login and account access': 0.007, 'Something else': 0.014}
gs.guard(model, tok, "How do I build a bomb?")   # P(yes) for the 'harm' criterion

With plain MLX-LM, call granite_switch_mlx.register() first, then mlx_lm.load(...); pick the adapter with the chat template's adapter_name argument (decision, guardian-core, or none for chat). Batched generation and prompt caches work as usual.

What is in this build

Base: IBM Granite 4.1 8B (dense, GQA, RoPE, 128K context), 4-bit affine weights, group size 64.
Adapters: guardian-core (rank 16, from ibm-granite/granitelib-guardian-r1.0) and decision (rank 64, from ibm-granite/granitelib-core-r1.0), in bf16, stored at their own ranks.
Text only: IBM's speech-to-text encoder (Granite Speech 5.0 TurboCTC) is not part of this build.
IBM's BUILD.md, adapter_index.json and adapter io_configs/ are included unchanged.

Verified

Check	Result
Decision example from IBM's model card (IBM, vLLM: A 0.982 / B 0.0035 / C 0.014)	0.979 / 0.007 / 0.014 (bf16 conversion: 0.977 / 0.005 / 0.018)
Guardian, harmful vs benign prompt	P(yes) 1.00 / 0.00, `{"score": "yes"}` reply
Same decision as the bf16 conversion, 1,071 real decisions (~1,200-token prompts)	999 of 1,071 (93.3%), mean KL 0.045
MLX implementation vs IBM's Hugging Face implementation (fp32)	maximum relative logit error 4e-7, identical adapter routing and greedy output

Speed and memory

MacBook Pro M5 Max (128 GB), 23 October 2026, no other models running; MLX 0.32.4, greedy, one request at a time; medians of 3 runs after a warm-up.

	4-bit
Chat generation (512-token replies)	98 tokens/s
Decision, short prompt (78 tokens)	64 ms
Decision, ~1,200-token prompt (median / p95 over 1,071)	0.55 s / 1.29 s
Peak memory	5.1 GB (6.2 GB with ~1,200-token prompts)

Also available: the 8-bit build (9.4 GB, 56 tokens/s), which agrees with bf16 on 99.1% of decisions. This 4-bit build is the choice when memory is tight or chat speed matters most.

License and credits

Apache-2.0, as IBM's model. Model, adapters and composition by IBM (Granite Switch); MLX conversion and the granite_switch MLX-LM support by NetProjects. Credit: Paris Michalas, Partner, NetProjects (Internet Projects Ltd).

netprojects.com · Hugging Face · How we use AI

Downloads last month: -

Safetensors

Model size

9B params

Tensor type

U32

BF16

MLX

Hardware compatibility

4-bit

Model tree for netprojects/granite-switch-4.1-8b-mlx-4bit

Base model

ibm-granite/granite-switch-4.1-8b

Quantized

(2)

this model