Instructions to use AtomicChat/Nanbeige4.3-33B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use AtomicChat/Nanbeige4.3-33B-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Use Docker

docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use AtomicChat/Nanbeige4.3-33B-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AtomicChat/Nanbeige4.3-33B-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AtomicChat/Nanbeige4.3-33B-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Ollama
How to use AtomicChat/Nanbeige4.3-33B-GGUF with Ollama:
```
ollama run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
```
Unsloth Desktop

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use AtomicChat/Nanbeige4.3-33B-GGUF with Docker Model Runner:
```
docker model run hf.co/AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M
```

Lemonade

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Nanbeige4.3-33B-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use AtomicChat/Nanbeige4.3-33B-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use AtomicChat/Nanbeige4.3-33B-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "AtomicChat/Nanbeige4.3-33B-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Nanbeige4.3-33B · GGUF

GGUF quants of Nanbeige's Nanbeige4.3-33B, built from Nanbeige's original BF16 weights on release day.

Nanbeige4.3-33B is Nanbeige's 33B dense agent and reasoning model (Llama architecture, 512K context, thinking on by default), Apache-2.0.
Atomic Dynamic (AD) files with our own importance matrix, next to llama.cpp's plain presets, every one measured against the BF16 original on one stand.

Built on release day, 25 October. Q8_0, Q6_K, Q5_K_M and Q4_K_M are llama.cpp's own presets, made without an importance matrix. The AD- files use our own importance matrix, collected on the BF16 weights (--parse-special) over 8,000 chunks of 512 tokens of our calibration corpus. The corpus build was rendered for Qwen3.8-27B, whose chat format (ChatML) Nanbeige4.3 shares; it is imatrix/imatrix.gguf in the metrics repo.

Pick a file

KL divergence is how far the quantized model's predictions drift from the original BF16 weights. Lower is better, and zero means identical. top-1 is how often it picks the same next token the original would have picked. Every file was measured on the same stand: our neutral multilingual text, 24 chunks of 4,096 tokens, llama.cpp 36e36e6. Raw logs and the importance matrix are in the metrics repo.

File	Size on disk	Built	KL divergence	top-1
Nanbeige4.3-33B-Q8_0.gguf	35.2 GB	plain 8-bit	0.0032	98.0%
Nanbeige4.3-33B-AD-Q6_K.gguf	30.1 GB	Atomic Dynamic, our imatrix	0.0052	97.5%
Nanbeige4.3-33B-AD-Q6_K-Q5_K.gguf	27.9 GB	Atomic Dynamic, our imatrix	0.0102	96.1%
Nanbeige4.3-33B-Q6_K.gguf	27.2 GB	llama.cpp preset, no imatrix	0.0165	95.5%
Nanbeige4.3-33B-AD-Q5_K_M.gguf	24.7 GB	Atomic Dynamic, our imatrix	0.0213	94.7%
Nanbeige4.3-33B-Q5_K_M.gguf	23.5 GB	llama.cpp preset, no imatrix	0.0304	93.4%
Nanbeige4.3-33B-AD-Q5_K_M-Q4_K_M.gguf	22.6 GB	Atomic Dynamic, our imatrix	0.0332	93.1%
Nanbeige4.3-33B-AD-Q4_K_M.gguf	21.2 GB	Atomic Dynamic, our imatrix	0.0380	92.6%
Nanbeige4.3-33B-AD-IQ4_XS.gguf	20.3 GB	Atomic Dynamic, our imatrix	0.0462	92.0%
Nanbeige4.3-33B-Q4_K_M.gguf	20.1 GB	llama.cpp preset, no imatrix	0.0693	89.9%
Nanbeige4.3-33B-AD-IQ3_S.gguf	17.1 GB	Atomic Dynamic, our imatrix	0.1019	87.5%
Nanbeige4.3-33B-AD-IQ3_XXS.gguf	14.9 GB	Atomic Dynamic, our imatrix	0.1593	84.6%
Nanbeige4.3-33B-AD-IQ2_S.gguf	13.7 GB	Atomic Dynamic, our imatrix	0.2448	80.5%

This model is unusually sensitive to quantization. llama.cpp's plain Q6_K picks the original's next token only 95.5% of the time (most models reach 98% at 6 bits) and plain Q4_K_M 89.9%. At about the same size on disk, the AD- files keep much more: AD-IQ4_XS (20.3 GB) has a third less KL divergence than Q4_K_M (20.1 GB), and AD-Q6_K-Q5_K (27.9 GB) 39% less than Q6_K (27.2 GB). AD-Q6_K (30.1 GB) is within reach of Q8_0 (97.5% against 98.0%). Below 4 bits the model loses a lot: AD-IQ3_XXS and AD-IQ2_S change the next token often (top-1 below 85%); use them only if nothing larger fits.

AD- marks an Atomic Dynamic layout: the FFN of the first and last layers one step up, attention output and the head at q6_k or higher, the small key and value projections at q8_0. The rest of the name is the type of the FFN up projections. Compare files by size on disk, not by name.

Run

llama.cpp:

llama-server -hf AtomicChat/Nanbeige4.3-33B-GGUF:AD-Q4_K_M --jinja -c 32768

Thinking is on by default. To turn it off: --chat-template-kwargs '{"enable_thinking": false}'.

Nanbeige's recommended sampling: reasoning and chat --temp 0.6 --top-p 0.95; agentic and tool use --temp 1.0 --top-p 0.95.

Or download a file and open it in Atomic Chat.

License and credits

The weights are Nanbeige's, released under Apache-2.0 (LICENSE in this repo). Quantized by AtomicChat.

Downloads last month: -

GGUF

Model size

33B params

Architecture

llama

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

Model tree for AtomicChat/Nanbeige4.3-33B-GGUF

Base model

Nanbeige/Nanbeige4.3-33B-Base

Finetuned

Nanbeige/Nanbeige4.3-33B

Quantized

(23)

this model

Collection including AtomicChat/Nanbeige4.3-33B-GGUF

Nanbeige4.3

Collection

GGUF quants of Nanbeige4.3-33B by AtomicChat • 2 items • Updated about 2 hours ago