Instructions to use AtomicChat/Maxwell-V1-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use AtomicChat/Maxwell-V1-Flash-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Use Docker
docker model run hf.co/AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use AtomicChat/Maxwell-V1-Flash-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "AtomicChat/Maxwell-V1-Flash-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "AtomicChat/Maxwell-V1-Flash-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
- Ollama
How to use AtomicChat/Maxwell-V1-Flash-GGUF with Ollama:
ollama run hf.co/AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use AtomicChat/Maxwell-V1-Flash-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use AtomicChat/Maxwell-V1-Flash-GGUF with Docker Model Runner:
docker model run hf.co/AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
- Lemonade
How to use AtomicChat/Maxwell-V1-Flash-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Maxwell-V1-Flash-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use AtomicChat/Maxwell-V1-Flash-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use AtomicChat/Maxwell-V1-Flash-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "AtomicChat/Maxwell-V1-Flash-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
How to Run Maxwell-V1.0-Flash Locally
The first GGUF of Maxwell-V1.0-Flash. We wrote its llama.cpp support. Built from the Maxwell team's original BF16 weights.
- Built by the team behind Atomic Chat, the app that runs open-weight models locally and offline.
- Needs llama.cpp with Maxwell support: our port (
maxwell-moe-port.patchin this repo) on llama.cppd3c3e242c. Stock llama.cpp, LM Studio and Ollama do not load these files yet.
Maxwell-V1.0-Flash is a 35B-total / 3B-active Mixture-of-Experts model by the Maxwell team, pre-trained on ~26T multilingual tokens and post-trained for reasoning, search and agentic work, Apache-2.0. Its tokenizer is built for Thai, Lao, Myanmar, Khmer and Shan as much as for CJK and Latin text. This repo has GGUF files of it for llama.cpp.
Pick a file
File names are Maxwell-V1-Flash-<name>.gguf; for the split BF16, point llama.cpp at the first part.
| file | GB on disk | bpw | suggested hardware | mean KLD | 99th pct KLD | same top token |
|---|---|---|---|---|---|---|
BF16 (2 parts, folder BF16/) |
72.0 | 16.02 | the original weights, bit-exact | 0 | 0 | 100% |
Q8_0 |
38.4 | 8.54 | 64 GB Macs, 48 GB of GPU memory | 0.0371 | 0.431 | 93.6% |
"Same top token" is how often the file picks the same next token as the BF16 original; "mean KLD" is the KL divergence of its next-token distribution from BF16's. Both are measured with llama-perplexity on our 30-language neutral corpus (114 chunks of 4096 tokens), llama.cpp d3c3e242c with the port, 2x H100 80 GB. Read them with the section below: for this model they include a rounding floor of about 0.03.
For comparison, llama.cpp's plain Q4_K_M preset without an importance matrix (21.9 GB) measures 0.281 mean KLD and 81.6% same top token. We do not publish it.
How precise this model is to run
Maxwell is unusually sensitive to rounding. The same BF16 weights give a different next-token distribution when llama.cpp multiplies them with bf16-rounded inputs than when it runs everything in F32: mean KLD 0.030, the same top token 93.2% of the time (full model, 3 chunks of 4096 tokens). The KV cache type makes no difference, and no single tensor group is the cause: keeping all attention, the head, the shared and dense FFNs in F32 brings it only to 0.023. Our reading is that small differences anywhere flip which of the 256 experts the router picks near its top-12 boundary, layer after layer. For scale: the vendor's transformers code in bf16 and our BF16 file differ by mean KLD 0.044 on the same tokens.
So the numbers in the table, measured against the BF16 file as for every AtomicChat release, include this floor: even Q8_0 reads 0.037. They still rank the files by how far quantization moves them.
Run it
Build llama.cpp at d3c3e242c with the port from this repo:
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout d3c3e242c16b8cbf51400fad410cd69b22d64f35
git apply /path/to/maxwell-moe-port.patch
cmake -B build -DGGML_CUDA=ON # Mac: drop -DGGML_CUDA=ON (Metal is on by default)
cmake --build build -j --config Release --target llama-server llama-cli
Then serve a file. The chat template (reasoning, tools) is embedded, so pass --jinja; the sampling below is the vendor's:
./build/bin/llama-server -m Maxwell-V1-Flash-AD-Q4_K-IQ4_XS.gguf --jinja -c 32768 -ngl 99 \
--temp 0.6 --top-p 0.95 --top-k 20
Thinking is on by default. To turn it off for a request, send "chat_template_kwargs": {"enable_thinking": false}, or start the server with --chat-template-kwargs '{"enable_thinking": false}'.
llama.cpp support
We wrote the maxwell-moe architecture for llama.cpp:
- 45 blocks: three dense SwiGLU blocks, then 42 blocks with 256 routed experts (12 active, sigmoid scores with a selection bias) plus one shared expert.
- GQA with 32 query and 4 KV heads, head size 128, per-head RMSNorm on Q and K, RoPE.
- Manifold-constrained hyper-connections: four residual streams mixed by Sinkhorn-normalized matrices, collapsed by a learned head (the arithmetic of DeepSeek-V4's mHC in llama.cpp).
- Its own pre-tokenizer: separate splits for newlines, Han, and Thai/Lao/Myanmar/Khmer runs before a GPT-4o-style word split. It matches the original tokenizer on all 5,599 texts we compared (30 languages, code, edge cases, a code-point sweep).
What it does not do yet:
- Block-sparse attention. The original attends, per KV group, to 8 prefix and 8 local blocks of 128 tokens plus 16 blocks picked by a learned indexer. Up to 4,096 tokens that is every block, so llama.cpp's dense attention is exact there; the vendor's own vLLM build also attends densely below 16K tokens. Beyond 16K tokens these files attend to the whole context where the original attends to its selected blocks. The indexer weights are in the files for a later sparse path.
- MTP. The multi-token-prediction head is left out.
How we checked it against the Maxwell team's modeling_maxwell_moe.py (fp32), with llama.cpp in F32 on the same tokens:
| check (2 chunks of 512 tokens, our neutral corpus) | mean KLD | same top token |
|---|---|---|
| the first block of Maxwell-V1.0-Flash + its head | 0.000002 | 100.0% |
| the first 3 blocks (dense FFN) + head | 0.000002 | 99.6% |
| the first 6 blocks (3 of them MoE) + head | 0.000055 | 100.0% |
| for scale: the vendor's code on CPU against itself on GPU, 6 blocks | 0.000002 | 100.0% |
The small growth with the MoE blocks comes from exact ties in the router: its selection bias is ~20.2, so in fp32 the score differences below ~2e-6 vanish and 1.8% of tokens tie at the top-12 boundary in the first MoE block. PyTorch and ggml break such ties differently.
test-llama-archs -a maxwell-moe passes on CUDA and CPU, and llama-server --jinja parses reasoning and tool calls from the embedded template.
Model facts
| parameters | 35B total, ~3B active per token (both without embeddings) |
| layers | 45 (3 dense FFN + 42 MoE) |
| hidden / expert FFN | 2048 / 512 (shared expert 1024, dense FFN 6144) |
| experts | 256 routed, 12 active, 1 shared |
| attention | 32 query heads, 4 KV heads, head dim 128; block-sparse in the original |
| context | 256K per the model card (config: 524,288) |
| vocabulary | 200,704 |
| license | Apache-2.0 (LICENSE in this repo, from the base model) |
Quantized by AtomicChat.
- Downloads last month
- -
2-bit
4-bit
6-bit
8-bit
16-bit
Model tree for AtomicChat/Maxwell-V1-Flash-GGUF
Base model
maxwell-model/Maxwell-V1-Flash


