Instructions to use bloomer010/Maxwell-V1-Flash-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use bloomer010/Maxwell-V1-Flash-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Use Docker
docker model run hf.co/bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use bloomer010/Maxwell-V1-Flash-GGUF with Ollama:
ollama run hf.co/bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use bloomer010/Maxwell-V1-Flash-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use bloomer010/Maxwell-V1-Flash-GGUF with Docker Model Runner:
docker model run hf.co/bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
- Lemonade
How to use bloomer010/Maxwell-V1-Flash-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.Maxwell-V1-Flash-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use bloomer010/Maxwell-V1-Flash-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use bloomer010/Maxwell-V1-Flash-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "bloomer010/Maxwell-V1-Flash-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Maxwell-V1-Flash GGUF
GGUF conversions of maxwell-model/Maxwell-V1-Flash (35B total / 3B active, MoE with manifold-constrained hyper-connections and block-sparse attention), converted directly from the released BF16 safetensors. Supports up to 256K context.
๐ฆ Requires the Maxwell port ๐ฆ
The maxwell-moe architecture is not in stock llama.cpp yet. Build the Maxwell port before loading these files:
git clone --branch maxwell-moe https://github.com/aetherbird/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-server llama-cli llama-quantize
Alternatively, apply maxwell-moe-port.patch to llama.cpp master (91979450).
The other published port (AtomicChat) uses dense attention everywhere and drops the MTP block. This port implements the model's block-sparse attention past its 16K dense prefix and exports a usable MTP head, so long-context behaviour matches training.
Quant Sizing
Generally... Larger files = more precision. Smaller files = more compression = more slop and misbehavin'.
Weights and context share your memory, so be sure to leave headroom. KV costs about 90 KB per token at fp16 (--cache-type-k q8_0 --cache-type-v q8_0 halves it).
| your memory | file | size |
|---|---|---|
| 56 GB+ | Q8_0 / Q8_K_XL | 39.2 / ~42 GB |
| 48 GB | Q6_K | ~31 GB |
| 40 GB | Q5_K_M | ~27 GB |
| 32 GB | Q4_K_M | 22.3 GB |
| 32 GB | IQ4_XS | ~21 GB |
| 24 GB | Q3_K_M | ~17 GB |
| 96 GB+ (or CPU spill) | BF16 (2 parts) | 73.7 GB |
Usage
Recommended sampling from the source model card: temperature 0.6, top_p 0.95, top_k 20. Thinking is on by default; disable per request with "chat_template_kwargs": {"enable_thinking": false}. The chat template and tool-call format are embedded, so pass --jinja.
./build/bin/llama-server \
-m Maxwell-V1-Flash-Q4_K_M.gguf \
-c 262144 \
-ngl 99 \
--flash-attn on \
--cache-type-k q8_0 --cache-type-v q8_0 \
--temp 0.6 --top-p 0.95 --top-k 20 \
--jinja
Attention Behaviour
The model attends densely to the first 16K query positions and uses block-sparse attention (8 prefix + 8 local + up to 16 indexer-selected 128-token blocks) beyond that, matching its training. The threshold is a GGUF key (attention.indexer.dense_prefix_tokens).
Flash attention is required for the sparse path; without it the model falls back to dense attention at all lengths.
MTP Drafting
Every quant bundles the MTP/NextN block. Enable it with --spec-type draft-mtp:
./build/bin/llama-server \
-m Maxwell-V1-Flash-Q8_0.gguf \
-c 262144 -ngl 99 --flash-attn on \
--temp 0.6 --top-p 0.95 --top-k 20 \
--jinja \
--spec-type draft-mtp
Measured on 2xV100 with the Q8_0 file: draft acceptance 0.32, mean accepted length 1.92 tokens per step, not a net speedup on that hardware.
Conversion and Quantization
- Block-sparse attention metadata: indexer head count / key length / top-k / block size / local and prefix blocks, plus the 16K dense prefix.
- Hyper-connection parameters: 4 residual streams, 20 Sinkhorn iterations, epsilon 1e-6.
- MTP block exported as a NextN layer.
- Chat template embedded;
<|maxwell_end|>(id 200001) written as the EOT stop token. - Script-aware tokenizer pre-type (
maxwell-moe). - Norms, routing tensors, expert bias, and hyper-connection matrices remain F32.
Importance Matrix
Generated from the Q8_0 model over a mixed code/English calibration corpus (not the vendor's calibration set):
- 604 chunks x 512 tokens = ~309,000 calibration tokens
- 573 matrix entries
- Uploaded as
imatrix/maxwell.imatrix.gguf
All K-quants and i-quants use this matrix. Q8_K_XL keeps F16 embeddings, output head, attention q/k/v/o and indexer projections, with everything else at Q8_0.
Validation Completed
- Architecture load and tensor round-trip through the fork's GGUF saver
test-llama-archs -a maxwell-moe(NMSE, round-trip, mixed batch, KV shift) on CPU and CUDA- Reasoning and tool-call parsing through
llama-server --jinja(embedded template) - Tool-call generation and tool-result round trips on both sides of the 16K boundary
- Tokenizer parity against the reference tokenizer on a 13-case sample
- Dense-prefix switching measured at 2K / 8K / 18K context
Notes
- The GGUF contains 46 blocks: 45 backbone layers (3 dense FFN + 42 MoE) plus one MTP layer.
- The tokenizer was built for Thai, Lao, Myanmar, Khmer and Shan as well as CJK and Latin text; the pre-type was validated on a small sample only.
- Agentic multi-step tool use is not reliable in this model; single tool calls and tool-result round trips work.
License
Apache-2.0, inherited from the base model.
- Downloads last month
- 2,954
Model tree for bloomer010/Maxwell-V1-Flash-GGUF
Base model
maxwell-model/Maxwell-V1-Flash