Maxwell-V1-Flash GGUF

GGUF conversions of maxwell-model/Maxwell-V1-Flash (35B total / 3B active, MoE with manifold-constrained hyper-connections and block-sparse attention), converted directly from the released BF16 safetensors. Supports up to 256K context.

๐Ÿฆ™ Requires the Maxwell port ๐Ÿฆ™

The maxwell-moe architecture is not in stock llama.cpp yet. Build the Maxwell port before loading these files:

git clone --branch maxwell-moe https://github.com/aetherbird/llama.cpp.git
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-server llama-cli llama-quantize

Alternatively, apply maxwell-moe-port.patch to llama.cpp master (91979450).

The other published port (AtomicChat) uses dense attention everywhere and drops the MTP block. This port implements the model's block-sparse attention past its 16K dense prefix and exports a usable MTP head, so long-context behaviour matches training.

Quant Sizing

Generally... Larger files = more precision. Smaller files = more compression = more slop and misbehavin'.

Weights and context share your memory, so be sure to leave headroom. KV costs about 90 KB per token at fp16 (--cache-type-k q8_0 --cache-type-v q8_0 halves it).

your memory file size
56 GB+ Q8_0 / Q8_K_XL 39.2 / ~42 GB
48 GB Q6_K ~31 GB
40 GB Q5_K_M ~27 GB
32 GB Q4_K_M 22.3 GB
32 GB IQ4_XS ~21 GB
24 GB Q3_K_M ~17 GB
96 GB+ (or CPU spill) BF16 (2 parts) 73.7 GB

Usage

Recommended sampling from the source model card: temperature 0.6, top_p 0.95, top_k 20. Thinking is on by default; disable per request with "chat_template_kwargs": {"enable_thinking": false}. The chat template and tool-call format are embedded, so pass --jinja.

./build/bin/llama-server \
  -m Maxwell-V1-Flash-Q4_K_M.gguf \
  -c 262144 \
  -ngl 99 \
  --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --jinja

Attention Behaviour

The model attends densely to the first 16K query positions and uses block-sparse attention (8 prefix + 8 local + up to 16 indexer-selected 128-token blocks) beyond that, matching its training. The threshold is a GGUF key (attention.indexer.dense_prefix_tokens).

Flash attention is required for the sparse path; without it the model falls back to dense attention at all lengths.

MTP Drafting

Every quant bundles the MTP/NextN block. Enable it with --spec-type draft-mtp:

./build/bin/llama-server \
  -m Maxwell-V1-Flash-Q8_0.gguf \
  -c 262144 -ngl 99 --flash-attn on \
  --temp 0.6 --top-p 0.95 --top-k 20 \
  --jinja \
  --spec-type draft-mtp

Measured on 2xV100 with the Q8_0 file: draft acceptance 0.32, mean accepted length 1.92 tokens per step, not a net speedup on that hardware.

Conversion and Quantization

  • Block-sparse attention metadata: indexer head count / key length / top-k / block size / local and prefix blocks, plus the 16K dense prefix.
  • Hyper-connection parameters: 4 residual streams, 20 Sinkhorn iterations, epsilon 1e-6.
  • MTP block exported as a NextN layer.
  • Chat template embedded; <|maxwell_end|> (id 200001) written as the EOT stop token.
  • Script-aware tokenizer pre-type (maxwell-moe).
  • Norms, routing tensors, expert bias, and hyper-connection matrices remain F32.

Importance Matrix

Generated from the Q8_0 model over a mixed code/English calibration corpus (not the vendor's calibration set):

  • 604 chunks x 512 tokens = ~309,000 calibration tokens
  • 573 matrix entries
  • Uploaded as imatrix/maxwell.imatrix.gguf

All K-quants and i-quants use this matrix. Q8_K_XL keeps F16 embeddings, output head, attention q/k/v/o and indexer projections, with everything else at Q8_0.

Validation Completed

  • Architecture load and tensor round-trip through the fork's GGUF saver
  • test-llama-archs -a maxwell-moe (NMSE, round-trip, mixed batch, KV shift) on CPU and CUDA
  • Reasoning and tool-call parsing through llama-server --jinja (embedded template)
  • Tool-call generation and tool-result round trips on both sides of the 16K boundary
  • Tokenizer parity against the reference tokenizer on a 13-case sample
  • Dense-prefix switching measured at 2K / 8K / 18K context

Notes

  • The GGUF contains 46 blocks: 45 backbone layers (3 dense FFN + 42 MoE) plus one MTP layer.
  • The tokenizer was built for Thai, Lao, Myanmar, Khmer and Shan as well as CJK and Latin text; the pre-type was validated on a small sample only.
  • Agentic multi-step tool use is not reliable in this model; single tool calls and tool-result round trips work.

License

Apache-2.0, inherited from the base model.

Downloads last month
2,954
GGUF
Model size
37B params
Architecture
maxwell-moe
Hardware compatibility
Log In to add your hardware

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for bloomer010/Maxwell-V1-Flash-GGUF

Quantized
(9)
this model