1bit-MONSTER/Zamba2-7B-Instruct-GGUF

Zamba2-7B-Instruct — GGUF

GGUF conversion of Zyphra's Zamba2-7B-Instruct: Mamba-2 layers with shared transformer blocks. It runs from our llama.cpp fork, which carries the Zamba2 model (ggml-org/llama.cpp#21412). The fork also adds a Vulkan Mamba-2 scan for this model's state size and fixes a converter bug that gave every token score -1000.

Contents

• Zamba2-7B-Instruct-Q8_0.gguf

Validation

• Converted after the token-score fix: tokenization matches the model's own tokenizer.
• Measured with llama-bench on Vulkan (Radeon 8060S), Q8_0: prefill 523 tok/s (pp256), decode 17.1 tok/s (tg64). Chat answers are correct.

Running it

With the 1bit engine:

1bit serve -m Zamba2-7B-Instruct-Q8_0.gguf --device vulkan

Attribution

• Base model: Zyphra/Zamba2-7B-Instruct, Apache 2.0.
• GGUF conversion and validation: the 1bit engine project, with the converter and model code in our llama.cpp fork.
• License: Apache 2.0, inherited from the base model.