--- license: apache-2.0 base_model: 169Pi/Alpie_seed base_model_relation: quantized pipeline_tag: image-text-to-text library_name: gguf language: [en, hi, bn, mr, gu, te, ml, pa, kn, or, ta] tags: [gguf, rocmfpx, amd, rocm, strix-halo, quantized, alpie, qwen3.5, vision-language, reasoning] --- # Alpie Seed 9B — ROCmFPX GGUF [169Pi/Alpie_seed](https://huggingface.co/169Pi/Alpie_seed) is a 9B vision-language model from 169Pi AI for reasoning, coding and image understanding in English and ten Indian languages. It is post-trained (SFT and GRPO) from Qwen3.5-9B. This repo has it in the [ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX) formats, a llama.cpp fork with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm). **Start with `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf`** if it fits: it is almost identical to the full model. The 6-bit file is smaller and still very close. Add `mmproj-Alpie-Seed-9B-F16.gguf` for images. > These files need [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX), built from > 7 November 2026 or later (older builds are untested with these files). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds will not load them. > **These files have not been tested on AMD hardware yet**; please report speed and quality in the Community tab. > > Not an official release of Alpie Seed. All credit for the model goes to 169Pi AI. Their own files in stock > llama.cpp formats (2-, 3- and 4-bit) are at [169Pi/Alpie_Seed_GGUF](https://huggingface.co/169Pi/Alpie_Seed_GGUF), > and the model is also on [Ollama](https://ollama.com/169pi/alpie-seed). ## Files | File | Size | KL to BF16 | Same top token | | |---|---:|---:|---:|---| | `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf` | 9.3 GB | 0.00039 | 99.5% | 8-bit, closest to the full model | | `Alpie-Seed-9B-Q6_K_ROCMFPX.gguf` | 7.4 GB | 0.0025 | 97.7% | 6-bit (ROCmFPX `Q6_0_ROCMFPX` weights, see below) | | `mmproj-Alpie-Seed-9B-F16.gguf` | 0.9 GB | | | vision encoder, F16, needed for images | Stock llama.cpp builds of this model, measured the same way: | Stock file | Size | KL to BF16 | Same top token | |---|---:|---:|---:| | Q8_0 | 9.6 GB | 0.00042 | 99.2% | | Q6_K | 7.4 GB | 0.0045 | 97.8% | | Q4_K_M | 5.7 GB | 0.0185 | 94.4% | The 8-bit file here is a little closer to BF16 than a stock Q8_0 and 0.3 GB smaller. The 6-bit file has about 45% less drift than a stock Q6_K at the same size. There is no 4-bit file: our best 4-bit recipe (6.6 GB, KL 0.0153) was bigger than a stock Q4_K_M for only a little less drift, so for 4-bit use 169Pi's stock files. **How this was measured:** llama.cpp `llama-perplexity --kl-divergence` against a BF16 GGUF converted from 169Pi's weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16 (the expansion is exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's. This is general English text, not reasoning, coding or Indian-language text. Both files also passed a short check on the CPU with greedy decoding: a chat question ("What is the capital of France?") answered correctly after the thinking block. With the vision encoder, the 8-bit file described two test images (a fox figurine and a bicycle helmet) correctly. The 6-bit file is named `...-Q6_K_ROCMFPX.gguf` so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX `Q6_0_ROCMFPX` type, not a stock llama.cpp `Q6_K`, and it needs the same fork as the 8-bit file. ## The gated DeltaNet fix (7 November 2026) ROCmFPX stores one scale per block of weights as an 8-bit float whose smallest step is fixed, so blocks of small weights lose precision. With `LLAMA_ROCMFPX_TENSOR_SCALE=1` the quantizer multiplies each weight matrix by a power of two first and stores the inverse as `.scale`, which the fork multiplies back after the matrix product. Until 7 November 2026 the quantizer skipped the gated DeltaNet projections (`attn_gate`, `ssm_out`, `ssm_alpha`, `ssm_beta`) of this architecture, so those stayed imprecise even in an 8-bit file. The fix is [`abdbcde56`](https://github.com/HarshaVardhan3002/ROCmFPX/commit/abdbcde56): | KL to BF16 | Before the fix | After | |---|---:|---:| | 8-bit | 0.0024 | 0.00039 | | 6-bit | 0.0064 | 0.0025 | The token embedding and output matrix use stock llama.cpp types: Q8_0 in the 8-bit file and Q6_K in the 6-bit file. ## How these were made 1. Source: 169Pi's BF16 safetensors, converted to a BF16 GGUF plus an F16 vision encoder with the fork's `convert_hf_to_gguf.py`. 2. Importance matrix from the BF16 file over a general calibration text, 512-token chunks. 3. `llama-quantize --imatrix` with `LLAMA_ROCMFPX_TENSOR_SCALE=1`; `--token-embedding-type q8_0 --output-tensor-type q8_0` for the 8-bit file and `q6_K` for both in the 6-bit file. The only change from the original is the weight quantization. The tokenizer and chat template come unchanged from 169Pi's files. ## Run ```bash git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900) cmake --build build --target llama-server -j build/bin/llama-server -m Alpie-Seed-9B-Q8_0_ROCMFPX.gguf --mmproj mmproj-Alpie-Seed-9B-F16.gguf \ -ngl 999 -c 32768 --jinja ``` - The model thinks before it answers by default. For direct answers, add `--chat-template-kwargs '{"enable_thinking":false}'`. - 169Pi suggest temperature 0.7 and top-p 0.8 in their examples, and used temperature 0 for their benchmarks. - The model supports up to 262,144 tokens of context; `-c` sets how much memory llama.cpp reserves for it. - llama.cpp takes text and images. Video input from the original model is not supported. ## License Apache-2.0, same as the original (see `LICENSE`). Alpie Seed is post-trained from [Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), released by Qwen under Apache 2.0. This repo is not affiliated with or endorsed by 169Pi AI or Qwen.