Harsha001/Alpie-Seed-9B-ROCmFPX

Alpie Seed 9B — ROCmFPX GGUF

169Pi/Alpie_seed is a 9B vision-language model from 169Pi AI for reasoning, coding and image understanding in English and ten Indian languages. It is post-trained (SFT and GRPO) from Qwen3.5-9B. This repo has it in the ROCmFPX formats, a llama.cpp fork with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

Start with Alpie-Seed-9B-Q8_0_ROCMFPX.gguf if it fits: it is almost identical to the full model. The 6-bit file is smaller and still very close. Add mmproj-Alpie-Seed-9B-F16.gguf for images.

These files need HarshaVardhan3002/ROCmFPX, built from 7 November 2026 or later (older builds are untested with these files). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds will not load them. These files have not been tested on AMD hardware yet; please report speed and quality in the Community tab.

Not an official release of Alpie Seed. All credit for the model goes to 169Pi AI. Their own files in stock llama.cpp formats (2-, 3- and 4-bit) are at 169Pi/Alpie_Seed_GGUF, and the model is also on Ollama.

Files

File	Size	KL to BF16	Same top token
Alpie-Seed-9B-Q8_0_ROCMFPX.gguf	9.3 GB	0.00039	99.5%	8-bit, closest to the full model
Alpie-Seed-9B-Q6_K_ROCMFPX.gguf	7.4 GB	0.0025	97.7%	6-bit (ROCmFPX Q6_0_ROCMFPX weights, see below)
mmproj-Alpie-Seed-9B-F16.gguf	0.9 GB			vision encoder, F16, needed for images

Stock llama.cpp builds of this model, measured the same way:

Stock file	Size	KL to BF16	Same top token
Q8_0	9.6 GB	0.00042	99.2%
Q6_K	7.4 GB	0.0045	97.8%
Q4_K_M	5.7 GB	0.0185	94.4%

The 8-bit file here is a little closer to BF16 than a stock Q8_0 and 0.3 GB smaller. The 6-bit file has about 45% less drift than a stock Q6_K at the same size. There is no 4-bit file: our best 4-bit recipe (6.6 GB, KL 0.0153) was bigger than a stock Q4_K_M for only a little less drift, so for 4-bit use 169Pi's stock files.

How this was measured: llama.cpp llama-perplexity --kl-divergence against a BF16 GGUF converted from 169Pi's weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16 (the expansion is exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token; lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's. This is general English text, not reasoning, coding or Indian-language text.

Both files also passed a short check on the CPU with greedy decoding: a chat question ("What is the capital of France?") answered correctly after the thinking block. With the vision encoder, the 8-bit file described two test images (a fox figurine and a bicycle helmet) correctly.

The 6-bit file is named ...-Q6_K_ROCMFPX.gguf so that Hugging Face lists it as a 6-bit quant. Its weights are the ROCmFPX Q6_0_ROCMFPX type, not a stock llama.cpp Q6_K, and it needs the same fork as the 8-bit file.

The gated DeltaNet fix (7 November 2026)

ROCmFPX stores one scale per block of weights as an 8-bit float whose smallest step is fixed, so blocks of small weights lose precision. With LLAMA_ROCMFPX_TENSOR_SCALE=1 the quantizer multiplies each weight matrix by a power of two first and stores the inverse as <name>.scale, which the fork multiplies back after the matrix product. Until 7 November 2026 the quantizer skipped the gated DeltaNet projections (attn_gate, ssm_out, ssm_alpha, ssm_beta) of this architecture, so those stayed imprecise even in an 8-bit file. The fix is abdbcde56:

KL to BF16	Before the fix	After
8-bit	0.0024	0.00039
6-bit	0.0064	0.0025

The token embedding and output matrix use stock llama.cpp types: Q8_0 in the 8-bit file and Q6_K in the 6-bit file.

How these were made

1. Source: 169Pi's BF16 safetensors, converted to a BF16 GGUF plus an F16 vision encoder with the fork's convert_hf_to_gguf.py.
2. Importance matrix from the BF16 file over a general calibration text, 512-token chunks.
3. llama-quantize --imatrix with LLAMA_ROCMFPX_TENSOR_SCALE=1; --token-embedding-type q8_0 --output-tensor-type q8_0 for the 8-bit file and q6_K for both in the 6-bit file.

The only change from the original is the weight quantization. The tokenizer and chat template come unchanged from 169Pi's files.

Run

git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m Alpie-Seed-9B-Q8_0_ROCMFPX.gguf --mmproj mmproj-Alpie-Seed-9B-F16.gguf \
  -ngl 999 -c 32768 --jinja

• The model thinks before it answers by default. For direct answers, add --chat-template-kwargs '{"enable_thinking":false}'.
• 169Pi suggest temperature 0.7 and top-p 0.8 in their examples, and used temperature 0 for their benchmarks.
• The model supports up to 262,144 tokens of context; -c sets how much memory llama.cpp reserves for it.
• llama.cpp takes text and images. Video input from the original model is not supported.

License

Apache-2.0, same as the original (see LICENSE). Alpie Seed is post-trained from Qwen/Qwen3.5-9B, released by Qwen under Apache 2.0. This repo is not affiliated with or endorsed by 169Pi AI or Qwen.