Instructions to use Harsha001/Alpie-Seed-9B-ROCmFPX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX
# Run inference directly in the terminal:
llama cli -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX
# Run inference directly in the terminal:
llama cli -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX
# Run inference directly in the terminal:
./build/bin/llama cli -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Use Docker

docker model run hf.co/Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Llama new
LM Studio
Jan

vLLM

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Harsha001/Alpie-Seed-9B-ROCmFPX"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Harsha001/Alpie-Seed-9B-ROCmFPX",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Ollama

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with Ollama:

ollama run hf.co/Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Unsloth Desktop

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use Harsha001/Alpie-Seed-9B-ROCmFPX with Docker Model Runner:
```
docker model run hf.co/Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX
```

Lemonade

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Run and chat with the model

lemonade run user.Alpie-Seed-9B-ROCmFPX-Q6_K_ROCMFPX

List all available models

lemonade list

Hermes Agent

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use Harsha001/Alpie-Seed-9B-ROCmFPX with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "Harsha001/Alpie-Seed-9B-ROCmFPX:Q6_K_ROCMFPX" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Alpie-Seed-9B-ROCmFPX / README.md

Harsha001

Model card and licence

181f83e verified 6 days ago

preview code

Raw History Blame Contribute Delete

6.14 kB

---
license: apache-2.0
base_model: 169Pi/Alpie_seed
base_model_relation: quantized
pipeline_tag: image-text-to-text
library_name: gguf
language: [en, hi, bn, mr, gu, te, ml, pa, kn, or, ta]
tags: [gguf, rocmfpx, amd, rocm, strix-halo, quantized, alpie, qwen3.5, vision-language, reasoning]
---
# Alpie Seed 9B — ROCmFPX GGUF

[169Pi/Alpie_seed](https://huggingface.co/169Pi/Alpie_seed) is a 9B vision-language model from 169Pi AI for
reasoning, coding and image understanding in English and ten Indian languages. It is post-trained (SFT and GRPO) from
Qwen3.5-9B. This repo has it in the [ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX) formats, a llama.cpp fork
with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

**Start with `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf`** if it fits: it is almost identical to the full model. The 6-bit
file is smaller and still very close. Add `mmproj-Alpie-Seed-9B-F16.gguf` for images.

> These files need [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX), built from
> 7 November 2026 or later (older builds are untested with these files). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds will not load them.
> **These files have not been tested on AMD hardware yet**; please report speed and quality in the Community tab.
>
> Not an official release of Alpie Seed. All credit for the model goes to 169Pi AI. Their own files in stock
> llama.cpp formats (2-, 3- and 4-bit) are at [169Pi/Alpie_Seed_GGUF](https://huggingface.co/169Pi/Alpie_Seed_GGUF),
> and the model is also on [Ollama](https://ollama.com/169pi/alpie-seed).

## Files

| File | Size | KL to BF16 | Same top token | |
|---|---:|---:|---:|---|
| `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf` | 9.3 GB | 0.00039 | 99.5% | 8-bit, closest to the full model |
| `Alpie-Seed-9B-Q6_K_ROCMFPX.gguf` | 7.4 GB | 0.0025 | 97.7% | 6-bit (ROCmFPX `Q6_0_ROCMFPX` weights, see below) |
| `mmproj-Alpie-Seed-9B-F16.gguf` | 0.9 GB | | | vision encoder, F16, needed for images |

Stock llama.cpp builds of this model, measured the same way:

| Stock file | Size | KL to BF16 | Same top token |
|---|---:|---:|---:|
| Q8_0 | 9.6 GB | 0.00042 | 99.2% |
| Q6_K | 7.4 GB | 0.0045 | 97.8% |
| Q4_K_M | 5.7 GB | 0.0185 | 94.4% |

The 8-bit file here is a little closer to BF16 than a stock Q8_0 and 0.3 GB smaller. The 6-bit file has about 45%
less drift than a stock Q6_K at the same size. There is no 4-bit file: our best 4-bit recipe (6.6 GB, KL 0.0153) was
bigger than a stock Q4_K_M for only a little less drift, so for 4-bit use 169Pi's stock files.

**How this was measured:** llama.cpp `llama-perplexity --kl-divergence` against a BF16 GGUF converted from 169Pi's
weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16
(the expansion is exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token;
lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's. This is general
English text, not reasoning, coding or Indian-language text.

Both files also passed a short check on the CPU with greedy decoding: a chat question ("What is the capital of
France?") answered correctly after the thinking block. With the vision encoder, the 8-bit file described two test
images (a fox figurine and a bicycle helmet) correctly.

The 6-bit file is named `...-Q6_K_ROCMFPX.gguf` so that Hugging Face lists it as a 6-bit quant. Its weights are the
ROCmFPX `Q6_0_ROCMFPX` type, not a stock llama.cpp `Q6_K`, and it needs the same fork as the 8-bit file.

## The gated DeltaNet fix (7 November 2026)

ROCmFPX stores one scale per block of weights as an 8-bit float whose smallest step is fixed, so blocks of small
weights lose precision. With `LLAMA_ROCMFPX_TENSOR_SCALE=1` the quantizer multiplies each weight matrix by a power of
two first and stores the inverse as `<name>.scale`, which the fork multiplies back after the matrix product. Until 7
November 2026 the quantizer skipped the gated DeltaNet projections (`attn_gate`, `ssm_out`, `ssm_alpha`, `ssm_beta`)
of this architecture, so those stayed imprecise even in an 8-bit file. The fix is
[`abdbcde56`](https://github.com/HarshaVardhan3002/ROCmFPX/commit/abdbcde56):

| KL to BF16 | Before the fix | After |
|---|---:|---:|
| 8-bit | 0.0024 | 0.00039 |
| 6-bit | 0.0064 | 0.0025 |

The token embedding and output matrix use stock llama.cpp types: Q8_0 in the 8-bit file and Q6_K in the 6-bit file.

## How these were made

1. Source: 169Pi's BF16 safetensors, converted to a BF16 GGUF plus an F16 vision encoder with the fork's
   `convert_hf_to_gguf.py`.
2. Importance matrix from the BF16 file over a general calibration text, 512-token chunks.
3. `llama-quantize --imatrix` with `LLAMA_ROCMFPX_TENSOR_SCALE=1`; `--token-embedding-type q8_0 --output-tensor-type
   q8_0` for the 8-bit file and `q6_K` for both in the 6-bit file.
The only change from the original is the weight quantization. The tokenizer and chat template come unchanged from
169Pi's files.
## Run
```bash
git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
cmake -B build -DGGML_VULKAN=ON          # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
cmake --build build --target llama-server -j
build/bin/llama-server -m Alpie-Seed-9B-Q8_0_ROCMFPX.gguf --mmproj mmproj-Alpie-Seed-9B-F16.gguf \
  -ngl 999 -c 32768 --jinja
```
- The model thinks before it answers by default. For direct answers, add
  `--chat-template-kwargs '{"enable_thinking":false}'`.
- 169Pi suggest temperature 0.7 and top-p 0.8 in their examples, and used temperature 0 for their benchmarks.
- The model supports up to 262,144 tokens of context; `-c` sets how much memory llama.cpp reserves for it.
- llama.cpp takes text and images. Video input from the original model is not supported.
## License
Apache-2.0, same as the original (see `LICENSE`). Alpie Seed is post-trained from
[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), released by Qwen under Apache 2.0. This repo is not
affiliated with or endorsed by 169Pi AI or Qwen.

	---
	license: apache-2.0
	base_model: 169Pi/Alpie_seed
	base_model_relation: quantized
	pipeline_tag: image-text-to-text
	library_name: gguf
	language: [en, hi, bn, mr, gu, te, ml, pa, kn, or, ta]
	tags: [gguf, rocmfpx, amd, rocm, strix-halo, quantized, alpie, qwen3.5, vision-language, reasoning]
	---

	# Alpie Seed 9B — ROCmFPX GGUF

	[169Pi/Alpie_seed](https://huggingface.co/169Pi/Alpie_seed) is a 9B vision-language model from 169Pi AI for
	reasoning, coding and image understanding in English and ten Indian languages. It is post-trained (SFT and GRPO) from
	Qwen3.5-9B. This repo has it in the [ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX) formats, a llama.cpp fork
	with low-bit weight types for AMD GPUs (Vulkan and HIP/ROCm).

	Start with `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf` if it fits: it is almost identical to the full model. The 6-bit
	file is smaller and still very close. Add `mmproj-Alpie-Seed-9B-F16.gguf` for images.

	> These files need [HarshaVardhan3002/ROCmFPX](https://github.com/HarshaVardhan3002/ROCmFPX), built from
	> 7 November 2026 or later (older builds are untested with these files). Stock llama.cpp, LM Studio, Ollama and other ROCmFPX builds will not load them.
	> These files have not been tested on AMD hardware yet; please report speed and quality in the Community tab.
	>
	> Not an official release of Alpie Seed. All credit for the model goes to 169Pi AI. Their own files in stock
	> llama.cpp formats (2-, 3- and 4-bit) are at [169Pi/Alpie_Seed_GGUF](https://huggingface.co/169Pi/Alpie_Seed_GGUF),
	> and the model is also on [Ollama](https://ollama.com/169pi/alpie-seed).

	## Files

	\| File \| Size \| KL to BF16 \| Same top token \| \|
	\|---\|---:\|---:\|---:\|---\|
	\| `Alpie-Seed-9B-Q8_0_ROCMFPX.gguf` \| 9.3 GB \| 0.00039 \| 99.5% \| 8-bit, closest to the full model \|
	\| `Alpie-Seed-9B-Q6_K_ROCMFPX.gguf` \| 7.4 GB \| 0.0025 \| 97.7% \| 6-bit (ROCmFPX `Q6_0_ROCMFPX` weights, see below) \|
	\| `mmproj-Alpie-Seed-9B-F16.gguf` \| 0.9 GB \| \| \| vision encoder, F16, needed for images \|

	Stock llama.cpp builds of this model, measured the same way:

	\| Stock file \| Size \| KL to BF16 \| Same top token \|
	\|---\|---:\|---:\|---:\|
	\| Q8_0 \| 9.6 GB \| 0.00042 \| 99.2% \|
	\| Q6_K \| 7.4 GB \| 0.0045 \| 97.8% \|
	\| Q4_K_M \| 5.7 GB \| 0.0185 \| 94.4% \|

	The 8-bit file here is a little closer to BF16 than a stock Q8_0 and 0.3 GB smaller. The 6-bit file has about 45%
	less drift than a stock Q6_K at the same size. There is no 4-bit file: our best 4-bit recipe (6.6 GB, KL 0.0153) was
	bigger than a stock Q4_K_M for only a little less drift, so for 4-bit use 169Pi's stock files.

	How this was measured: llama.cpp `llama-perplexity --kl-divergence` against a BF16 GGUF converted from 169Pi's
	weights, 40 chunks of 512 tokens of English Wikipedia text (wikitext-2), run on a GPU after expanding each file to F16
	(the expansion is exact). KL is how far the file's next-token probabilities drift from BF16, averaged over every token;
	lower is better. "Same top token" is how often the file's most likely next token is the same as BF16's. This is general
	English text, not reasoning, coding or Indian-language text.

	Both files also passed a short check on the CPU with greedy decoding: a chat question ("What is the capital of
	France?") answered correctly after the thinking block. With the vision encoder, the 8-bit file described two test
	images (a fox figurine and a bicycle helmet) correctly.

	The 6-bit file is named `...-Q6_K_ROCMFPX.gguf` so that Hugging Face lists it as a 6-bit quant. Its weights are the
	ROCmFPX `Q6_0_ROCMFPX` type, not a stock llama.cpp `Q6_K`, and it needs the same fork as the 8-bit file.

	## The gated DeltaNet fix (7 November 2026)

	ROCmFPX stores one scale per block of weights as an 8-bit float whose smallest step is fixed, so blocks of small
	weights lose precision. With `LLAMA_ROCMFPX_TENSOR_SCALE=1` the quantizer multiplies each weight matrix by a power of
	two first and stores the inverse as `<name>.scale`, which the fork multiplies back after the matrix product. Until 7
	November 2026 the quantizer skipped the gated DeltaNet projections (`attn_gate`, `ssm_out`, `ssm_alpha`, `ssm_beta`)
	of this architecture, so those stayed imprecise even in an 8-bit file. The fix is
	[`abdbcde56`](https://github.com/HarshaVardhan3002/ROCmFPX/commit/abdbcde56):

	\| KL to BF16 \| Before the fix \| After \|
	\|---\|---:\|---:\|
	\| 8-bit \| 0.0024 \| 0.00039 \|
	\| 6-bit \| 0.0064 \| 0.0025 \|

	The token embedding and output matrix use stock llama.cpp types: Q8_0 in the 8-bit file and Q6_K in the 6-bit file.

	## How these were made

	1. Source: 169Pi's BF16 safetensors, converted to a BF16 GGUF plus an F16 vision encoder with the fork's
	`convert_hf_to_gguf.py`.
	2. Importance matrix from the BF16 file over a general calibration text, 512-token chunks.
	3. `llama-quantize --imatrix` with `LLAMA_ROCMFPX_TENSOR_SCALE=1`; `--token-embedding-type q8_0 --output-tensor-type
	q8_0` for the 8-bit file and `q6_K` for both in the 6-bit file.

	The only change from the original is the weight quantization. The tokenizer and chat template come unchanged from
	169Pi's files.

	## Run

	```bash
	git clone https://github.com/HarshaVardhan3002/ROCmFPX && cd ROCmFPX
	cmake -B build -DGGML_VULKAN=ON # or -DGGML_HIP=ON -DAMDGPU_TARGETS=gfx1151 (gfx1100 for RX 7900)
	cmake --build build --target llama-server -j
	build/bin/llama-server -m Alpie-Seed-9B-Q8_0_ROCMFPX.gguf --mmproj mmproj-Alpie-Seed-9B-F16.gguf \
	-ngl 999 -c 32768 --jinja
	```

	- The model thinks before it answers by default. For direct answers, add
	`--chat-template-kwargs '{"enable_thinking":false}'`.
	- 169Pi suggest temperature 0.7 and top-p 0.8 in their examples, and used temperature 0 for their benchmarks.
	- The model supports up to 262,144 tokens of context; `-c` sets how much memory llama.cpp reserves for it.
	- llama.cpp takes text and images. Video input from the original model is not supported.

	## License

	Apache-2.0, same as the original (see `LICENSE`). Alpie Seed is post-trained from
	[Qwen/Qwen3.5-9B](https://huggingface.co/Qwen/Qwen3.5-9B), released by Qwen under Apache 2.0. This repo is not
	affiliated with or endorsed by 169Pi AI or Qwen.