Spark-X2.5-4B Q8_0 GGUF
Community Q8_0 quantized GGUF conversion of XHToken/Spark-X2.5-4B for local inference with Ollama and llama.cpp-compatible runtimes.
Converted to GGUF, quantized to Q8_0, tested, and packaged by Yousef Burhan Ahmad.
Quick Start — Hugging Face to Ollama
The Q8_0 GGUF model is available on Hugging Face:
https://huggingface.co/YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF
The basic workflow is:
Hugging Face
↓
Download Q8_0 GGUF
↓
Create Modelfile
↓
Import into Ollama
↓
Run locally
Model Details
Model: Spark-X2.5-4B
Original Developer: XHToken
Base Model: XHToken/Spark-X2.5-4B
GGUF Conversion: Yousef Burhan Ahmad
Q8_0 Quantization: Yousef Burhan Ahmad
Runtime Preparation: Yousef Burhan Ahmad
CUDA Testing: Yousef Burhan Ahmad
Ollama Packaging: Yousef Burhan Ahmad
Quantization: Q8_0
Format: GGUF
Size: ~4.07 GiB
Ollama Download Size: ~4.4 GB
Architecture: spark2_5
Parameters: ~4B
Native Context Length: up to 1,048,576 tokens
Recommended Local Context: 16K–32K depending on available VRAM
License: Apache-2.0
Conversion Tool: llama.cpp b10828
Step 1 — Download the GGUF Model
Option A — Download from Hugging Face
Open:
https://huggingface.co/YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF
Download:
Spark-X2.5-4B-Q8_0.gguf
Create a folder for the model:
Spark-X2.5-4B-Q8_0-GGUF
Place the downloaded GGUF file inside it.
Your folder will eventually look like:
Spark-X2.5-4B-Q8_0-GGUF/
│
├── Spark-X2.5-4B-Q8_0.gguf
└── Modelfile
Option B — Download with Hugging Face CLI
Install the Hugging Face CLI:
pip install -U huggingface_hub
Create a folder:
Windows PowerShell
mkdir Spark-X2.5-4B-Q8_0-GGUF
cd Spark-X2.5-4B-Q8_0-GGUF
Linux / macOS
mkdir Spark-X2.5-4B-Q8_0-GGUF
cd Spark-X2.5-4B-Q8_0-GGUF
Download the GGUF:
hf download YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF Spark-X2.5-4B-Q8_0.gguf --local-dir .
Step 2 — Create the Modelfile
Inside the same folder as the GGUF file, create a file named exactly:
Modelfile
Do not add .txt.
Paste:
FROM ./Spark-X2.5-4B-Q8_0.gguf
PARAMETER num_ctx 32768
PARAMETER num_predict 8192
PARAMETER temperature 1.0
PARAMETER top_p 0.95
PARAMETER top_k -1
PARAMETER repeat_penalty 1.0
PARAMETER repeat_last_n 64
Your folder should now look like:
Spark-X2.5-4B-Q8_0-GGUF/
│
├── Spark-X2.5-4B-Q8_0.gguf
└── Modelfile
Step 3 — Import the Model into Ollama
Open a terminal inside the model folder.
Windows PowerShell
ollama create spark-x2.5-4b:q8 -f .\Modelfile
Linux / macOS
ollama create spark-x2.5-4b:q8 -f Modelfile
After the import finishes, the local model name will be:
spark-x2.5-4b:q8
Step 4 — Run the Model
Run:
ollama run spark-x2.5-4b:q8
Check installed models with:
ollama list
Important — Spark-X2.5 Runtime Compatibility
Spark-X2.5 uses the architecture:
spark2_5
Some Ollama releases may not include support for this architecture.
If you receive:
unknown model architecture: 'spark2_5'
this does not necessarily mean that the GGUF model is corrupted.
It means that your current Ollama runtime does not support the spark2_5 architecture.
In that case, use a Spark-X2.5-compatible Ollama build.
Spark-X2.5-Compatible Ollama Runtime
The Spark-compatible setup uses:
Official Ollama Source
+
Spark-Compatible llama.cpp
ollama-sparkis only the local folder name used in the commands below. It is not a separate official Ollama project.
Step 5 — Download Ollama Source
Clone the official Ollama repository:
git clone https://github.com/ollama/ollama.git ollama-spark
Then enter the repository:
cd ollama-spark
Official Ollama repository:
https://github.com/ollama/ollama
You should now have:
ollama-spark/
Step 6 — Download Spark-Compatible llama.cpp
Clone the Spark-X2.5-compatible llama.cpp version:
git clone -b b10828 https://github.com/ggml-org/llama.cpp.git llama.cpp-spark
Official llama.cpp repository:
https://github.com/ggml-org/llama.cpp
The folder will be:
llama.cpp-spark/
Example structure:
workspace/
│
├── ollama-spark/
└── llama.cpp-spark/
Step 7 — Point Ollama to Spark-Compatible llama.cpp
Enter the Ollama source folder:
cd ollama-spark
On Windows PowerShell:
$env:OLLAMA_LLAMA_CPP_SOURCE="$(Resolve-Path ..\llama.cpp-spark)"
Verify:
$env:OLLAMA_LLAMA_CPP_SOURCE
It should point to:
llama.cpp-spark
Step 8 — Build Ollama
Windows + NVIDIA CUDA
Requirements may include:
Git
Go
CMake
Visual Studio 2022
Desktop Development with C++
CUDA Toolkit
MSYS2 / UCRT64 GCC for CGO
Configure:
cmake -S . -B build `
-G "Visual Studio 17 2022" `
-A x64 `
-DOLLAMA_LLAMA_BACKENDS=cuda_v13 `
-DCMAKE_CUDA_ARCHITECTURES=native
Build:
cmake --build build --config Release --parallel 8
If the Ollama Go executable still needs to be generated:
cmake --build build --config Release --target ollama-go --parallel 8
Windows GCC / CGO Setup
If you receive:
cgo: C compiler "gcc" not found
install MSYS2:
winget install -e --id MSYS2.MSYS2
Install GCC:
C:\msys64\usr\bin\bash.exe -lc "pacman -S --needed --noconfirm mingw-w64-ucrt-x86_64-gcc"
Then expose GCC:
$env:Path = "C:\msys64\ucrt64\bin;$env:Path"
$env:CGO_ENABLED = "1"
$env:CC = "C:\msys64\ucrt64\bin\gcc.exe"
$env:CXX = "C:\msys64\ucrt64\bin\g++.exe"
Verify:
gcc --version
Then build:
cmake --build build --config Release --target ollama-go --parallel 8
Step 9 — Start Spark-Compatible Ollama
From the ollama-spark directory:
.\ollama.exe serve
Keep this PowerShell window open.
Step 10 — Import the GGUF with the Spark-Compatible Runtime
Open another PowerShell window.
Go to the folder containing:
Spark-X2.5-4B-Q8_0.gguf
Modelfile
Example:
cd "C:\path\to\Spark-X2.5-4B-Q8_0-GGUF"
Then create the model using the Spark-compatible Ollama executable:
"C:\path\to\ollama-spark\ollama.exe" create spark-x2.5-4b:q8 -f .\Modelfile
Run:
"C:\path\to\ollama-spark\ollama.exe" run spark-x2.5-4b:q8
Complete Workflow
Standard Ollama
Hugging Face
↓
Spark-X2.5-4B-Q8_0.gguf
↓
Modelfile
↓
ollama create spark-x2.5-4b:q8
↓
ollama run spark-x2.5-4b:q8
If it works, no additional setup is required.
If spark2_5 Is Unsupported
Clone Official Ollama
↓
ollama-spark/
+
Clone llama.cpp b10828
↓
llama.cpp-spark/
↓
Connect Ollama to compatible llama.cpp
↓
Build with CUDA
↓
Start custom Ollama server
↓
Import Q8_0 GGUF
↓
Run Spark-X2.5-4B
Q8_0 Quantization
The original Spark-X2.5-4B model weights were converted from the Hugging Face format to GGUF FP16 and then quantized to Q8_0 by Yousef Burhan Ahmad.
The quantization pipeline was:
XHToken/Spark-X2.5-4B
↓
Hugging Face Safetensors
↓
GGUF FP16
↓
Q8_0 Quantization
↓
Spark-X2.5-4B-Q8_0.gguf
The Q8_0 quantization was performed using:
llama.cpp b10828
Approximate sizes:
FP16 GGUF: ~7.66 GiB
Q8_0 GGUF: ~4.07 GiB
Ollama distribution: ~4.4 GB
This reduced the GGUF model weight file size from approximately:
7.66 GiB
↓
4.07 GiB
which represents approximately a:
47% reduction
in model weight file size.
Why Q8_0?
The goal was not maximum compression.
Lower-bit quantization formats such as Q4, Q5, or Q6 can produce smaller model files.
However, they also apply stronger compression to model weights and may introduce greater quantization error.
Q8_0 was selected to balance:
Weight Precision
Model Quality
Memory Usage
VRAM Usage
Inference Efficiency
The objective was:
Reduce memory requirements
+
Preserve high weight precision
+
Keep inference quality close to higher-precision weights
Recommended Settings
temperature: 1.0
top_p: 0.95
top_k: -1
repeat_penalty: 1.0
context: 32768
max output tokens: 8192
For GPUs with approximately 8 GB VRAM:
Recommended starting context:
16K–32K
Tested Hardware
GPU: NVIDIA GeForce RTX 5070
VRAM: ~8 GB
CUDA Toolkit: 13.2
Quantization: Q8_0
Recommended Context: 16K–32K
Operating System: Windows 11
Actual memory usage depends on:
- Context length
- KV cache size
- Batch size
- GPU backend
- Other applications using GPU memory
- Runtime configuration
Checking GPU Usage
Check Ollama:
ollama ps
Check NVIDIA GPU usage:
nvidia-smi
GPU memory usage should increase after the model is loaded.
Architecture
This model uses:
spark2_5
A runtime with Spark-X2.5 architecture support is required.
If you receive:
unknown model architecture: 'spark2_5'
use the Spark-compatible Ollama build instructions above.
Original Model
This community quantized release is based on the official:
XHToken — Spark-X2.5-4B
Original model:
https://huggingface.co/XHToken/Spark-X2.5-4B
Official project:
https://github.com/XHToken/Spark-X2.5
All credit for the original:
- Model architecture
- Training
- Original weights
- Research
- Base model development
belongs to XHToken and the original Spark-X2.5 contributors.
Yousef Burhan Ahmad Contribution
This community release was:
Converted to GGUF, quantized to Q8_0, tested, and packaged by Yousef Burhan Ahmad.
My contribution included:
- Hugging Face to GGUF conversion
- FP16 GGUF generation
- Q8_0 quantization
- Spark-compatible llama.cpp validation
- CUDA runtime preparation
- NVIDIA GPU testing
- Ollama-oriented packaging
- Local deployment testing
- Public distribution preparation
- Runtime compatibility documentation
I do not claim ownership or authorship of the original Spark-X2.5 model.
The original model remains the work of:
XHToken and the Spark-X2.5 contributors.
Model Conversion Pipeline
XHToken/Spark-X2.5-4B
↓
Hugging Face Safetensors
↓
Spark-Compatible llama.cpp
↓
GGUF FP16
↓
Q8_0 Quantization
↓
Spark-X2.5-4B-Q8_0.gguf
↓
CUDA Runtime Testing
↓
Ollama Packaging
↓
Hugging Face Distribution
Distribution
Hugging Face
YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF
https://huggingface.co/YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF
Ollama
yousefburhanahmad/spark-x2.5-4b:q8
Attribution
Original Model
XHToken — Spark-X2.5-4B
Community GGUF Conversion & Q8_0 Quantization
Yousef Burhan Ahmad
Hugging Face Distribution
YousefAhmad121/Spark-X2.5-4B-Q8_0-GGUF
Ollama Distribution
yousefburhanahmad/spark-x2.5-4b:q8
License
Apache License 2.0
License
The original Spark-X2.5-4B model is distributed under the Apache License 2.0.
Please review the original model repository and model card for the authoritative license terms and usage requirements.
Disclaimer
This is an unofficial community GGUF conversion and Q8_0 quantized release by Yousef Burhan Ahmad.
It is not an official XHToken release unless explicitly stated by the original developers.
I do not claim ownership of the original Spark-X2.5 architecture, training methodology, research, or original model weights.
My contribution is specifically:
GGUF Conversion
Q8_0 Quantization
Runtime Preparation
CUDA Testing
Ollama Packaging
Distribution Documentation
For authoritative information about the original model, refer to:
Original model:
https://huggingface.co/XHToken/Spark-X2.5-4B
Official project:
- Downloads last month
- 179
8-bit