Instructions to use XHToken/Spark-X2.5-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use XHToken/Spark-X2.5-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XHToken/Spark-X2.5-4B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("XHToken/Spark-X2.5-4B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XHToken/Spark-X2.5-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XHToken/Spark-X2.5-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XHToken/Spark-X2.5-4B

SGLang

How to use XHToken/Spark-X2.5-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XHToken/Spark-X2.5-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XHToken/Spark-X2.5-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.5-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XHToken/Spark-X2.5-4B with Docker Model Runner:
```
docker model run hf.co/XHToken/Spark-X2.5-4B
```

Benchmark against Nanbeige 4.2 3B and/or Agents A1 4B?

by EnderOLED - opened Sep 1

Discussion

EnderOLED

Sep 1

•

edited Sep 11

Nanbeige, Agents, and now Spark X2.5 4B are all competing for the same 3B-4B SOTA spot. Would love to see someone benchmark them all!

Edit:
More models:

Nanbeige 4.2 3B
Agents A1 4B
Spark X2.5 4B
LFM 2.5 2.6B
LFM 2.5 3B VL
Granite 4.2 3B
MiniCPM 5 2B

Tylerwtzhang3

SparkLLM org Sep 2

Would love to see someone benchmark them too.

engrtipusultan

Sep 2

Why not LFM? They are SOTA in this range

EnderOLED

Sep 3

Why not LFM? They are SOTA in this range

Sure, LFM 2.5 2.6B and maybe LFM 2.5 3B VL too.

engrtipusultan

Sep 3

LFM has proper llama.cpp support. This team did not open PR to add support. So user base will remain low.
Also they have QAD ggufs with dspark speculative decoding so it is super fast as edge models should be.