Instructions to use XHToken/Spark-X2.6-4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use XHToken/Spark-X2.6-4B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="XHToken/Spark-X2.6-4B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("XHToken/Spark-X2.6-4B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use XHToken/Spark-X2.6-4B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "XHToken/Spark-X2.6-4B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.6-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/XHToken/Spark-X2.6-4B

SGLang

How to use XHToken/Spark-X2.6-4B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "XHToken/Spark-X2.6-4B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.6-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "XHToken/Spark-X2.6-4B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "XHToken/Spark-X2.6-4B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use XHToken/Spark-X2.6-4B with Docker Model Runner:
```
docker model run hf.co/XHToken/Spark-X2.6-4B
```

Unbelievable performance improvements on v2.6! (Overthinking issue fixed)

by aidinmc - opened 19 days ago

Discussion

aidinmc

19 days ago

Just wanted to share a quick feedback thread in case it saves someone some time.
I spent a lot of time struggling to manage reasoning/overthinking behavior on v2.5, but couldn't get consistent results:

Uncapped thinking budget: It would spend nearly 20 minutes reasoning.
Strict budget cap: The output quality degraded significantly.

However, v2.6 seems to have completely resolved this issue! It honestly punches way above its weight class—performing on par with +100B models.
I haven't stress-tested it with extremely heavy workflows yet, but it executed all my prompt tasks almost flawlessly without any serious hiccups.

Huge thanks to the maintainers/devs for this release! Fantastic work. 🚀

dandandelion

17 days ago

It definitely punches above its weight class but idk about 100B+ claim

sverinn

7 days ago

•

edited 7 days ago

I don't know about that, in my setup, after it spends reasoning budget, it just starts thinking the same way but in plain output stream. This model looooves to yap. Still impressive for its size

aidinmc

4 days ago

Let me clarify one thing:

As I said, I didn't give the model any super heavy tasks; but it's genuinely impressive that a small model is aware — for example, catching the cases of an enum that was defined elsewhere under a different name, and instead of just creating a brand-new enum, it uses the question tool to ask for my opinion. That's really cool.

Basically, what I meant was that its behavior felt like a +100B model, not necessarily its problem-solving ability.

The second thing is that the code it writes is almost always complete and free of syntax errors. Same-weight models tend to have a common flaw — they deliver code with syntax mistakes or incomplete — which this model didn't seem to have.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment