Instructions to use zjunlp/zhixi-13b-diff with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zjunlp/zhixi-13b-diff with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zjunlp/zhixi-13b-diff")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("zjunlp/zhixi-13b-diff")
model = AutoModelForCausalLM.from_pretrained("zjunlp/zhixi-13b-diff", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zjunlp/zhixi-13b-diff with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zjunlp/zhixi-13b-diff"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/zhixi-13b-diff",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/zjunlp/zhixi-13b-diff

SGLang

How to use zjunlp/zhixi-13b-diff with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zjunlp/zhixi-13b-diff" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/zhixi-13b-diff",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zjunlp/zhixi-13b-diff" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/zhixi-13b-diff",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use zjunlp/zhixi-13b-diff with Docker Model Runner:
```
docker model run hf.co/zjunlp/zhixi-13b-diff
```

zhixi-13b-diff

Commit History

May 10, 2025

21:49
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
34eab8b

Jun 22, 2023

02:43
Update README.md

liangxz

6c3aaf3

Jun 21, 2023

09:02
Update README.md

Ningyu

0f812a1

Jun 19, 2023

06:24
Update README.md

wangxh07

71e6977
06:23
Update README.md

wangxh07

ced6ff7
06:15
Update README.md

liangxz

a4f5093
05:24
Update README.md

Ningyu

74a41f0
04:50
Update README.md

wangxh07

9d1dff5
03:52
Update README.md

wangxh07

353b4d8
03:40
Update README.md

wangxh07

ff931de
03:38
Update README.md

wangxh07

c808e0b
03:35
Update README.md

wangxh07

c30ffa3

Jun 18, 2023

11:44
Upload zhixi_logo.png

Ningyu

3e83f2b

May 26, 2023

03:05
Update README.md

MikeDean

5edb68d

May 25, 2023

14:31
Update README.md

MikeDean

c07e1c0
14:30
Update README.md

MikeDean

47fe80d
14:29
Update README.md

MikeDean

f5606e7
14:26
Create README.md

MikeDean

0eceb86
05:58
Upload pytorch_model.bin.index.json with huggingface_hub

MikeDean

ee257c6

May 24, 2023

03:45
Upload special_tokens_map.json with huggingface_hub

MikeDean

215a66b
03:45
Upload tokenizer_config.json with huggingface_hub

MikeDean

58a8e42
03:45
Upload tokenizer.model with huggingface_hub

MikeDean

42da2c7
03:45
Upload generation_config.json with huggingface_hub

MikeDean

3120db9
03:45
Upload config.json with huggingface_hub

MikeDean

e0608af

May 23, 2023

20:02
Upload pytorch_model-00006-of-00006.bin with huggingface_hub

MikeDean

05cdb69
19:55
Upload pytorch_model-00005-of-00006.bin with huggingface_hub

MikeDean

970ba4a
19:09
Upload pytorch_model-00004-of-00006.bin with huggingface_hub

MikeDean

3ac72f5
18:32
Upload pytorch_model-00003-of-00006.bin with huggingface_hub

MikeDean

d2be6ce
17:52
Upload pytorch_model-00002-of-00006.bin with huggingface_hub

MikeDean

240f2a8
17:11
Upload pytorch_model-00001-of-00006.bin with huggingface_hub

MikeDean

40aeb13
15:43
initial commit

MikeDean

17b2258

Commit History

May 10, 2025

Adding `safetensors` variant of this model⁠

Jun 22, 2023

Update README.md

Jun 21, 2023

Update README.md

Jun 19, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Jun 18, 2023

Upload zhixi_logo.png

May 26, 2023

Update README.md

May 25, 2023

Update README.md

Update README.md

Update README.md

Create README.md

Upload pytorch_model.bin.index.json with huggingface_hub

May 24, 2023

Upload special_tokens_map.json with huggingface_hub

Upload tokenizer_config.json with huggingface_hub

Upload tokenizer.model with huggingface_hub

Upload generation_config.json with huggingface_hub

Upload config.json with huggingface_hub

May 23, 2023

Upload pytorch_model-00006-of-00006.bin with huggingface_hub

Upload pytorch_model-00005-of-00006.bin with huggingface_hub

Upload pytorch_model-00004-of-00006.bin with huggingface_hub

Upload pytorch_model-00003-of-00006.bin with huggingface_hub

Upload pytorch_model-00002-of-00006.bin with huggingface_hub

Upload pytorch_model-00001-of-00006.bin with huggingface_hub

initial commit