Instructions to use TsinghuaAI/CPM-Generate with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use TsinghuaAI/CPM-Generate with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="TsinghuaAI/CPM-Generate")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("TsinghuaAI/CPM-Generate")
model = AutoModelForCausalLM.from_pretrained("TsinghuaAI/CPM-Generate", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use TsinghuaAI/CPM-Generate with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "TsinghuaAI/CPM-Generate"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TsinghuaAI/CPM-Generate",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/TsinghuaAI/CPM-Generate

SGLang

How to use TsinghuaAI/CPM-Generate with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "TsinghuaAI/CPM-Generate" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TsinghuaAI/CPM-Generate",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "TsinghuaAI/CPM-Generate" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "TsinghuaAI/CPM-Generate",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use TsinghuaAI/CPM-Generate with Docker Model Runner:
```
docker model run hf.co/TsinghuaAI/CPM-Generate
```

CPM-Generate

Commit History

Apr 13, 2023

07:54
Adding `safetensors` variant of this model⁠

SFconvertbot

9b81c34

Jul 29, 2021

19:03
upload fast tokenizer

jetrunner

0e4bcd9

May 21, 2021

11:34
allow flax

patrickvonplaten

c782d8f

Apr 10, 2021

14:13
Update config.json

Canwen Xu

feb3a20

Dec 11, 2020

18:10
Update README.md

Canwen Xu

73ef9bd
17:53
Update README.md

Canwen Xu

ce1c7c6
17:38
Update README.md

Canwen Xu

05d7855
17:35
Update README.md

Canwen Xu

4da8245
17:12
Update README.md

Canwen Xu

84626c7
17:12
Create README.md

Canwen Xu

e13d7bc
14:44
Upload files

Canwen Xu

51b8a71

Dec 2, 2020

15:50
initial commit

system HF Staff

7f2cbb9

Commit History

Apr 13, 2023

Adding `safetensors` variant of this model⁠

Jul 29, 2021

upload fast tokenizer

May 21, 2021

allow flax

Apr 10, 2021

Update config.json

Dec 11, 2020

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Create README.md

Upload files

Dec 2, 2020

initial commit