Instructions to use zjunlp/chatcell-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use zjunlp/chatcell-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="zjunlp/chatcell-large")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("zjunlp/chatcell-large")
model = AutoModelForSeq2SeqLM.from_pretrained("zjunlp/chatcell-large", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use zjunlp/chatcell-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "zjunlp/chatcell-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/chatcell-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/zjunlp/chatcell-large

SGLang

How to use zjunlp/chatcell-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "zjunlp/chatcell-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/chatcell-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "zjunlp/chatcell-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "zjunlp/chatcell-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use zjunlp/chatcell-large with Docker Model Runner:
```
docker model run hf.co/zjunlp/chatcell-large
```

chatcell-large

Commit History

Nov 18, 2024

10:30
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
61e11cc

Feb 19, 2024

11:59
Update README.md

Ningyu

Verified signature
3faf080
08:15
Update README.md

Ningyu

Verified signature
e1bf81a
08:08
Update README.md

Ningyu

Verified signature
bd19b9e
06:40
Update README.md

Ningyu

Verified signature
642027d
06:38
Update README.md

Ningyu

Verified signature
f15df98

Feb 17, 2024

02:22
Update README.md

Ningyu

Verified signature
7ce53b6

Feb 14, 2024

15:51
Create handler.py

Linear-Matrix-Probability

Verified signature
5e439c3
08:20
Update README.md

Yin Fang

Verified signature
6354e14

Feb 13, 2024

10:40
Update README.md

sadgaj

Verified signature
c54400c
07:38
Update README.md

Yin Fang

Verified signature
3d14fc8

Feb 11, 2024

16:26
Upload 8 files

Yin Fang

Verified signature
899cacc
14:06
Upload 8 files

Yin Fang

Verified signature
5b55003
14:03
initial commit

Yin Fang

Verified signature
cca7fbc

Commit History

Nov 18, 2024

Adding `safetensors` variant of this model⁠

Feb 19, 2024

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Feb 17, 2024

Update README.md

Feb 14, 2024

Create handler.py

Update README.md

Feb 13, 2024

Update README.md

Update README.md

Feb 11, 2024

Upload 8 files

Upload 8 files

initial commit