Instructions to use FreedomIntelligence/AceGPT-13B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FreedomIntelligence/AceGPT-13B-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FreedomIntelligence/AceGPT-13B-chat")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/AceGPT-13B-chat")
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/AceGPT-13B-chat", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FreedomIntelligence/AceGPT-13B-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FreedomIntelligence/AceGPT-13B-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FreedomIntelligence/AceGPT-13B-chat

SGLang

How to use FreedomIntelligence/AceGPT-13B-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FreedomIntelligence/AceGPT-13B-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FreedomIntelligence/AceGPT-13B-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-13B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FreedomIntelligence/AceGPT-13B-chat with Docker Model Runner:
```
docker model run hf.co/FreedomIntelligence/AceGPT-13B-chat
```

AceGPT-13B-chat

Commit History

May 6, 2024

09:11
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
e7c5385

Dec 1, 2023

23:32
Delete pytorch_model.bin.index.json

jianqing666

ab87ccb
23:30
Delete pytorch_model-00006-of-00006.bin

jianqing666

a38f6ce
23:30
Delete pytorch_model-00005-of-00006.bin

jianqing666

d593c41
23:30
Delete pytorch_model-00004-of-00006.bin

jianqing666

b35610e
23:29
Delete pytorch_model-00003-of-00006.bin

jianqing666

262e2e2
23:29
Delete pytorch_model-00002-of-00006.bin

jianqing666

4551a72
23:29
Delete pytorch_model-00001-of-00006.bin

jianqing666

831fd69
19:26
Upload pytorch_model.bin⁠

jianqing666

0efc20d
16:30
Upload 2 files

jianqing666

eabdecd

Sep 26, 2023

08:15
Upload LICENSE.txt

hh2017

633dcfb

Sep 22, 2023

06:09
Upload LlamaForCausalLM

sxn6144

7497f50
06:07
Upload tokenizer

sxn6144

8ec9736
04:35
Upload LlamaForCausalLM

sxn6144

c7124da
04:13
Upload tokenizer

sxn6144

9f18e1a
03:23
Upload tokenizer

sxn6144

ab6637e

Sep 21, 2023

12:51
Update README.md

jianqing666

e964366
08:57
Update README.md

sxn6144

90697af
08:55
Update README.md

sxn6144

07bb792
08:41
Update README.md

sxn6144

16b27a5
08:01
Upload LlamaForSequenceClassification

sxn6144

8c4e8e1
07:50
Rename README1.md to README.md

sxn6144

a854fc0
07:44
Upload README1.md

sxn6144

a3a249e
07:29
Upload tokenizer

sxn6144

b76c855
04:45
initial commit

sxn6144

0b37a37

Commit History

May 6, 2024

Adding `safetensors` variant of this model⁠

Dec 1, 2023

Delete pytorch_model.bin.index.json

Delete pytorch_model-00006-of-00006.bin

Delete pytorch_model-00005-of-00006.bin

Delete pytorch_model-00004-of-00006.bin

Delete pytorch_model-00003-of-00006.bin

Delete pytorch_model-00002-of-00006.bin

Delete pytorch_model-00001-of-00006.bin

Upload pytorch_model.bin⁠

Upload 2 files

Sep 26, 2023

Upload LICENSE.txt

Sep 22, 2023

Upload LlamaForCausalLM

Upload tokenizer

Upload LlamaForCausalLM

Upload tokenizer

Upload tokenizer

Sep 21, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Upload LlamaForSequenceClassification

Rename README1.md to README.md

Upload README1.md

Upload tokenizer

initial commit