Instructions to use FreedomIntelligence/AceGPT-7B-chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FreedomIntelligence/AceGPT-7B-chat with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FreedomIntelligence/AceGPT-7B-chat")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/AceGPT-7B-chat")
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/AceGPT-7B-chat", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FreedomIntelligence/AceGPT-7B-chat with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FreedomIntelligence/AceGPT-7B-chat"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FreedomIntelligence/AceGPT-7B-chat

SGLang

How to use FreedomIntelligence/AceGPT-7B-chat with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FreedomIntelligence/AceGPT-7B-chat" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FreedomIntelligence/AceGPT-7B-chat" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B-chat",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FreedomIntelligence/AceGPT-7B-chat with Docker Model Runner:
```
docker model run hf.co/FreedomIntelligence/AceGPT-7B-chat
```

AceGPT-7B-chat

Commit History

Apr 7, 2024

17:24
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
c084883

Mar 4, 2024

02:23
Update README.md

Benyou

Verified signature
252565c
02:22
Update README.md

Benyou

Verified signature
3a28f67
02:21
Update README.md

Benyou

Verified signature
d02df60

Dec 1, 2023

16:10
Delete pytorch_model.bin.index.json

jianqing666

2b45551
16:09
Delete pytorch_model-00003-of-00003.bin

jianqing666

405bdd6
16:09
Delete pytorch_model-00002-of-00003.bin

jianqing666

c7c6684
16:09
Delete pytorch_model-00001-of-00003.bin

jianqing666

fece925
15:59
Upload pytorch_model.bin⁠

jianqing666

a88b515
14:58
Upload 2 files⁠

jianqing666

67ae819

Sep 26, 2023

08:15
Upload LICENSE.txt

hh2017

80b1d19

Sep 22, 2023

06:07
Upload LlamaForCausalLM

sxn6144

359d71c
06:05
Upload tokenizer

sxn6144

9630177
03:59
Upload LlamaForCausalLM

sxn6144

48e829e
03:32
Upload tokenizer

sxn6144

71a81ba

Sep 21, 2023

12:47
Update README.md

jianqing666

e75a07a
08:58
Update README.md

sxn6144

4cb711c
08:40
Update README.md

sxn6144

1ca4341
07:42
Update README.md

sxn6144

7b36a86
07:41
Update README.md

sxn6144

acd6394
07:28
Upload README.md

sxn6144

4a8e94e
06:25
Upload LlamaForSequenceClassification

sxn6144

bd98979
06:06
Upload tokenizer

sxn6144

c6da984
06:05
initial commit

sxn6144

3aa5729

Commit History

Apr 7, 2024

Adding `safetensors` variant of this model⁠

Mar 4, 2024

Update README.md

Update README.md

Update README.md

Dec 1, 2023

Delete pytorch_model.bin.index.json

Delete pytorch_model-00003-of-00003.bin

Delete pytorch_model-00002-of-00003.bin

Delete pytorch_model-00001-of-00003.bin

Upload pytorch_model.bin⁠

Upload 2 files⁠

Sep 26, 2023

Upload LICENSE.txt

Sep 22, 2023

Upload LlamaForCausalLM

Upload tokenizer

Upload LlamaForCausalLM

Upload tokenizer

Sep 21, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Upload README.md

Upload LlamaForSequenceClassification

Upload tokenizer

initial commit