Instructions to use FreedomIntelligence/AceGPT-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use FreedomIntelligence/AceGPT-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="FreedomIntelligence/AceGPT-7B")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("FreedomIntelligence/AceGPT-7B")
model = AutoModelForCausalLM.from_pretrained("FreedomIntelligence/AceGPT-7B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use FreedomIntelligence/AceGPT-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "FreedomIntelligence/AceGPT-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/FreedomIntelligence/AceGPT-7B

SGLang

How to use FreedomIntelligence/AceGPT-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "FreedomIntelligence/AceGPT-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "FreedomIntelligence/AceGPT-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "FreedomIntelligence/AceGPT-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use FreedomIntelligence/AceGPT-7B with Docker Model Runner:
```
docker model run hf.co/FreedomIntelligence/AceGPT-7B
```

AceGPT-7B

Commit History

Mar 31, 2024

03:11
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
58dc382

Nov 29, 2023

16:09
Upload 6 files⁠

jianqing666

c907e27
16:05
Delete pytorch_model.bin.index.json

jianqing666

51a28ee
16:05
Delete pytorch_model-00003-of-00003.bin

jianqing666

fa5c662
16:05
Delete pytorch_model-00002-of-00003.bin

jianqing666

3ff711e
16:04
Delete pytorch_model-00001-of-00003.bin

jianqing666

96ebabb
16:04
Upload pytorch_model.bin⁠

jianqing666

ecac0f4

Sep 26, 2023

08:14
Upload LICENSE.txt

hh2017

6b241c4

Sep 22, 2023

07:55
Upload LlamaForCausalLM

sxn6144

331c841
07:54
Upload tokenizer

sxn6144

b56f70e
07:54
Upload tokenizer

sxn6144

8e56f45
07:53
Delete tokenizer_config.json

sxn6144

95d0c58
07:53
Delete config.json

sxn6144

30d3d79
07:53
Delete pytorch_model-00003-of-00003.bin

sxn6144

f2ef242
07:53
Delete pytorch_model-00002-of-00003.bin

sxn6144

0978776
07:53
Delete pytorch_model-00001-of-00003.bin

sxn6144

a69fd9d
07:51
Delete generation_config.json

sxn6144

bd1c797
07:51
Delete tokenizer.model

sxn6144

5e09324
07:51
Delete tokenizer.json

sxn6144

b76dc57
07:50
Delete pytorch_model.bin.index.json

sxn6144

b499984
07:50
Delete special_tokens_map.json

sxn6144

b8fc258
06:11
Upload LlamaForCausalLM

sxn6144

d28f3a0
06:10
Upload tokenizer

sxn6144

1281977
05:37
Delete pytorch_model.bin

sxn6144

7d6948a
04:41
Upload LlamaForCausalLM

sxn6144

589d7ff
04:17
Upload tokenizer

sxn6144

c9d479f

Sep 21, 2023

12:48
Update README.md

jianqing666

edd80fe
08:59
Update README.md

sxn6144

770a7e2
08:34
Update README.md

sxn6144

a0b3cef
08:33
Update README.md

sxn6144

6d15fa6

Sep 18, 2023

06:19
Update README.md

sxn6144

d111416
06:03
Update README.md

sxn6144

07e9b24
06:02
Update README.md

sxn6144

10b98d3
06:02
Update README.md

sxn6144

2feec51
05:39
Update README.md

sxn6144

0281286

Sep 17, 2023

08:33
Update README.md

sxn6144

7c769e8
08:33
Create README.md⁠

sxn6144

1e0cfac
06:36
Delete README.md

sxn6144

3b6e1bd

Sep 15, 2023

04:24
del some files

jianqing666

a07d6c5

Sep 14, 2023

09:37
ACEGPT-first-put

jianqing666

5ddd685
08:56
initial commit

sxn6144

35b642c

Commit History

Mar 31, 2024

Adding `safetensors` variant of this model⁠

Nov 29, 2023

Upload 6 files⁠

Delete pytorch_model.bin.index.json

Delete pytorch_model-00003-of-00003.bin

Delete pytorch_model-00002-of-00003.bin

Delete pytorch_model-00001-of-00003.bin

Upload pytorch_model.bin⁠

Sep 26, 2023

Upload LICENSE.txt

Sep 22, 2023

Upload LlamaForCausalLM

Upload tokenizer

Upload tokenizer

Delete tokenizer_config.json

Delete config.json

Delete pytorch_model-00003-of-00003.bin

Delete pytorch_model-00002-of-00003.bin

Delete pytorch_model-00001-of-00003.bin

Delete generation_config.json

Delete tokenizer.model

Delete tokenizer.json

Delete pytorch_model.bin.index.json

Delete special_tokens_map.json

Upload LlamaForCausalLM

Upload tokenizer

Delete pytorch_model.bin

Upload LlamaForCausalLM

Upload tokenizer

Sep 21, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Sep 18, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Sep 17, 2023

Update README.md

Create README.md⁠

Delete README.md

Sep 15, 2023

del some files

Sep 14, 2023

ACEGPT-first-put

initial commit