Instructions to use aubmindlab/aragpt2-mega with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use aubmindlab/aragpt2-mega with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="aubmindlab/aragpt2-mega", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("aubmindlab/aragpt2-mega", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use aubmindlab/aragpt2-mega with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "aubmindlab/aragpt2-mega"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aubmindlab/aragpt2-mega",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/aubmindlab/aragpt2-mega

SGLang

How to use aubmindlab/aragpt2-mega with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "aubmindlab/aragpt2-mega" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aubmindlab/aragpt2-mega",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "aubmindlab/aragpt2-mega" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "aubmindlab/aragpt2-mega",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use aubmindlab/aragpt2-mega with Docker Model Runner:
```
docker model run hf.co/aubmindlab/aragpt2-mega
```

aragpt2-mega

Commit History

Aug 24, 2023

18:35
Adding `safetensors` variant of this model⁠

SFconvertbot

77bab6e

Nov 15, 2022

11:09
Update README.md

wissamantoun

f39a1bd

Apr 7, 2022

11:43
added tensorboard logs

wissamantoun

d7de129

Jul 31, 2021

16:33
Update README.md

wissamantoun

1ad45c4

May 21, 2021

13:49
allow flax

patrickvonplaten

4c65fd0

Apr 18, 2021

21:10
added citation

wissamantoun

89e59b0

Jan 13, 2021

19:04
Update README.md

AUB MIND LAB

a2387e4

Jan 1, 2021

12:02
Added model files

aubmindlab

2a332cf
11:22
initial commit

system HF Staff

a27f413

Commit History

Aug 24, 2023

Adding `safetensors` variant of this model⁠

Nov 15, 2022

Update README.md

Apr 7, 2022

added tensorboard logs

Jul 31, 2021

Update README.md

May 21, 2021

allow flax

Apr 18, 2021

added citation

Jan 13, 2021

Update README.md

Jan 1, 2021

Added model files

initial commit