Instructions to use distilbert/distilgpt2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use distilbert/distilgpt2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="distilbert/distilgpt2")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("distilbert/distilgpt2")
model = AutoModelForCausalLM.from_pretrained("distilbert/distilgpt2", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use distilbert/distilgpt2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "distilbert/distilgpt2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "distilbert/distilgpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/distilbert/distilgpt2

SGLang

How to use distilbert/distilgpt2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "distilbert/distilgpt2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "distilbert/distilgpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "distilbert/distilgpt2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "distilbert/distilgpt2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use distilbert/distilgpt2 with Docker Model Runner:
```
docker model run hf.co/distilbert/distilgpt2
```

distilgpt2

Commit History

Feb 19, 2024

11:09
Adds the tokenizer configuration file (#11)⁠

lysandre HF Staff

Verified signature
2290a62

Apr 29, 2023

12:24
Add Core ML conversion (#10)⁠

pcuenq HF Staff

38cc92e

Jan 24, 2023

13:57
Adding generation config file(s)

joaogante

f241065
13:55
Delete generation_config.json

joaogante

a748e05
13:55
Delete {pipeline}_generation_config.json

joaogante

32007bf
13:52
Adding generation config file(s)

joaogante

9e4cf98

Nov 14, 2022

21:36
Adding `safetensors` variant of this model (#7)⁠

lysandre +1: Narsil HF Staff

ee0b29d

Sep 5, 2022

17:45
Add CoreML model (#6)⁠

julien-c HF Staff

c3772e6

Jul 22, 2022

08:12
Fix emissions metadata (#5)⁠

sgugger +1: Marissa

ca98be8

Jun 2, 2022

08:10
Limit use of collapsible sections; fix emissions info (#3)⁠

julien-c +1: Marissa HF Staff

b754fe8

May 26, 2022

01:10
New model card for distilgpt2 (#1)⁠

patrickvonplaten +1: Marissa

737e343

May 21, 2021

09:15
upload flax model

patrickvonplaten

96699c9
09:15
allow flax

patrickvonplaten

2ddd647
09:09
allow flax

patrickvonplaten

92838a2

Dec 11, 2020

21:24
Migrate model card from transformers-repo⁠

julien-c HF Staff

9851938

Dec 9, 2020

18:29
For clarity, delete deprecated modelcard.json⁠

julien-c HF Staff

58ada35

Oct 12, 2020

12:56
Update tokenizer.json

system HF Staff

b6d23df

May 11, 2020

21:02
Update config.json

system HF Staff

e87a92b

Apr 24, 2020

19:42
Update rust_model.ot

system HF Staff

17ef8ca
15:58
Update config.json

system HF Staff

b09826e

Jan 31, 2020

23:00
Update config.json

system HF Staff

51ab088

Dec 20, 2019

12:28
Update modelcard.json

system HF Staff

baa9aea

Nov 29, 2019

21:18
Update 64.tflite

system HF Staff

2677794

Oct 7, 2019

16:29
Update pytorch_model.bin

system HF Staff

36840f1
16:29
Update tf_model.h5

system HF Staff

b70d85d

Oct 3, 2019

18:46
Update config.json

system HF Staff

0799984
14:08
Update merges.txt

system HF Staff

0c7d1e1
14:08
Update config.json

system HF Staff

09daaa0
14:08
Update vocab.json

system HF Staff

db73fa3
14:08
initial commit

system HF Staff

1b04b2c

Commit History

Feb 19, 2024

Adds the tokenizer configuration file (#11)⁠

Apr 29, 2023

Add Core ML conversion (#10)⁠

Jan 24, 2023

Adding generation config file(s)

Delete generation_config.json

Delete {pipeline}_generation_config.json

Adding generation config file(s)

Nov 14, 2022

Adding `safetensors` variant of this model (#7)⁠

Sep 5, 2022

Add CoreML model (#6)⁠

Jul 22, 2022

Fix emissions metadata (#5)⁠

Jun 2, 2022

Limit use of collapsible sections; fix emissions info (#3)⁠

May 26, 2022

New model card for distilgpt2 (#1)⁠

May 21, 2021

upload flax model

allow flax

allow flax

Dec 11, 2020

Migrate model card from transformers-repo⁠

Dec 9, 2020

For clarity, delete deprecated modelcard.json⁠

Oct 12, 2020

Update tokenizer.json

May 11, 2020

Update config.json

Apr 24, 2020

Update rust_model.ot

Update config.json

Jan 31, 2020

Update config.json

Dec 20, 2019

Update modelcard.json

Nov 29, 2019

Update 64.tflite

Oct 7, 2019

Update pytorch_model.bin

Update tf_model.h5

Oct 3, 2019

Update config.json

Update merges.txt

Update config.json

Update vocab.json

initial commit