Instructions to use benjamin/gerpt2-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use benjamin/gerpt2-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="benjamin/gerpt2-large")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("benjamin/gerpt2-large")
model = AutoModelForCausalLM.from_pretrained("benjamin/gerpt2-large", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use benjamin/gerpt2-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "benjamin/gerpt2-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "benjamin/gerpt2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/benjamin/gerpt2-large

SGLang

How to use benjamin/gerpt2-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "benjamin/gerpt2-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "benjamin/gerpt2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "benjamin/gerpt2-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "benjamin/gerpt2-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use benjamin/gerpt2-large with Docker Model Runner:
```
docker model run hf.co/benjamin/gerpt2-large
```

benjamin commited on Dec 27, 2020

Commit

5b7a90a

1 Parent(s): 1ec98f3

max_length -> 100

Browse files

Files changed (2) hide show

README.md +5 -3
config.json +1 -1

README.md CHANGED Viewed

@@ -11,6 +11,8 @@ license: mit
 A large German GPT2.
 See the [GPT2 model card](https://huggingface.co/gpt2) for considerations on limitations and bias. See the [GPT2 documentation](https://huggingface.co/transformers/model_doc/gpt2.html) for details on GPT2.
 ## Comparison to [dbmdz/german-gpt2](https://huggingface.co/dbmdz/german-gpt2)
@@ -61,8 +63,8 @@ print(tokenizer.decode(output))
 ## Training details
-GerPT2 is trained on the entire German data (67GB) from the [CC-100 Corpus](http://data.statmt.org/cc-100/) and weights were initialized from the [English GPT2 model](https://huggingface.co/gpt2-large).
-GerPT2 was trained with:
 - a batch size of 256
 - using OneCycle learning rate with a maximum of 5e-3
@@ -71,7 +73,7 @@ GerPT2 was trained with:
 Training took roughly 12 days on 8 TPUv3 cores.
-To train GerPT2, follow these steps. Scripts are located in the [Github repository](https://github.com/bminixhofer/gerpt2):
 0. Download and unzip training data from http://data.statmt.org/cc-100/.
 1. Train a tokenizer using `prepare/train_tokenizer.py`. As training data for the tokenizer I used a random subset of 5% of the CC-100 data.

 A large German GPT2.
+Also check out [GerPT2](https://huggingface.co/benjamin/gerpt2), a small version of this model.
 See the [GPT2 model card](https://huggingface.co/gpt2) for considerations on limitations and bias. See the [GPT2 documentation](https://huggingface.co/transformers/model_doc/gpt2.html) for details on GPT2.
 ## Comparison to [dbmdz/german-gpt2](https://huggingface.co/dbmdz/german-gpt2)
 ## Training details
+GerPT2-large is trained on the entire German data (67GB) from the [CC-100 Corpus](http://data.statmt.org/cc-100/) and weights were initialized from the [English GPT2 model](https://huggingface.co/gpt2-large).
+GerPT2-large was trained with:
 - a batch size of 256
 - using OneCycle learning rate with a maximum of 5e-3
 Training took roughly 12 days on 8 TPUv3 cores.
+To train GerPT2-large, follow these steps. Scripts are located in the [Github repository](https://github.com/bminixhofer/gerpt2):
 0. Download and unzip training data from http://data.statmt.org/cc-100/.
 1. Train a tokenizer using `prepare/train_tokenizer.py`. As training data for the tokenizer I used a random subset of 5% of the CC-100 data.

config.json CHANGED Viewed

@@ -32,7 +32,7 @@
   "task_specific_params": {
     "text-generation": {
       "do_sample": true,
-      "max_length": 500
     }
   },
   "vocab_size": 50257

   "task_specific_params": {
     "text-generation": {
       "do_sample": true,
+      "max_length": 100
     }
   },
   "vocab_size": 50257