Instructions to use BluebrainAI/gpt2-medium-wikitext with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use BluebrainAI/gpt2-medium-wikitext with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="BluebrainAI/gpt2-medium-wikitext")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("BluebrainAI/gpt2-medium-wikitext")
model = AutoModelForCausalLM.from_pretrained("BluebrainAI/gpt2-medium-wikitext", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use BluebrainAI/gpt2-medium-wikitext with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "BluebrainAI/gpt2-medium-wikitext"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BluebrainAI/gpt2-medium-wikitext",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/BluebrainAI/gpt2-medium-wikitext

SGLang

How to use BluebrainAI/gpt2-medium-wikitext with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "BluebrainAI/gpt2-medium-wikitext" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BluebrainAI/gpt2-medium-wikitext",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "BluebrainAI/gpt2-medium-wikitext" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "BluebrainAI/gpt2-medium-wikitext",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use BluebrainAI/gpt2-medium-wikitext with Docker Model Runner:
```
docker model run hf.co/BluebrainAI/gpt2-medium-wikitext
```

gpt2-medium-wikitext / README.md

shivanandmn

Model save

69a0cd3 verified over 1 year ago

preview code

Raw

History Blame Contribute Delete

2.89 kB

	---
	library_name: transformers
	tags:
	- generated_from_trainer
	metrics:
	- accuracy
	- bleu
	model-index:
	- name: gpt2-medium-wikitext
	results: []
	---

	<!-- This model card has been generated automatically according to the information the Trainer had access to. You
	should probably proofread and complete it, then remove this comment. -->

	# gpt2-medium-wikitext

	This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
	It achieves the following results on the evaluation set:
	- Loss: 3.1666
	- Accuracy: 0.4218
	- Perplexity: 23.7262
	- Bleu: 0.1462

	## Model description

	More information needed

	## Intended uses & limitations

	More information needed

	## Training and evaluation data

	More information needed

	## Training procedure

	### Training hyperparameters

	The following hyperparameters were used during training:
	- learning_rate: 0.0001
	- train_batch_size: 64
	- eval_batch_size: 64
	- seed: 42
	- optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
	- lr_scheduler_type: linear
	- lr_scheduler_warmup_ratio: 0.1
	- num_epochs: 5

	### Training results

	\| Training Loss \| Epoch \| Step \| Validation Loss \| Accuracy \| Perplexity \| Bleu \|
	\|:-------------:\|:------:\|:----:\|:---------------:\|:--------:\|:----------:\|:------:\|
	\| 6.078 \| 0.2806 \| 500 \| 5.9534 \| 0.1875 \| 385.0606 \| 0.0310 \|
	\| 5.0653 \| 0.5612 \| 1000 \| 4.9232 \| 0.2616 \| 137.4410 \| 0.0633 \|
	\| 4.3357 \| 0.8418 \| 1500 \| 4.2163 \| 0.3222 \| 67.7828 \| 0.0857 \|
	\| 3.9453 \| 1.1223 \| 2000 \| 3.8824 \| 0.3534 \| 48.5418 \| 0.1107 \|
	\| 3.7572 \| 1.4029 \| 2500 \| 3.7058 \| 0.3684 \| 40.6810 \| 0.1217 \|
	\| 3.6475 \| 1.6835 \| 3000 \| 3.5827 \| 0.3788 \| 35.9700 \| 0.1306 \|
	\| 3.5431 \| 1.9641 \| 3500 \| 3.4927 \| 0.3878 \| 32.8733 \| 0.1347 \|
	\| 3.4221 \| 2.2447 \| 4000 \| 3.4283 \| 0.3939 \| 30.8231 \| 0.1356 \|
	\| 3.36 \| 2.5253 \| 4500 \| 3.3719 \| 0.3996 \| 29.1351 \| 0.1384 \|
	\| 3.3281 \| 2.8058 \| 5000 \| 3.3257 \| 0.4041 \| 27.8193 \| 0.1369 \|
	\| 3.2071 \| 3.0864 \| 5500 \| 3.2885 \| 0.4080 \| 26.8024 \| 0.1442 \|
	\| 3.2002 \| 3.3670 \| 6000 \| 3.2594 \| 0.4117 \| 26.0335 \| 0.1477 \|
	\| 3.1778 \| 3.6476 \| 6500 \| 3.2319 \| 0.4142 \| 25.3278 \| 0.1436 \|
	\| 3.1523 \| 3.9282 \| 7000 \| 3.2091 \| 0.4167 \| 24.7565 \| 0.1462 \|
	\| 3.0842 \| 4.2088 \| 7500 \| 3.1917 \| 0.4185 \| 24.3289 \| 0.1434 \|
	\| 3.0465 \| 4.4893 \| 8000 \| 3.1789 \| 0.4201 \| 24.0197 \| 0.1460 \|
	\| 3.0563 \| 4.7699 \| 8500 \| 3.1666 \| 0.4218 \| 23.7262 \| 0.1462 \|


	### Framework versions

	- Transformers 4.49.0
	- Pytorch 2.6.0+cu124
	- Datasets 3.3.2
	- Tokenizers 0.21.0