Instructions to use KoboldAI/PPO_Pygway-6b-Mix with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KoboldAI/PPO_Pygway-6b-Mix with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KoboldAI/PPO_Pygway-6b-Mix")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("KoboldAI/PPO_Pygway-6b-Mix")
model = AutoModelForCausalLM.from_pretrained("KoboldAI/PPO_Pygway-6b-Mix", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KoboldAI/PPO_Pygway-6b-Mix with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KoboldAI/PPO_Pygway-6b-Mix"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KoboldAI/PPO_Pygway-6b-Mix",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/KoboldAI/PPO_Pygway-6b-Mix

SGLang

How to use KoboldAI/PPO_Pygway-6b-Mix with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KoboldAI/PPO_Pygway-6b-Mix" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KoboldAI/PPO_Pygway-6b-Mix",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KoboldAI/PPO_Pygway-6b-Mix" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KoboldAI/PPO_Pygway-6b-Mix",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use KoboldAI/PPO_Pygway-6b-Mix with Docker Model Runner:
```
docker model run hf.co/KoboldAI/PPO_Pygway-6b-Mix
```

PPO_Pygway-6b-Mix

Commit History

Mar 24, 2024

11:49
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
a2af604

Mar 27, 2023

22:34
Update README.md

TehVenom

b31d258
22:32
Update README.md

TehVenom

84fe278
22:32
Update README.md

TehVenom

346d759
22:24
Update README.md

TehVenom

0322ec7
22:22
Update README.md

TehVenom

1c28a64
22:16
Update README.md

TehVenom

464f88e
22:09
Update README.md

TehVenom

fe89b4a
22:05
Update README.md

TehVenom

85d4d0e
22:04
Update README.md

TehVenom

7104caf
19:49
Update README.md

TehVenom

3cf067a
19:20
Update README.md

TehVenom

eb010f1

Mar 23, 2023

01:37
Update config.json

TehVenom

6f24e8f

Mar 15, 2023

03:46
Update README.md

TehVenom

4f64f1c
03:46
Update README.md

TehVenom

a970269
03:45
Update README.md

TehVenom

886da05

Mar 14, 2023

20:03
20|20|60 = Janeway|PPO_HH|Pygmalion

TehVenom

b26c45d
19:15
Create README.md

TehVenom

33e41d6
19:13
initial commit

TehVenom

cacf3f9

Commit History

Mar 24, 2024

Adding `safetensors` variant of this model⁠

Mar 27, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Mar 23, 2023

Update config.json

Mar 15, 2023

Update README.md

Update README.md

Update README.md

Mar 14, 2023

20|20|60 = Janeway|PPO_HH|Pygmalion

Create README.md

initial commit