Instructions to use lightblue/openorca_stx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lightblue/openorca_stx with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lightblue/openorca_stx")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lightblue/openorca_stx")
model = AutoModelForCausalLM.from_pretrained("lightblue/openorca_stx", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lightblue/openorca_stx with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lightblue/openorca_stx"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightblue/openorca_stx",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/lightblue/openorca_stx

SGLang

How to use lightblue/openorca_stx with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lightblue/openorca_stx" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightblue/openorca_stx",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lightblue/openorca_stx" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lightblue/openorca_stx",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use lightblue/openorca_stx with Docker Model Runner:
```
docker model run hf.co/lightblue/openorca_stx
```

openorca_stx

Commit History

Nov 7, 2024

04:05
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
7e713cf

Oct 2, 2023

10:25
Update README.md

ptrdvn

db29ec0
10:24
Update README.md

ptrdvn

5ab010b
10:15
Update README.md

ptrdvn

d33dcfe

Sep 27, 2023

05:04
Update generation_config.json

ptrdvn

539b8bf
04:36
Update README.md

ptrdvn

7b833f2
04:34
Update README.md

ptrdvn

f6d7433
04:34
Update README.md

ptrdvn

26abdeb
04:33
Update README.md

ptrdvn

8c1ce9a

Sep 14, 2023

01:53
Update README.md

ptrdvn

f72c6f1

Sep 13, 2023

07:05
Update README.md

ptrdvn

c2fc322
02:54
Update README.md

ptrdvn

26a395b
02:53
Update README.md

ptrdvn

ddde9aa

Sep 12, 2023

10:16
Update README.md

ptrdvn

3cefbdb
10:07
Update README.md

ptrdvn

8da1560
10:05
Update README.md

ptrdvn

51ca017
10:04
Update README.md

ptrdvn

608ede5
10:03
Create README.md

ptrdvn

f503b95
09:48
Upload LlamaForCausalLM

ptrdvn

587d621
09:35
Upload tokenizer

ptrdvn

de6de69
09:29
initial commit

ptrdvn

db407a5

Commit History

Nov 7, 2024

Adding `safetensors` variant of this model⁠

Oct 2, 2023

Update README.md

Update README.md

Update README.md

Sep 27, 2023

Update generation_config.json

Update README.md

Update README.md

Update README.md

Update README.md

Sep 14, 2023

Update README.md

Sep 13, 2023

Update README.md

Update README.md

Update README.md

Sep 12, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Create README.md

Upload LlamaForCausalLM

Upload tokenizer

initial commit