Instructions to use declare-lab/starling-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use declare-lab/starling-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="declare-lab/starling-7B")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("declare-lab/starling-7B")
model = AutoModelForCausalLM.from_pretrained("declare-lab/starling-7B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use declare-lab/starling-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "declare-lab/starling-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "declare-lab/starling-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/declare-lab/starling-7B

SGLang

How to use declare-lab/starling-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "declare-lab/starling-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "declare-lab/starling-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "declare-lab/starling-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "declare-lab/starling-7B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use declare-lab/starling-7B with Docker Model Runner:
```
docker model run hf.co/declare-lab/starling-7B
```

starling-7B

Commit History

Feb 17

02:54
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
9216ff3

Mar 4, 2024

14:05
Adding Evaluation Results (#1)⁠

soujanyaporia +1: leaderboard-pr-bot

Verified signature
cd5873c

Feb 21, 2024

17:43
Update README.md

RishabhBhardwaj

Verified signature
13094c8
10:26
Update README.md

soujanyaporia

Verified signature
66a1c60
10:25
Update README.md

soujanyaporia

Verified signature
738abf4

Aug 21, 2023

07:34
Update README.md

Declare

837b74b
05:55
Update README.md

soujanyaporia

833aeeb

Aug 20, 2023

23:35
Update README.md

soujanyaporia

09e0324
22:52
Update README.md

soujanyaporia

452d29b
19:03
Update README.md

Declare

89de5d4
18:57
Update README.md

Declare

8d9bf15
01:42
Update README.md

soujanyaporia

3ffbf1b
01:40
Update README.md

soujanyaporia

a1866be
01:35
Update README.md

soujanyaporia

7e303b0
00:56
Update README.md

soujanyaporia

71623b0
00:56
Update README.md

soujanyaporia

bb0f400
00:55
Update README.md

soujanyaporia

6a21650
00:55
Update README.md

soujanyaporia

184fc21
00:52
Update README.md

soujanyaporia

cf4928f
00:47
Update README.md

soujanyaporia

17fc37d

Aug 19, 2023

23:50
Update README.md

soujanyaporia

83bd990
09:27
Update README.md

soujanyaporia

5c47791
01:23
Update README.md

soujanyaporia

3837e13
01:23
Update README.md

soujanyaporia

5666a16
01:22
Update README.md

soujanyaporia

c9c96cf
01:22
Update README.md

soujanyaporia

8993d83

Aug 18, 2023

20:37
adding remaining files

Declare

1249c08
20:26
Upload model shards

Declare

cf2c634
17:56
First commit

Declare

ad43702
16:51
initial commit

RishabhBhardwaj

ee3928c

Commit History

Feb 17

Adding `safetensors` variant of this model⁠

Mar 4, 2024

Adding Evaluation Results (#1)⁠

Feb 21, 2024

Update README.md

Update README.md

Update README.md

Aug 21, 2023

Update README.md

Update README.md

Aug 20, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Aug 19, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Aug 18, 2023

adding remaining files

Upload model shards

First commit

initial commit