Instructions to use seeweb/SeewebLLM-it with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use seeweb/SeewebLLM-it with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="seeweb/SeewebLLM-it")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("seeweb/SeewebLLM-it")
model = AutoModelForCausalLM.from_pretrained("seeweb/SeewebLLM-it", device_map="auto")

Inference
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use seeweb/SeewebLLM-it with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "seeweb/SeewebLLM-it"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "seeweb/SeewebLLM-it",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/seeweb/SeewebLLM-it

SGLang

How to use seeweb/SeewebLLM-it with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "seeweb/SeewebLLM-it" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "seeweb/SeewebLLM-it",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "seeweb/SeewebLLM-it" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "seeweb/SeewebLLM-it",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use seeweb/SeewebLLM-it with Docker Model Runner:
```
docker model run hf.co/seeweb/SeewebLLM-it
```

SeewebLLM-it

Commit History

Jan 14, 2025

13:55
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
a50defb

Aug 22, 2023

06:46
Update README.md

itsrocchi

8e69389
06:45
Update README.md

itsrocchi

3047a3b

Aug 21, 2023

10:33
Update README.md

itsrocchi

a997584
10:26
Update README.md

itsrocchi

01fc5b7
10:24
Update README.md

itsrocchi

a6d1714
10:23
Update README.md

itsrocchi

1f60be9
10:13
Update README.md

itsrocchi

4bc5150
10:13
Update README.md

itsrocchi

6158502
10:11
Update README.md

itsrocchi

c883dca
10:11
Update README.md

itsrocchi

b1702c6
09:51
Update README.md

itsrocchi

a64a1ca
09:50
Update README.md

itsrocchi

df75952
08:16
Update README.md

itsrocchi

9687a4e
08:15
Update README.md

itsrocchi

ddb2d80
08:14
Create README.md

itsrocchi

e429ceb

Aug 18, 2023

14:38
Upload tokenizer

itsrocchi

ae6890d
14:38
Upload LlamaForCausalLM

itsrocchi

d78dad6
14:14
initial commit

itsrocchi

542f34e

Commit History

Jan 14, 2025

Adding `safetensors` variant of this model⁠

Aug 22, 2023

Update README.md

Update README.md

Aug 21, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Create README.md

Aug 18, 2023

Upload tokenizer

Upload LlamaForCausalLM

initial commit