Instructions to use jmeadows17/MathT5-large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jmeadows17/MathT5-large with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jmeadows17/MathT5-large")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

tokenizer = AutoTokenizer.from_pretrained("jmeadows17/MathT5-large")
model = AutoModelForSeq2SeqLM.from_pretrained("jmeadows17/MathT5-large", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jmeadows17/MathT5-large with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jmeadows17/MathT5-large"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jmeadows17/MathT5-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jmeadows17/MathT5-large

SGLang

How to use jmeadows17/MathT5-large with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jmeadows17/MathT5-large" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jmeadows17/MathT5-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jmeadows17/MathT5-large" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jmeadows17/MathT5-large",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jmeadows17/MathT5-large with Docker Model Runner:
```
docker model run hf.co/jmeadows17/MathT5-large
```

MathT5-large

Commit History

Jul 26, 2023

04:04
Adding `safetensors` variant of this model⁠

SFconvertbot

ff0bd9e

Jul 21, 2023

23:35
Update README.md

jmeadows17

7073d08
23:33
Upload training_prompts.json⁠

jmeadows17

41c3742
23:27
Update README.md

jmeadows17

14ae145

Jul 3, 2023

05:59
Update README.md

jmeadows17

2342c5c
05:58
Update README.md

jmeadows17

a0952fb

Jul 2, 2023

20:39
Update README.md

jmeadows17

98e291f
20:39
Update README.md

jmeadows17

33a58f1

Jun 29, 2023

23:42
Update MathT5.py

jmeadows17

8f65215
23:30
Update README.md

jmeadows17

ef39e18
23:18
Update README.md

jmeadows17

74f800f
23:16
Update README.md

jmeadows17

bc328a0
23:15
Update README.md

jmeadows17

b9a84e0
23:11
Update README.md

jmeadows17

3da82ae
23:06
Upload MathT5.py

jmeadows17

dab8570
22:51
Update README.md

jmeadows17

a6d13ee
19:39
Update README.md

jmeadows17

a36cb6f
19:36
Update README.md

jmeadows17

c420fdf
19:35
Update README.md

jmeadows17

114a0e7
19:34
Update README.md

jmeadows17

6b66619
19:33
Update README.md

jmeadows17

97a1a0d
19:33
Update README.md

jmeadows17

c25c89a
19:32
Update README.md

jmeadows17

cf1fbcd
19:32
Update README.md

jmeadows17

7f96148
19:31
Update README.md

jmeadows17

da27179
18:51
Upload 6 files

jmeadows17

71a9a2f
18:29
initial commit

jmeadows17

e3804cb

Commit History

Jul 26, 2023

Adding `safetensors` variant of this model⁠

Jul 21, 2023

Update README.md

Upload training_prompts.json⁠

Update README.md

Jul 3, 2023

Update README.md

Update README.md

Jul 2, 2023

Update README.md

Update README.md

Jun 29, 2023

Update MathT5.py

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Upload MathT5.py

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Upload 6 files

initial commit