Instructions to use AdaptLLM/medicine-LLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AdaptLLM/medicine-LLM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AdaptLLM/medicine-LLM")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AdaptLLM/medicine-LLM")
model = AutoModelForCausalLM.from_pretrained("AdaptLLM/medicine-LLM", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AdaptLLM/medicine-LLM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AdaptLLM/medicine-LLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/medicine-LLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AdaptLLM/medicine-LLM

SGLang

How to use AdaptLLM/medicine-LLM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AdaptLLM/medicine-LLM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/medicine-LLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AdaptLLM/medicine-LLM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/medicine-LLM",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AdaptLLM/medicine-LLM with Docker Model Runner:
```
docker model run hf.co/AdaptLLM/medicine-LLM
```

medicine-LLM

Commit History

Apr 17, 2024

07:44
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
14bf7c1

Mar 11, 2024

02:41
Update README.md

AdaptLLM

Verified signature
9a639a0

Feb 7, 2024

12:29
Update README.md

AdaptLLM

Verified signature
abd2cb4

Feb 2, 2024

06:11
Update README.md

AdaptLLM

Verified signature
76a0b57

Jan 22, 2024

12:58
Update README.md

AdaptLLM

Verified signature
a7e68c2

Jan 16, 2024

13:42
Update README.md

AdaptLLM

Verified signature
413fa19

Jan 2, 2024

14:29
Update README.md

AdaptLLM

9792066
14:27
Update README.md

AdaptLLM

8f6cff6
07:47
Update README.md

AdaptLLM

f0ed7ab
05:57
Update README.md

AdaptLLM

5564758
05:54
Update README.md

AdaptLLM

849c7c9

Dec 29, 2023

10:04
Update README.md

AdaptLLM

4791417

Dec 28, 2023

02:27
Update README.md

AdaptLLM

6b69dca

Dec 19, 2023

13:24
Update README.md

AdaptLLM

0316b09
13:19
Update README.md

AdaptLLM

73ed9cb

Dec 10, 2023

04:57
Update README.md

AdaptLLM

a28090d

Dec 9, 2023

04:39
Update README.md

AdaptLLM

8149bd9
04:36
Update README.md

AdaptLLM

84fc133
04:33
Update README.md

AdaptLLM

0556965
04:24
Update README.md

AdaptLLM

aba103c
04:21
Update README.md

AdaptLLM

b2a3c31
04:19
Update README.md

AdaptLLM

c8ac95c

Nov 7, 2023

11:56
Update README.md

AdaptLLM

7d1f7c1

Nov 4, 2023

04:01
Update README.md

AdaptLLM

cc31e2a

Oct 21, 2023

11:47
Update README.md

AdaptLLM

f8c257d

Sep 20, 2023

07:11
merge

AdaptLLM

789e88b
07:07
update readme

AdaptLLM

1f4222a
02:04
Create README.md

AdaptLLM

fc97b55

Sep 18, 2023

13:52
AdaptLLM-med-v0

AdaptLLM

1e6eba6
07:59
initial commit

AdaptLLM

056bdea

Commit History

Apr 17, 2024

Adding `safetensors` variant of this model⁠

Mar 11, 2024

Update README.md

Feb 7, 2024

Update README.md

Feb 2, 2024

Update README.md

Jan 22, 2024

Update README.md

Jan 16, 2024

Update README.md

Jan 2, 2024

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Dec 29, 2023

Update README.md

Dec 28, 2023

Update README.md

Dec 19, 2023

Update README.md

Update README.md

Dec 10, 2023

Update README.md

Dec 9, 2023

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Nov 7, 2023

Update README.md

Nov 4, 2023

Update README.md

Oct 21, 2023

Update README.md

Sep 20, 2023

merge

update readme

Create README.md

Sep 18, 2023

AdaptLLM-med-v0

initial commit