Instructions to use AdaptLLM/finance-LLM-13B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use AdaptLLM/finance-LLM-13B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="AdaptLLM/finance-LLM-13B")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("AdaptLLM/finance-LLM-13B")
model = AutoModelForCausalLM.from_pretrained("AdaptLLM/finance-LLM-13B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use AdaptLLM/finance-LLM-13B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "AdaptLLM/finance-LLM-13B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/finance-LLM-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/AdaptLLM/finance-LLM-13B

SGLang

How to use AdaptLLM/finance-LLM-13B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "AdaptLLM/finance-LLM-13B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/finance-LLM-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "AdaptLLM/finance-LLM-13B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "AdaptLLM/finance-LLM-13B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use AdaptLLM/finance-LLM-13B with Docker Model Runner:
```
docker model run hf.co/AdaptLLM/finance-LLM-13B
```

finance-LLM-13B

Commit History

Oct 26, 2024

23:24
Adding `safetensors` variant of this model⁠

SFconvertbot

Verified signature
2f56750

Aug 29, 2024

06:19
Update README.md

AdaptLLM

Verified signature
74fa0c6
05:09
Update README.md

AdaptLLM

Verified signature
04e033e
04:52
Update README.md

AdaptLLM

Verified signature
ccca136

Jul 19, 2024

02:48
Update README.md

AdaptLLM

Verified signature
75b83be

Jun 25, 2024

03:02
Update README.md

AdaptLLM

Verified signature
22e92f8

Jun 21, 2024

12:30
Update README.md

AdaptLLM

Verified signature
c6c5708

Mar 11, 2024

02:43
Update README.md

AdaptLLM

Verified signature
4a5f3c1

Feb 7, 2024

12:30
Update README.md

AdaptLLM

Verified signature
7ca423c

Feb 2, 2024

06:11
Update README.md

AdaptLLM

Verified signature
aec6f74

Jan 22, 2024

12:57
Update README.md

AdaptLLM

Verified signature
94b686f

Jan 16, 2024

13:42
Update README.md

AdaptLLM

Verified signature
ef96ea3

Jan 2, 2024

07:44
Update README.md

AdaptLLM

636f34b
07:43
Update README.md

AdaptLLM

c99bc54
06:00
Update README.md

AdaptLLM

0777c7f
05:52
Update README.md

AdaptLLM

7ce9bb5
05:48
Update README.md

AdaptLLM

99d48dd

Dec 29, 2023

10:06
Update README.md

AdaptLLM

6699e7b

Dec 19, 2023

13:14
Update README.md

AdaptLLM

fe15cf6
13:14
Update README.md

AdaptLLM

288cbcd
10:35
init finance-LLM-13B

AdaptLLM

f7a93ee
10:04
initial commit

AdaptLLM

edbec33

Commit History

Oct 26, 2024

Adding `safetensors` variant of this model⁠

Aug 29, 2024

Update README.md

Update README.md

Update README.md

Jul 19, 2024

Update README.md

Jun 25, 2024

Update README.md

Jun 21, 2024

Update README.md

Mar 11, 2024

Update README.md

Feb 7, 2024

Update README.md

Feb 2, 2024

Update README.md

Jan 22, 2024

Update README.md

Jan 16, 2024

Update README.md

Jan 2, 2024

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Dec 29, 2023

Update README.md

Dec 19, 2023

Update README.md

Update README.md

init finance-LLM-13B

initial commit