Instructions to use jth2/fin_model with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use jth2/fin_model with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("DragonLLM/Llama-Open-Finance-8B")
model = PeftModel.from_pretrained(base_model, "jth2/fin_model")

Transformers

How to use jth2/fin_model with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="jth2/fin_model")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("jth2/fin_model", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use jth2/fin_model with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jth2/fin_model"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jth2/fin_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/jth2/fin_model

SGLang

How to use jth2/fin_model with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "jth2/fin_model" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jth2/fin_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "jth2/fin_model" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jth2/fin_model",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use jth2/fin_model with Docker Model Runner:
```
docker model run hf.co/jth2/fin_model
```

jth2 commited on Aug 4

Commit

7ccb0fc

verified ·

1 Parent(s): e334e59

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -1

README.md CHANGED Viewed

@@ -14,7 +14,7 @@ tags:
 Presently, it is difficult for people to educate themselves on financial concepts without facing significant hurdles. Tools & resources are often locked behind some form of paywall, and other forms of gatekeeping make it difficult for people to develop their knowledge. As financial literacy is something that has a large impact on the ability of an individual to participate in their communities, it seems crucial that there be free tools to enable education as well as those already offered by businesses. For this project, the focus was to develop a model which could assist in answering and explaining the solution behind a financial question so that people could develop their understanding. To do so, a base model was pulled from DragonLLM/Llama-Open-Finance-8B (the name of the repository was recently updated to OVHaiLLM/Llama-Open-Finance-8B) and tuned to attempt to develop a model better at these tasks.
 ## Data:
-For this project, the model was finetuned on a FinRAG based dataset, which can be found in the references below. For the project, the columns were recombined to form the full prompt in the following order (pre_text, table, post_text, question, choices). The model was then prompted to attach a step-by-step explanation to each problem to provide the logic necessary to achieve the correct solution before the training process. In addition, a randomized set of 1000 of these questions were pulled from the dataset using a random seed of 42. These questions were then split into the train, validate, and test datasets in an 80%/10%/10% split respectively.
 ## Methodology:

 Presently, it is difficult for people to educate themselves on financial concepts without facing significant hurdles. Tools & resources are often locked behind some form of paywall, and other forms of gatekeeping make it difficult for people to develop their knowledge. As financial literacy is something that has a large impact on the ability of an individual to participate in their communities, it seems crucial that there be free tools to enable education as well as those already offered by businesses. For this project, the focus was to develop a model which could assist in answering and explaining the solution behind a financial question so that people could develop their understanding. To do so, a base model was pulled from DragonLLM/Llama-Open-Finance-8B (the name of the repository was recently updated to OVHaiLLM/Llama-Open-Finance-8B) and tuned to attempt to develop a model better at these tasks.
 ## Data:
+For this project, the model was finetuned on a FinRAG based dataset, which can be found in the references below. For the project, the columns were recombined to form the full prompt in the following order (pre_text, table, post_text, question, choices). The model was then prompted to attach a step-by-step explanation to each problem to provide the logic necessary to achieve the correct solution before the training process. A randomized set of 1000 of these questions were pulled from the dataset using a random seed of 42. These questions were then split into the train, validate, and test datasets in an 80%/10%/10% split respectively.
 ## Methodology: