Instructions to use globalnebula/FinoLLama with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use globalnebula/FinoLLama with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")
model = PeftModel.from_pretrained(base_model, "globalnebula/FinoLLama")

Transformers

How to use globalnebula/FinoLLama with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="globalnebula/FinoLLama")

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("globalnebula/FinoLLama", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use globalnebula/FinoLLama with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "globalnebula/FinoLLama"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "globalnebula/FinoLLama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/globalnebula/FinoLLama

SGLang

How to use globalnebula/FinoLLama with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "globalnebula/FinoLLama" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "globalnebula/FinoLLama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "globalnebula/FinoLLama" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "globalnebula/FinoLLama",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use globalnebula/FinoLLama with Docker Model Runner:
```
docker model run hf.co/globalnebula/FinoLLama
```

FinoLLama / model_samples.txt

globalnebula

init commit

edfde39 verified 10 months ago

Raw

History Blame Contribute Delete

2.84 kB

	Sample Generations from Tax Domain Adapted Model
	============================================================

	Prompt: Tax compliance requirements
	Generated: for digital platforms providing services of transport vehicles
	12. The provisions of clause (viii) of sub-section (3) of section 203A, second proviso to section 206C and
	third proviso to rule 115B of the Income-tax Rules shall be applied accordingly to the digital platform
	referred to in clause (k) of section 2 of the Information Technology Act, 2000 (21 of 2000) so as to
	require it to intimate its Permanent Account Number to the person providing services through such
	digital platform.
	Special provision for deduction of tax at source for non-filers of income-tax return.
	285B. (1) Notwithstanding anything contained in any other provisions of this Act, where tax is required to
	be deducted at source under the provisions of Chapter XVIIB, other than section 192, 192A, 194B,
	194BA, 194BB, 194-IA, 194-
	==================================================
	Prompt: Income tax calculation
	Generated: for salary income above 2.5 lakhs in india for fy 2018-19. This calculator is based on new tax regime.
	Tax Calculator For Salary Income Above 2.5 Lakhs In India For FY 2018-19
	Income tax calculation for salary income above 2.5 lakhs in india for fy 2018-19. This calculator is based on new tax regime.
	Tax Calculator For Salary Income Above 2.5 Lakhs In India For FY 2018-19
	Income tax calculation for salary income above 2.5 lakhs in india for fy 2018-19. This calculator is based on new tax regime.
	Tax Calculator For Salary Income Above 2.5 Lakhs In India For FY 2018-19
	Income tax calculation for salary income above 2.5 lakhs in india for fy 2018-19. This calculator is based on new tax regime.
	Tax Calculator For Salary Income
	==================================================
	Prompt: CBDT circular guidelines
	Generated: on faceless assessment
	On 29 ^{th} July 2024, the Central Board of Direct Taxes (CBDT) issued circular number CBDT Circular No. 7 of 2024, 9 giving instructions and procedures for introduction of a new faceless assessment feature in certain respect. The said circular will come into effect from 1 ^{st} April 2025 and will apply to all persons having income chargeable to tax under the provisions of Income Tax Act, 1961 (hereinafter referred to as the Act).
	As per the said circular, during the faceless assessment, the Assessing Officer shall not be allowed to see or receive any evidence or information, except the evidence or information which is available in the electronic record of the case file of the assessee. However, if the Assessing Officer requires any further enquiry or investigation in respect of any point, he may serve a direction on the National Faceless Assessment Centre (NFAC)
	==================================================