Instructions to use sixfingerdev/SixFinger-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sixfingerdev/SixFinger-8B with PEFT:

from peft import PeftModel
from transformers import AutoModelForCausalLM

base_model = AutoModelForCausalLM.from_pretrained("unsloth/meta-llama-3.1-8b-bnb-4bit")
model = PeftModel.from_pretrained(base_model, "sixfingerdev/SixFinger-8B")

Transformers

How to use sixfingerdev/SixFinger-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sixfingerdev/SixFinger-8B")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("sixfingerdev/SixFinger-8B", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sixfingerdev/SixFinger-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sixfingerdev/SixFinger-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sixfingerdev/SixFinger-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/sixfingerdev/SixFinger-8B

SGLang

How to use sixfingerdev/SixFinger-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sixfingerdev/SixFinger-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sixfingerdev/SixFinger-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sixfingerdev/SixFinger-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sixfingerdev/SixFinger-8B",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Desktop
Docker Model Runner
How to use sixfingerdev/SixFinger-8B with Docker Model Runner:
```
docker model run hf.co/sixfingerdev/SixFinger-8B
```

SixFinger-8B — Llama 3.1 8B LoRA adapter

This repository contains a LoRA adapter, not a standalone 8B model. To run it, you need the original Llama 3.1 8B base model and the PEFT library. This is an experimental Turkish-language fine-tuning project.

What is included

Item	Details
Base model	unsloth/Meta-Llama-3.1-8B-bnb-4bit
Fine-tuning	LoRA / supervised fine-tuning
Base-model loading	4-bit bitsandbytes checkpoint
Training dataset	Turkish QA Multi-Dialog
Published weights	Adapter weights, not the full base model

The base model identifier is taken from this repository's adapter_config.json.

Installation

pip install -U transformers accelerate peft bitsandbytes torch

A compatible GPU and enough memory to load the quantized base model are required.

Loading and generating text

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_id = "unsloth/meta-llama-3.1-8b-bnb-4bit"
adapter_id = "sixfingerdev/SixFinger-8B"

tokenizer = AutoTokenizer.from_pretrained(base_id)
base = AutoModelForCausalLM.from_pretrained(base_id, device_map="auto")
model = PeftModel.from_pretrained(base, adapter_id)
model.eval()

prompt = "Soru: Yapay zekâ nedir?\nCevap:"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=128,
        do_sample=False,
        pad_token_id=tokenizer.eos_token_id,
    )

new_tokens = output_ids[0, inputs["input_ids"].shape[-1]:]
print(tokenizer.decode(new_tokens, skip_special_tokens=True))

This is a loading example, not a verified quality benchmark. Exact outputs depend on the prompt, decoding settings, and installed library versions.

Evaluation and limitations

Experimental checkpoint: no independently reproducible benchmark results have been published for this adapter.
The repository is not an 8B model trained from scratch; its behavior depends on the upstream model.
Generated information may be incorrect. Evaluate on your own use case before deployment.
Data quality and provenance should be reviewed before production use.

License and attribution

The adapter is published with the repository's Apache-2.0 license metadata. The underlying Llama 3.1 base model is separately governed by the Llama 3.1 Community License. Combining or redistributing model weights requires complying with the applicable upstream terms.

See PEFT documentation for loading and merging adapters.

Downloads last month: 29

sixfingerdev
/

SixFinger-8B