Instructions to use lrds-code/boana-7b-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use lrds-code/boana-7b-instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="lrds-code/boana-7b-instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("lrds-code/boana-7b-instruct")
model = AutoModelForCausalLM.from_pretrained("lrds-code/boana-7b-instruct", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

PEFT
How to use lrds-code/boana-7b-instruct with PEFT:
```
Task type is invalid.
```
Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use lrds-code/boana-7b-instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "lrds-code/boana-7b-instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/lrds-code/boana-7b-instruct

SGLang

How to use lrds-code/boana-7b-instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "lrds-code/boana-7b-instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "lrds-code/boana-7b-instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "lrds-code/boana-7b-instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use lrds-code/boana-7b-instruct with Docker Model Runner:
```
docker model run hf.co/lrds-code/boana-7b-instruct
```

boana-7b-instruct

Commit History

Apr 4, 2024

00:38
Fixing some errors of the leaderboard evaluation results in the ModelCard yaml⁠

leaderboard-pt-pr-bot

Verified signature
2358a54
00:35
Fixing some errors of the leaderboard evaluation results in the ModelCard yaml⁠

leaderboard-pt-pr-bot

Verified signature
12a2bfc

Apr 3, 2024

19:25
Adding the Open Portuguese LLM Leaderboard Evaluation Results⁠

leaderboard-pt-pr-bot

Verified signature
c210cd8
01:03
Update README.md

lrds-code

Verified signature
dc3e94b
01:01
Update README.md

lrds-code

Verified signature
50eb170
00:55
Upload tokenizer

lrds-code

Verified signature
eae7f76
00:55
Upload LlamaForCausalLM

lrds-code

Verified signature
6260631

Jan 23, 2024

17:38
Update README.md

lrds-code

Verified signature
96039b0
17:23
Update README.md

lrds-code

Verified signature
5ba825a
17:20
Update README.md

lrds-code

Verified signature
e9d4188
17:11
Update README.md

lrds-code

Verified signature
368eda3
17:08
Update README.md

lrds-code

Verified signature
4c2cfcd
02:37
Update README.md

lrds-code

Verified signature
c93c33a
02:28
Update README.md

lrds-code

Verified signature
791eb6a
02:24
Update README.md

lrds-code

Verified signature
52b3731
02:22
Update README.md

lrds-code

Verified signature
7a45691

Jan 22, 2024

14:32
Update README.md

lrds-code

Verified signature
7550383
14:06
Update README.md

lrds-code

Verified signature
716777f
13:55
Update README.md

lrds-code

Verified signature
abe9546
13:44
Create README.md

lrds-code

Verified signature
d9249ae

Jan 20, 2024

14:12
Upload tokenizer

lrds-code

Verified signature
ffd9e4e
14:12
Upload LlamaForCausalLM

lrds-code

Verified signature
017cd5d
14:06
initial commit

lrds-code

Verified signature
c6044ae

Commit History

Apr 4, 2024

Fixing some errors of the leaderboard evaluation results in the ModelCard yaml⁠

Fixing some errors of the leaderboard evaluation results in the ModelCard yaml⁠

Apr 3, 2024

Adding the Open Portuguese LLM Leaderboard Evaluation Results⁠

Update README.md

Update README.md

Upload tokenizer

Upload LlamaForCausalLM

Jan 23, 2024

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Update README.md

Jan 22, 2024

Update README.md

Update README.md

Update README.md

Create README.md

Jan 20, 2024

Upload tokenizer

Upload LlamaForCausalLM

initial commit