theophilusowiti/Caracal_Instruct_Dataset
Viewer โข Updated โข 248k
How to use theophilusowiti/Caracal_instruct_v2 with PEFT:
from peft import PeftModel
from transformers import AutoModelForCausalLM
base_model = AutoModelForCausalLM.from_pretrained("theophilusowiti/Caracal_instruct")
model = PeftModel.from_pretrained(base_model, "theophilusowiti/Caracal_instruct_v2")This model is a fine-tuned LoRA adapter for the Caracal African LLM, optimized for instruction-following and dialogue using the ChatML format.
trl SFTTrainer.import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
base_model_id = "theophilusowiti/Caracal_GPT"
adapter_id = "theophilusowiti/Caracal_instruct_v2"
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_id,
torch_dtype=torch.bfloat16,
device_map="auto"
)
model = PeftModel.from_pretrained(base_model, adapter_id)
messages = [
{"role": "system", "content": "You are a helpful AI assistant."},
{"role": "user", "content": "Who is the president of Kenya?"}
]
inputs = tokenizer.apply_chat_template(
messages,
return_tensors="pt",
add_generation_prompt=True
).to(model.device)
outputs = model.generate(inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))