Instructions to use adamsw/aphrc-v2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use adamsw/aphrc-v2 with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.3") model = PeftModel.from_pretrained(base_model, "adamsw/aphrc-v2") - Transformers
How to use adamsw/aphrc-v2 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="adamsw/aphrc-v2") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("adamsw/aphrc-v2") model = AutoModelForCausalLM.from_pretrained("adamsw/aphrc-v2", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use adamsw/aphrc-v2 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "adamsw/aphrc-v2" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adamsw/aphrc-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/adamsw/aphrc-v2
- SGLang
How to use adamsw/aphrc-v2 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "adamsw/aphrc-v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adamsw/aphrc-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "adamsw/aphrc-v2" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "adamsw/aphrc-v2", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use adamsw/aphrc-v2 with Docker Model Runner:
docker model run hf.co/adamsw/aphrc-v2
# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("adamsw/aphrc-v2")
model = AutoModelForCausalLM.from_pretrained("adamsw/aphrc-v2", device_map="auto")
messages = [
{"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
tokenize=True,
return_dict=True,
return_tensors="pt",
).to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))aphrc-v2
This is a fine-tuned LoRA adapter for the Mistral-7B-Instruct-v0.3 model, specifically trained on the aphrc dataset to improve performance on African health and development questions.
Model Details
- Base Model: mistralai/Mistral-7B-Instruct-v0.3
- Adapter Type: LoRA (Low-Rank Adaptation)
- Target Task: Question answering on African health and development topics
- Training Method: Supervised Fine-Tuning (SFT)
- Author: Adams Wakhungu, Adanian Labs
How to Use
With Transformers and PEFT
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch
# Load base model
base_model = "mistralai/Mistral-7B-Instruct-v0.3"
model = AutoModelForCausalLM.from_pretrained(
base_model,
torch_dtype=torch.float16,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(base_model)
# Load the LoRA adapter
model = PeftModel.from_pretrained(model, "adamsw/aphrc-v2")
# Generate text
question = "What are the major health challenges facing Africa today?"
messages = [
{"role": "user", "content": question}
]
input_ids = tokenizer.apply_chat_template(messages, return_tensors="pt").to(model.device)
outputs = model.generate(input_ids=input_ids, max_new_tokens=512, temperature=0.7)
response = tokenizer.decode(outputs[0][input_ids.shape[1]:], skip_special_tokens=True)
print(response)
Using Hugging Face Pipeline
from transformers import pipeline
from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer
config = PeftConfig.from_pretrained("adamsw/aphrc-v2")
model = AutoModelForCausalLM.from_pretrained(
config.base_model_name_or_path,
torch_dtype=torch.float16,
device_map="auto"
)
model = PeftModel.from_pretrained(model, "adamsw/aphrc-v2")
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)
generator = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
device_map="auto"
)
question = "What are the major health challenges facing Africa today?"
messages = [
{"role": "user", "content": question}
]
output = generator(
messages,
max_new_tokens=512,
temperature=0.7,
pad_token_id=tokenizer.eos_token_id
)
print(output[0]['generated_text'][-1]['content'])
Training Procedure
This model was trained using Supervised Fine-Tuning (SFT) with the TRL library. The training data consisted of African health and development questions with reference answers.
Framework Versions
- PEFT 0.17.1
- TRL: 0.23.0
- Transformers: 4.56.2
- Pytorch: 2.7.1+cu118
- Datasets: 4.1.1
- Tokenizers: 0.22.1
About the Author
This model was developed by Adams Wakhungu at Adanian Labs, focusing on improving language model performance for African health and development contexts.
Citation
If you use this model, please cite:
Cite TRL as:
@misc{vonwerra2022trl,
title = {{TRL: Transformer Reinforcement Learning}},
author = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallou{\'e}dec},
year = 2020,
journal = {GitHub repository},
publisher = {GitHub},
howpublished = {\url{https://github.com/huggingface/trl}}
}
- Downloads last month
- 2
Model tree for adamsw/aphrc-v2
Base model
mistralai/Mistral-7B-v0.3
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="adamsw/aphrc-v2") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)