Text Generation
PEFT
Safetensors
English
tinyllama
lora
faq
customer-support
enterprise-ai
vipplav-ai
conversational
Instructions to use Vipplav/tinyllama-finetuned-faq with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use Vipplav/tinyllama-finetuned-faq with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0") model = PeftModel.from_pretrained(base_model, "Vipplav/tinyllama-finetuned-faq") - Notebooks
- Google Colab
- Kaggle
π¦ TinyLlama Fine-Tuned Enterprise FAQ Assistant
Lightweight, Low-Latency Customer Support & FAQ Generation
Developed by Vipplav AI β India's Most Specialized AI Studio
π Model Overview
Vipplav/tinyllama-finetuned-faq is a specialized LoRA adapter fine-tuned on top of TinyLlama/TinyLlama-1.1B-Chat-v1.0.
It is engineered for businesses needing an ultra-low-latency, edge-deployable question-answering assistant that delivers concise, context-grounded answers without requiring expensive cloud GPUs or massive API infrastructure.
- Base Model:
TinyLlama/TinyLlama-1.1B-Chat-v1.0 - Adapter Type: Parameter-Efficient Fine-Tuning (LoRA)
- Target Modules:
q_proj,v_proj - Ideal Footprint: Runs seamlessly on consumer laptops, CPUs, Raspberry Pi, or edge nodes (<2GB VRAM).
π Usage with peft and transformers
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_name = "Vipplav/tinyllama-finetuned-faq"
# 1. Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
base_model_name,
torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
device_map="auto"
)
# 2. Attach Vipplav LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()
# 3. Format FAQ Prompt
prompt = "<|system|>\nYou are a helpful enterprise support assistant. Answer the customer FAQ concisely.</s>\n<|user|>\nHow can I reset my account password?</s>\n<|assistant|>\n"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
outputs = model.generate(
**inputs,
max_new_tokens=150,
temperature=0.3,
repetition_penalty=1.1,
eos_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print("Response:", response.strip())
π’ About Vipplav AI
Vipplav AI is an enterprise AI studio engineering foundational Indic Small Language Models (Rachana) and verifiable agentic automation pipelines.
- Website: https://www.vipplavai.com
- GitHub: https://github.com/vipplavai
- Contact: hello@vipplavai.com
- Downloads last month
- 20
Model tree for Vipplav/tinyllama-finetuned-faq
Base model
TinyLlama/TinyLlama-1.1B-Chat-v1.0