πŸ¦™ TinyLlama Fine-Tuned Enterprise FAQ Assistant

Lightweight, Low-Latency Customer Support & FAQ Generation

Developed by Vipplav AI β€” India's Most Specialized AI Studio

Website GitHub Base Model License


πŸ“Œ Model Overview

Vipplav/tinyllama-finetuned-faq is a specialized LoRA adapter fine-tuned on top of TinyLlama/TinyLlama-1.1B-Chat-v1.0.

It is engineered for businesses needing an ultra-low-latency, edge-deployable question-answering assistant that delivers concise, context-grounded answers without requiring expensive cloud GPUs or massive API infrastructure.

  • Base Model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
  • Adapter Type: Parameter-Efficient Fine-Tuning (LoRA)
  • Target Modules: q_proj, v_proj
  • Ideal Footprint: Runs seamlessly on consumer laptops, CPUs, Raspberry Pi, or edge nodes (<2GB VRAM).

πŸš€ Usage with peft and transformers

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
adapter_name = "Vipplav/tinyllama-finetuned-faq"

# 1. Load tokenizer and base model
tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# 2. Attach Vipplav LoRA adapter
model = PeftModel.from_pretrained(base_model, adapter_name)
model.eval()

# 3. Format FAQ Prompt
prompt = "<|system|>\nYou are a helpful enterprise support assistant. Answer the customer FAQ concisely.</s>\n<|user|>\nHow can I reset my account password?</s>\n<|assistant|>\n"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=150,
        temperature=0.3,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print("Response:", response.strip())

🏒 About Vipplav AI

Vipplav AI is an enterprise AI studio engineering foundational Indic Small Language Models (Rachana) and verifiable agentic automation pipelines.

Downloads last month
20
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Vipplav/tinyllama-finetuned-faq

Adapter
(1,614)
this model