README.md

---
license: mit
language:
- en
metrics:
- bertscore
- bleu
- rouge
- accuracy
base_model:
- MBZUAI/LaMini-Flan-T5-248M
pipeline_tag: question-answering
library_name: transformers
tags:
- RAG
- FAISS
- Telecom
- Question-Answering
- Flan-T5
- Sentence-Transformers
---



# Telecom Plan Advisor – (RAG-LLM) based Question Answering System

Telecom Plan Advisor is a **Retrieval-Augmented Generation (RAG)** system that helps users compare and choose wireless plans from **Bell, Virgin Plus, and Lucky Mobile**. 
It combines **FAISS vector search** (MiniLM embeddings) with a lightweight **seq2seq LLM** (`flan-alpaca-base` or `LaMini-Flan-T5-783M`) to answer plan-related questions in natural language.

## How it works
- **Retrieve**: FAISS finds the most relevant plan descriptions.  
- **Generate**: LLM produces a concise, friendly answer grounded in retrieved plans.  
- **Evaluate**: System performance measured with BLEU, ROUGE, and BERTScore.

# Datasets:
- Synthetic Wireless Plans Dataset (curated from Bell, Virgin Plus and Lucky Mobile)

## Quickstart
```python
from transformers import pipeline, AutoTokenizer, AutoModelForSeq2SeqLM
import faiss, pandas as pd
from sentence_transformers import SentenceTransformer

# Load model
tokenizer = AutoTokenizer.from_pretrained("declare-lab/flan-alpaca-base")
model = AutoModelForSeq2SeqLM.from_pretrained("declare-lab/flan-alpaca-base")
qa = pipeline("text2text-generation", model=model, tokenizer=tokenizer)

# Example
print(qa("Best BYOD plan under $50 from Virgin Plus?", max_new_tokens=100)[0]["generated_text"])
```

📊 Evaluation Results

BLEU: 0.46

ROUGE-1: 0.57

ROUGE-2: 0.35

ROUGE-L: 0.40

BERTScore-F1: 0.93

- The system was evaluated on a small set of plan-related queries using BLEU, ROUGE, and BERTScore.  
- Sample results show **strong semantic similarity** between generated answers and reference plan descriptions, with **BERTScore F1 around 0.9+**.  

> Note: BLEU/ROUGE are conservative for free-form LLM outputs, and exact values may vary depending on the test set and chosen base model.


## Limitations

- Works only on the curated dataset (does not fetch live pricing).
- Region support (Ontario, Quebec, Alberta) is inferred from plan names.