Update README with auto-metrics docs and fix card rendering bugs

README changes:
- Updated "What It Does" to describe automatic metric chips, winner/hallucination
badges, and the Auto-Benchmark runner instead of manual rating system
- Added "Auto-Evaluation Metrics" section documenting Accuracy, ROUGE-L,
Groundedness, Context Relevance, Latency, Win count, and Low Confidence
- Resolved merge conflict by keeping project README (not model card template)
demo.py changes:
- Add safe_answer() helper: html.escape() + \n -> <br> to prevent Streamlit's
markdown parser from exiting the HTML block on blank lines in LLM answers
- Fix _header_badge() returning "" (empty string) causing a whitespace-only line
that terminated the HTML block, rendering card HTML as literal text —
now returns <span></span> as a safe non-empty placeholder
- Fix run_benchmark() using wrong key 'sem_sim' instead of 'accuracy', causing
all benchmark accuracy/win scores to show as 0%
- Remove stale rating_submitted session state reference

README.md

@@ -1,207 +1,180 @@
----
-base_model: Qwen/Qwen2.5-1.5B-Instruct
-library_name: peft
-pipeline_tag: text-generation
-tags:
-- base_model:adapter:Qwen/Qwen2.5-1.5B-Instruct
-- lora
-- transformers
----
-
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
-
-## Bias, Risks, and Limitations
-
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
-
-[More Information Needed]
-
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
-
-## Training Details
-
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+# 🧙 CodeSage
 
-[More Information Needed]
+**A comparative study of three LLM strategies for domain-specific question answering.**
 
-### Training Procedure
+CodeSage puts a Baseline LLM, a RAG pipeline, and a Fine-Tuned model side-by-side so you can ask the same programming question and immediately see how each approach differs in accuracy, grounding, and speed — all scored automatically with no manual input required.
 
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
+---
 
-## Environmental Impact
+## Why This Project?
 
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+When building an AI assistant for a specific domain (here: programming / DSA / web dev), you have three realistic options:
 
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+| Approach | Core idea | Trade-off |
+|---|---|---|
+| **Baseline LLM** | Prompt a large general-purpose model | Fast, zero setup — but may hallucinate or give vague answers |
+| **RAG** | Retrieve relevant docs first, then generate | Grounded answers, updatable knowledge — retrieval adds latency |
+| **Fine-Tuning** | Train a smaller model on domain data | Potentially fastest, very focused — expensive to update |
 
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
+There is no universal winner. The goal of CodeSage is to **make that trade-off visible** with real side-by-side outputs and fully automatic evaluation metrics.
 
-## Technical Specifications [optional]
+---
 
-### Model Architecture and Objective
+## What It Does
 
-[More Information Needed]
+1. Type (or click) a programming question.
+2. All three systems answer it **simultaneously**.
+3. Each answer card shows auto-computed metric chips: **🎯 Accuracy**, **⚓ Groundedness**, **📄 Context Relevance**, **ROUGE-L**, **⏱ Latency**.
+4. A **🏆 Best Answer** badge highlights the winner per question; a **⚠️ Low Confidence** badge flags possible hallucinations (accuracy < 40%).
+5. Click **Run Auto-Benchmark** to run all 6 sample questions automatically and populate:
+   - KPI cards (Accuracy %, ROUGE-L, Avg Latency, Win count)
+   - Plotly bar charts (Accuracy vs ROUGE-L · Latency vs Low-Confidence count)
+   - A full evaluation table with colour-coded badges
 
-### Compute Infrastructure
+---
 
-[More Information Needed]
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────┐
+│                    Streamlit UI                      │
+│  ┌──────────────┐  ┌──────────────┐  ┌───────────┐  │
+│  │  System 1    │  │  System 2    │  │ System 3  │  │
+│  │  Baseline    │  │    RAG       │  │ Fine-Tuned│  │
+│  └──────┬───────┘  └──────┬───────┘  └─────┬─────┘  │
+└─────────┼─────────────────┼────────────────┼─────────┘
+          │                 │                │
+          ▼                 ▼                ▼
+   Groq API           FAISS index      Qwen2.5-1.5B
+   Llama 3.1 8B    +  HuggingFace   (LoRA adapter loaded
+   (prompt only)      embeddings     locally via PEFT)
+                          │
+                      Groq API
+                      Llama 3.1 8B
+                      (with context)
+```
+
+### System 1 — Baseline LLM
+Sends the question directly to **Llama 3.1 8B** (via Groq) with a short system prompt. No extra knowledge.
+
+### System 2 — RAG (Retrieval-Augmented Generation)
+1. The question is embedded with `all-MiniLM-L6-v2` (HuggingFace Sentence Transformers).
+2. Top-3 most similar chunks are retrieved from a **FAISS** vector store built from 22 documents (`.txt` files + any PDFs dropped into `data/pdfs/`).
+3. Those chunks are injected as context into the prompt sent to **Llama 3.1 8B** via Groq.
+
+### System 3 — Fine-Tuned Model
+**Qwen2.5-1.5B-Instruct** fine-tuned with **LoRA** on ~20 curated programming Q&A pairs. Training is done on Google Colab (free GPU). The adapter is loaded locally via `peft`.
 
-#### Hardware
+---
 
-[More Information Needed]
+## Auto-Evaluation Metrics
 
-#### Software
+All metrics are computed automatically — no user ratings needed.
 
-[More Information Needed]
+| Metric | How it's computed | Where it appears |
+|---|---|---|
+| **Accuracy** | Cosine similarity between answer embedding and reference answer embedding | Chip on each card · KPI card · Eval table |
+| **ROUGE-L** | Longest Common Subsequence F-measure vs reference answer | Chip on each card · KPI card · Eval table |
+| **Groundedness** | Cosine similarity between answer embedding and retrieved context embedding (RAG only) | Chip on RAG card |
+| **Context Relevance** | Cosine similarity between question embedding and retrieved context embedding (RAG only) | Chip on RAG card |
+| **Latency** | Wall-clock response time in seconds | Chip on each card · KPI card · Eval table |
+| **Win count** | System with highest accuracy per question gets a win | KPI card · Eval table |
+| **Low Confidence** | Answers with accuracy < 40% (possible hallucination) | ⚠️ badge · Eval table |
 
-## Citation [optional]
+Embeddings use `all-MiniLM-L6-v2` via LangChain. ROUGE-L uses the `rouge-score` library.
 
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+---
 
-**BibTeX:**
+## Tech Stack
+
+| Layer | Technology |
+|---|---|
+| UI | Streamlit |
+| LLM inference | Groq API — `llama-3.1-8b-instant` |
+| Embeddings | `sentence-transformers/all-MiniLM-L6-v2` |
+| Vector store | FAISS (CPU) via LangChain |
+| Fine-tuning | HuggingFace Transformers + PEFT (LoRA) |
+| Base model | Qwen/Qwen2.5-1.5B-Instruct |
+| Auto metrics | ROUGE-L (`rouge-score`) + cosine semantic similarity |
+| Charts | Plotly |
+| Training platform | Google Colab |
+| Background animation | HTML5 Canvas 3D particle network |
 
-[More Information Needed]
+---
 
-**APA:**
+## Deploy to HuggingFace Spaces
 
-[More Information Needed]
+1. Create a new Space at [huggingface.co/new-space](https://huggingface.co/new-space) — SDK: **Streamlit**
+2. Push this repo to the Space:
+   ```bash
+   git remote add space https://huggingface.co/spaces/YOUR_USERNAME/codesage
+   git push space main
+   ```
+3. Add your `GROQ_API_KEY` as a Space secret (Settings → Variables and secrets)
+4. The app launches automatically — the FAISS index is built on first run (~30s)
 
-## Glossary [optional]
+> System 3 (fine-tuned) will show "model unavailable" on Spaces unless you upload the `finetuned_model/` adapter folder to the Space repo.
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+---
 
-[More Information Needed]
+## How to Run Locally
+
+### 1. Clone & install
+```bash
+git clone https://github.com/Adityax-07/LLM-vs-RAG-vs-Fine-Tuning-.git
+cd LLM-vs-RAG-vs-Fine-Tuning-
+pip install -r requirements.txt
+```
+
+### 2. Set your Groq API key
+Create a `.env` file:
+```
+GROQ_API_KEY=your_key_here
+```
+Get a free key at [console.groq.com](https://console.groq.com).
+
+### 3. Run the app
+```bash
+streamlit run demo.py
+```
+The FAISS index is built automatically on first launch.
+
+### 4. (Optional) Expand the knowledge base with PDFs
+Drop any `.pdf` files into `data/pdfs/`, then delete `data/faiss_index/` and restart — PDFs are ingested automatically.
+
+### 5. (Optional) Load the fine-tuned model
+1. Open `system3_finetune_colab.ipynb` in Google Colab.
+2. Run all cells — trains **Qwen2.5-1.5B-Instruct** with LoRA, takes ~15 min on a free T4 GPU.
+3. Download `finetuned_model.zip` and unzip it in the project root.
+4. Restart the app — System 3 returns real answers.
 
-## More Information [optional]
+---
 
-[More Information Needed]
+## Knowledge Base
 
-## Model Card Authors [optional]
+The RAG system retrieves from 22 documents in `data/docs/`:
 
-[More Information Needed]
+`binary_search` · `sorting_algorithms` · `dynamic_programming` · `graph_algorithms` · `trees` · `linked_list` · `stack_queue` · `recursion` · `backtracking` · `greedy_algorithms` · `hashing` · `string_algorithms` · `two_pointers` · `react_hooks` · `rest_api` · `javascript_promises` · `css_flexbox` · `big_o_notation` · `heaps` · `sql_basics` · `typescript_basics` · `git_basics`
 
-## Model Card Contact
+Plus any PDFs placed in `data/pdfs/`.
 
-[More Information Needed]
-### Framework versions
+---
 
-- PEFT 0.18.1
+## Project Structure
+
+```
+├── demo.py                       # Streamlit app
+├── system1_baseline.py           # Baseline LLM (Groq)
+├── system2_rag.py                # RAG pipeline (FAISS + PDF support + Groq)
+├── system3_inference.py          # Fine-tuned model inference (Qwen2.5-1.5B)
+├── system3_finetune_colab.ipynb  # LoRA training notebook
+├── data/
+│   ├── docs/                     # 22 knowledge base documents (.txt)
+│   ├── pdfs/                     # Drop PDFs here to expand the knowledge base
+│   ├── faiss_index/              # FAISS vector store (auto-built)
+│   ├── reference_answers.json    # Reference answers for ROUGE-L / semantic similarity
+│   ├── finetune_data.jsonl       # Training data for fine-tuning
+│   └── questions.csv             # Evaluation question set
+├── finetuned_model/              # LoRA adapter (after training)
+└── requirements.txt
+```