Major upgrade: auto-metrics, Plotly charts, Qwen2.5, PDF support, expanded KB

- Add ROUGE-L and semantic similarity auto-metrics per answer card
- Add Plotly bar charts for aggregate quality/speed comparison in Analytics section
- Upgrade fine-tuned model from TinyLlama to Qwen/Qwen2.5-1.5B-Instruct
- Update Colab notebook with Qwen2.5 model and ChatML prompt format
- Add PDF ingestion to RAG pipeline (drop PDFs into data/pdfs/)
- Add 5 new knowledge base docs: big_o_notation, heaps, sql_basics, typescript_basics, git_basics
- Add data/reference_answers.json with 21 Q&A pairs for automated scoring
- Add HuggingFace Spaces frontmatter and deploy guide to README
- Add plotly, rouge-score, pypdf, langchain-huggingface, numpy to requirements

README.md

@@ -1,3 +1,14 @@
+---
+title: CodeSage
+emoji: 🧙
+colorFrom: blue
+colorTo: purple
+sdk: streamlit
+sdk_version: 1.45.0
+app_file: demo.py
+pinned: false
+---
+
 # 🧙 CodeSage
 
 **A comparative study of three LLM strategies for domain-specific question answering.**
@@ -24,9 +35,9 @@ There is no universal winner. The goal of CodeSage is to **make that trade-off v
 
 1. You type (or click) a programming question.
 2. All three systems answer it simultaneously.
-3. You see each answer, the RAG system's retrieved context, and response times.
+3. Each answer shows auto-computed **ROUGE-L** and **semantic similarity** scores against a reference answer.
 4. You rate each answer (correctness, completeness, hallucination).
-5. A live metrics table aggregates your ratings across all questions.
+5. A live metrics table and **Plotly bar charts** aggregate your ratings across all questions.
 
 ---
 
@@ -42,7 +53,7 @@ There is no universal winner. The goal of CodeSage is to **make that trade-off v
 └─────────┼─────────────────┼────────────────┼─────────┘
           │                 │                │
           ▼                 ▼                ▼
-   Groq API           FAISS index      TinyLlama 1.1B
+   Groq API           FAISS index      Qwen2.5-1.5B
    Llama 3.1 8B    +  HuggingFace   (LoRA adapter loaded
    (prompt only)      embeddings     locally via PEFT)
                           │
@@ -52,15 +63,15 @@ There is no universal winner. The goal of CodeSage is to **make that trade-off v
 ```
 
 ### System 1 — Baseline LLM
-Sends the question directly to **Llama 3.1 8B** (via Groq) with a short system prompt. No extra knowledge. Represents what an off-the-shelf LLM can do.
+Sends the question directly to **Llama 3.1 8B** (via Groq) with a short system prompt. No extra knowledge.
 
 ### System 2 — RAG (Retrieval-Augmented Generation)
 1. The question is embedded with `all-MiniLM-L6-v2` (HuggingFace Sentence Transformers).
-2. Top-3 most similar chunks are retrieved from a **FAISS** vector store built from 17 hand-written `.txt` documents covering DSA topics and web dev concepts.
+2. Top-3 most similar chunks are retrieved from a **FAISS** vector store built from 22 documents (`.txt` files + any PDFs dropped into `data/pdfs/`).
 3. Those chunks are injected as context into the prompt sent to **Llama 3.1 8B** via Groq.
 
 ### System 3 — Fine-Tuned Model
-**TinyLlama 1.1B** fine-tuned with **LoRA (Low-Rank Adaptation)** on ~20 curated programming Q&A pairs. Training is done on Google Colab (free GPU). The resulting adapter is loaded locally via the `peft` library for inference.
+**Qwen2.5-1.5B-Instruct** fine-tuned with **LoRA** on ~20 curated programming Q&A pairs. Training is done on Google Colab (free GPU). The adapter is loaded locally via `peft`.
 
 ---
 
@@ -73,13 +84,30 @@ Sends the question directly to **Llama 3.1 8B** (via Groq) with a short system p
 | Embeddings | `sentence-transformers/all-MiniLM-L6-v2` |
 | Vector store | FAISS (CPU) via LangChain |
 | Fine-tuning | HuggingFace Transformers + PEFT (LoRA) |
-| Base model | TinyLlama/TinyLlama-1.1B-Chat-v1.0 |
+| Base model | Qwen/Qwen2.5-1.5B-Instruct |
+| Auto metrics | ROUGE-L (`rouge-score`) + cosine semantic similarity |
+| Charts | Plotly |
 | Training platform | Google Colab |
-| Background animation | HTML5 Canvas (3D particle network via `components.html`) |
+| Background animation | HTML5 Canvas 3D particle network |
+
+---
+
+## Deploy to HuggingFace Spaces
+
+1. Create a new Space at [huggingface.co/new-space](https://huggingface.co/new-space) — SDK: **Streamlit**
+2. Push this repo to the Space:
+   ```bash
+   git remote add space https://huggingface.co/spaces/YOUR_USERNAME/codesage
+   git push space main
+   ```
+3. Add your `GROQ_API_KEY` as a Space secret (Settings → Variables and secrets)
+4. The app launches automatically — the FAISS index is built on first run (~30s)
+
+> System 3 (fine-tuned) will show "model unavailable" on Spaces unless you upload the `finetuned_model/` adapter folder to the Space repo.
 
 ---
 
-## How to Run
+## How to Run Locally
 
 ### 1. Clone & install
 ```bash
@@ -101,19 +129,24 @@ streamlit run demo.py
 ```
 The FAISS index is built automatically on first launch.
 
-### 4. (Optional) Load the fine-tuned model
+### 4. (Optional) Expand the knowledge base with PDFs
+Drop any `.pdf` files into `data/pdfs/`, then delete `data/faiss_index/` and restart — PDFs are ingested automatically.
+
+### 5. (Optional) Load the fine-tuned model
 1. Open `system3_finetune_colab.ipynb` in Google Colab.
-2. Run all cells — training takes ~10 min on a free T4 GPU.
+2. Run all cells — trains **Qwen2.5-1.5B-Instruct** with LoRA, takes ~15 min on a free T4 GPU.
 3. Download `finetuned_model.zip` and unzip it in the project root.
-4. Restart the Streamlit app — System 3 will now return real answers.
+4. Restart the app — System 3 returns real answers.
 
 ---
 
 ## Knowledge Base
 
-The RAG system retrieves from 17 topic documents in `data/docs/`:
+The RAG system retrieves from 22 documents in `data/docs/`:
+
+`binary_search` · `sorting_algorithms` · `dynamic_programming` · `graph_algorithms` · `trees` · `linked_list` · `stack_queue` · `recursion` · `backtracking` · `greedy_algorithms` · `hashing` · `string_algorithms` · `two_pointers` · `react_hooks` · `rest_api` · `javascript_promises` · `css_flexbox` · `big_o_notation` · `heaps` · `sql_basics` · `typescript_basics` · `git_basics`
 
-`binary_search` · `sorting_algorithms` · `dynamic_programming` · `graph_algorithms` · `trees` · `linked_list` · `stack_queue` · `recursion` · `backtracking` · `greedy_algorithms` · `hashing` · `string_algorithms` · `two_pointers` · `react_hooks` · `rest_api` · `javascript_promises` · `css_flexbox`
+Plus any PDFs placed in `data/pdfs/`.
 
 ---
 
@@ -122,12 +155,14 @@ The RAG system retrieves from 17 topic documents in `data/docs/`:
 ```
 ├── demo.py                       # Streamlit app
 ├── system1_baseline.py           # Baseline LLM (Groq)
-├── system2_rag.py                # RAG pipeline (FAISS + Groq)
-├── system3_inference.py          # Fine-tuned model inference
+├── system2_rag.py                # RAG pipeline (FAISS + PDF support + Groq)
+├── system3_inference.py          # Fine-tuned model inference (Qwen2.5-1.5B)
 ├── system3_finetune_colab.ipynb  # LoRA training notebook
 ├── data/
-│   ├── docs/                     # 17 knowledge base documents
+│   ├── docs/                     # 22 knowledge base documents (.txt)
+│   ├── pdfs/                     # Drop PDFs here to expand the knowledge base
 │   ├── faiss_index/              # FAISS vector store (auto-built)
+│   ├── reference_answers.json    # Reference answers for ROUGE-L / semantic similarity
 │   ├── finetune_data.jsonl       # Training data for fine-tuning
 │   └── questions.csv             # Evaluation question set
 ├── finetuned_model/              # LoRA adapter (after training)

data/docs/big_o_notation.txt

@@ -0,0 +1,38 @@
+Big O Notation and Time Complexity
+
+Big O notation describes the upper bound of an algorithm's time or space complexity as input size (n) grows. It answers: "How does performance scale?"
+
+Common Complexities (best to worst):
+- O(1) – Constant: always same time. Example: array index access, hash map lookup.
+- O(log n) – Logarithmic: halves problem each step. Example: binary search, balanced BST operations.
+- O(n) – Linear: proportional to input. Example: linear search, single loop over array.
+- O(n log n) – Linearithmic: divide and conquer sorts. Example: merge sort, heap sort, quick sort (average).
+- O(n²) – Quadratic: nested loops. Example: bubble sort, insertion sort, selection sort.
+- O(2^n) – Exponential: doubles with each addition. Example: recursive Fibonacci, subset generation.
+- O(n!) – Factorial: permutations. Example: brute-force traveling salesman.
+
+Rules for calculating Big O:
+1. Drop constants: O(2n) → O(n)
+2. Drop lower-order terms: O(n² + n) → O(n²)
+3. Different inputs = different variables: two arrays of size a and b → O(a + b) not O(n²)
+
+Space Complexity:
+- Also uses Big O notation
+- Measures extra memory used by the algorithm (not including input)
+- Example: merge sort uses O(n) extra space for the merge step
+
+Best, Average, Worst Case:
+- Best case (Ω): minimum operations needed (often not useful)
+- Average case (Θ): expected operations for random input
+- Worst case (O): maximum operations (most commonly used for guarantees)
+
+Amortized Analysis:
+- Average cost per operation over a sequence
+- Example: dynamic array append is O(1) amortized even though occasional resize is O(n)
+
+Common Data Structure Operations:
+- Array: access O(1), search O(n), insert/delete O(n)
+- Linked List: access O(n), search O(n), insert/delete at head O(1)
+- Hash Table: search/insert/delete average O(1), worst O(n)
+- BST (balanced): search/insert/delete O(log n)
+- Heap: insert O(log n), get-min/max O(1), delete O(log n)

data/docs/git_basics.txt

@@ -0,0 +1,62 @@
+Git Basics
+
+Git is a distributed version control system that tracks changes in source code over time.
+
+Setup:
+  git config --global user.name "Your Name"
+  git config --global user.email "you@example.com"
+
+Core Workflow:
+  git init              — initialize a new repository
+  git clone <url>       — copy a remote repository locally
+  git status            — show working tree status (staged, unstaged, untracked)
+  git add <file>        — stage a file for commit
+  git add .             — stage all changes
+  git commit -m "msg"   — save staged changes with a message
+  git push origin main  — push commits to remote
+  git pull              — fetch and merge from remote
+
+Branching:
+  git branch              — list all local branches
+  git branch feature-x    — create a new branch
+  git checkout feature-x  — switch to branch (older syntax)
+  git switch feature-x    — switch to branch (newer syntax)
+  git checkout -b feature-x  — create and switch in one step
+  git merge feature-x     — merge branch into current branch
+  git branch -d feature-x — delete a branch (after merge)
+
+Remote:
+  git remote -v            — show remote URLs
+  git remote add origin <url>  — add a remote
+  git fetch origin         — download remote changes (don't merge)
+  git pull origin main     — fetch + merge remote main into current branch
+
+Undoing Changes:
+  git restore <file>          — discard unstaged changes in a file
+  git restore --staged <file> — unstage a file
+  git revert <commit>         — create a new commit that undoes a commit (safe)
+  git reset --soft HEAD~1     — undo last commit, keep changes staged
+  git reset --hard HEAD~1     — undo last commit AND discard changes (destructive)
+
+Viewing History:
+  git log                 — full commit history
+  git log --oneline       — compact one-line per commit
+  git diff                — show unstaged changes
+  git diff --staged       — show staged changes
+  git show <commit>       — show changes in a specific commit
+
+Stashing:
+  git stash               — temporarily save uncommitted changes
+  git stash pop           — restore last stash and remove it
+  git stash list          — show all stashes
+
+Rebasing vs Merging:
+- Merge: creates a merge commit, preserves history of both branches
+- Rebase: replays commits on top of another branch, creates linear history
+  git rebase main         — rebase current branch onto main
+- Rule: never rebase commits that have been pushed to a shared remote
+
+.gitignore:
+- Lists files/patterns Git should not track
+- Common entries: node_modules/, __pycache__/, .env, *.pyc, dist/, build/
+- Check gitignore.io for language-specific templates

data/docs/heaps.txt

@@ -0,0 +1,46 @@
+Heaps and Priority Queues
+
+A heap is a complete binary tree that satisfies the heap property:
+- Min-Heap: every parent node is smaller than or equal to its children. Root is the minimum.
+- Max-Heap: every parent node is greater than or equal to its children. Root is the maximum.
+
+Heap Properties:
+- Complete binary tree: all levels fully filled except possibly the last, filled left to right.
+- Stored efficiently as an array: for node at index i, left child = 2i+1, right child = 2i+2, parent = (i-1)//2.
+
+Core Operations and Complexities:
+- Insert (push): Add at end, bubble up (heapify up). O(log n)
+- Get min/max: Return root. O(1)
+- Remove min/max (pop): Replace root with last element, bubble down (heapify down). O(log n)
+- Build heap from array (heapify): O(n) — better than inserting one by one O(n log n)
+- Search: O(n) — heaps do not support efficient search
+
+Priority Queue:
+- Abstract data type where each element has a priority
+- Element with highest (or lowest) priority is dequeued first
+- Implemented with a heap
+- Python: use `heapq` module (min-heap by default)
+  - heapq.heappush(heap, item)
+  - heapq.heappop(heap) — removes and returns smallest
+  - heapq.heapify(list) — converts list to heap in O(n)
+  - For max-heap in Python: negate values (push -val, pop and negate result)
+
+Common Use Cases:
+- Dijkstra's shortest path algorithm
+- Prim's minimum spanning tree
+- K largest / K smallest elements
+- Merge K sorted lists
+- Task scheduling / job queues
+- Median maintenance (two heaps: max-heap for lower half, min-heap for upper half)
+
+Heap Sort:
+1. Build a max-heap from the array: O(n)
+2. Repeatedly extract max and place at end: O(n log n)
+3. Total: O(n log n) time, O(1) space (in-place)
+- Not stable (doesn't preserve relative order of equal elements)
+
+Example — K largest elements using min-heap of size K:
+- Maintain a min-heap of the K largest seen so far
+- For each element, push it; if heap size > K, pop the smallest
+- Final heap contains K largest elements
+- Time: O(n log k), Space: O(k)

data/docs/sql_basics.txt

@@ -0,0 +1,61 @@
+SQL Basics
+
+SQL (Structured Query Language) is used to manage and query relational databases.
+
+Core Statements:
+- SELECT: retrieve data
+- INSERT: add new rows
+- UPDATE: modify existing rows
+- DELETE: remove rows
+- CREATE TABLE: define a new table
+- DROP TABLE: delete a table
+
+SELECT Syntax:
+  SELECT column1, column2 FROM table_name WHERE condition ORDER BY column LIMIT n;
+- SELECT *: selects all columns (avoid in production for performance)
+- DISTINCT: removes duplicate rows — SELECT DISTINCT city FROM users
+- WHERE: filters rows — WHERE age > 18 AND country = 'India'
+- ORDER BY: sorts results — ORDER BY name ASC / DESC
+- LIMIT: restricts number of rows returned — LIMIT 10
+
+Aggregate Functions:
+- COUNT(*): number of rows
+- SUM(col): total of numeric column
+- AVG(col): average value
+- MIN(col) / MAX(col): minimum and maximum values
+- Used with GROUP BY to aggregate per group
+
+GROUP BY and HAVING:
+  SELECT department, AVG(salary) FROM employees GROUP BY department HAVING AVG(salary) > 50000;
+- GROUP BY groups rows sharing a value into summary rows
+- HAVING filters groups (like WHERE but for aggregates)
+
+JOINs — combining tables:
+- INNER JOIN: only rows with matching values in both tables
+- LEFT JOIN: all rows from left table, matched rows from right (NULL if no match)
+- RIGHT JOIN: all rows from right table, matched rows from left
+- FULL OUTER JOIN: all rows from both tables
+
+  SELECT orders.id, users.name FROM orders INNER JOIN users ON orders.user_id = users.id;
+
+Subqueries:
+  SELECT name FROM employees WHERE salary > (SELECT AVG(salary) FROM employees);
+- Can appear in WHERE, FROM, or SELECT clause
+
+Indexes:
+- Speed up SELECT queries at the cost of slower INSERT/UPDATE/DELETE
+- CREATE INDEX idx_name ON table(column)
+- Automatically created on PRIMARY KEY and UNIQUE columns
+- Avoid over-indexing — each index takes storage and slows writes
+
+Constraints:
+- PRIMARY KEY: uniquely identifies each row
+- FOREIGN KEY: links to primary key of another table (enforces referential integrity)
+- UNIQUE: no duplicate values in column
+- NOT NULL: column cannot have NULL value
+- CHECK: enforces a condition on column values
+
+Transactions:
+- BEGIN / COMMIT / ROLLBACK
+- ACID properties: Atomicity, Consistency, Isolation, Durability
+- All statements in a transaction succeed together or all fail together

data/docs/typescript_basics.txt

@@ -0,0 +1,72 @@
+TypeScript Basics
+
+TypeScript is a statically typed superset of JavaScript that compiles to plain JavaScript. It adds optional type annotations, interfaces, and generics.
+
+Basic Types:
+  let name: string = "Alice";
+  let age: number = 30;
+  let active: boolean = true;
+  let items: string[] = ["a", "b"];          // array of strings
+  let tuple: [string, number] = ["x", 1];   // fixed-length typed array
+  let value: any = "anything";              // opt out of type checking
+  let unknown: unknown = 5;                 // safer than any — must check type before use
+  let nothing: void = undefined;            // for functions that return nothing
+  let n: null = null;
+  let u: undefined = undefined;
+
+Union and Intersection Types:
+  let id: string | number;          // can be string OR number
+  type AB = TypeA & TypeB;          // must satisfy both types
+
+Type Aliases:
+  type Point = { x: number; y: number };
+  type ID = string | number;
+
+Interfaces:
+  interface User {
+    id: number;
+    name: string;
+    email?: string;   // optional property
+    readonly createdAt: Date;   // cannot be reassigned
+  }
+- Interfaces can be extended: interface Admin extends User { role: string }
+- Classes can implement interfaces: class MyUser implements User { ... }
+
+Functions:
+  function add(a: number, b: number): number { return a + b; }
+  const greet = (name: string): string => `Hello, ${name}`;
+  function log(msg: string, level?: string): void { ... }  // optional param
+
+Generics:
+  function identity<T>(arg: T): T { return arg; }
+  const result = identity<string>("hello");
+
+  interface ApiResponse<T> { data: T; status: number; }
+  // Can use any type: ApiResponse<User>, ApiResponse<string[]>
+
+Enums:
+  enum Direction { Up = "UP", Down = "DOWN", Left = "LEFT", Right = "RIGHT" }
+  let dir: Direction = Direction.Up;
+
+Type Assertions:
+  const input = document.getElementById("name") as HTMLInputElement;
+  input.value = "Alice";
+
+keyof and typeof:
+  type Keys = keyof User;          // "id" | "name" | "email" | "createdAt"
+  type MyType = typeof someObject; // infer type from value
+
+Utility Types:
+  Partial<T>     — all properties optional
+  Required<T>    — all properties required
+  Readonly<T>    — all properties readonly
+  Pick<T, K>     — only keep listed keys
+  Omit<T, K>     — remove listed keys
+  Record<K, V>   — object with keys K and values V
+  ReturnType<T>  — return type of a function type
+
+tsconfig.json key options:
+  "strict": true      — enables all strict type checks
+  "target": "ES2020"  — compile output JS version
+  "module": "CommonJS" or "ESNext"
+  "outDir": "./dist"  — output folder

data/reference_answers.json

@@ -0,0 +1,23 @@
+{
+  "what is binary search?": "Binary search is an efficient searching algorithm for sorted arrays. It works by repeatedly dividing the search interval in half. Time complexity: O(log n), Space complexity: O(1). Requires the array to be sorted beforehand.",
+  "what is the time complexity of binary search?": "Binary search has a time complexity of O(log n) because it halves the search space with each step. Best case is O(1) when the middle element is the target. Space complexity is O(1) for iterative implementation.",
+  "what is the difference between stack and queue?": "Stack uses LIFO (Last In First Out) - the last element added is the first to be removed. Operations: push and pop. Queue uses FIFO (First In First Out) - the first element added is the first to be removed. Operations: enqueue and dequeue. Both have O(1) operations.",
+  "stack vs queue?": "Stack uses LIFO (Last In First Out) - the last element added is the first to be removed. Operations: push and pop. Queue uses FIFO (First In First Out) - the first element added is the first to be removed. Operations: enqueue and dequeue. Both have O(1) operations.",
+  "explain merge sort.": "Merge sort is a divide-and-conquer sorting algorithm. It splits the array in half, recursively sorts each half, then merges the two sorted halves. Time complexity: O(n log n) in all cases. Space complexity: O(n). It is a stable sorting algorithm, making it good for sorting linked lists.",
+  "what is quicksort?": "Quicksort is a divide-and-conquer algorithm that picks a pivot element and partitions the array into elements less than and greater than the pivot, then recursively sorts each partition. Average time: O(n log n), Worst case: O(n^2) when pivot is always smallest or largest element. Space: O(log n).",
+  "what is a linked list?": "A linked list is a linear data structure where elements are stored in nodes. Each node contains data and a pointer to the next node. Types: Singly (one direction), Doubly (both directions), Circular (last points to first). Advantage: dynamic size, O(1) insertion at head. Disadvantage: no random access, O(n) search.",
+  "what is a binary search tree?": "A Binary Search Tree (BST) is a binary tree where for every node, all values in the left subtree are less than the node's value, and all values in the right subtree are greater. This property enables O(log n) average search, insert, and delete operations. Worst case is O(n) for unbalanced trees.",
+  "what is dynamic programming?": "Dynamic programming solves complex problems by breaking them into overlapping subproblems and storing results to avoid recomputation. Two approaches: Memoization (top-down, recursion with cache) and Tabulation (bottom-up, iterative). Requires optimal substructure and overlapping subproblems. Examples: Fibonacci, Knapsack, LCS, Coin Change.",
+  "what are react hooks?": "React hooks are functions that let you use state and lifecycle features in functional components, introduced in React 16.8. Key hooks: useState for state management, useEffect for side effects like API calls, useContext for consuming context, useRef for DOM references. Rules: only call hooks at top level, only in React functions.",
+  "what is usestate in react?": "useState is a React hook for managing local component state. Syntax: const [state, setState] = useState(initialValue). It returns the current state value and a setter function. Calling setState triggers a re-render with the new value. Example: const [count, setCount] = useState(0) then setCount(count + 1) to increment.",
+  "what is useeffect in react?": "useEffect is a React hook for handling side effects like API calls, subscriptions, and DOM updates. It runs after every render by default. The dependency array controls when it runs: empty array [] runs only on mount, [value] runs when value changes. Return a cleanup function to run on unmount. Syntax: useEffect(() => { effect }, [deps]).",
+  "what is a rest api?": "REST API (Representational State Transfer) is an architectural style for web services using standard HTTP methods. GET retrieves resources, POST creates new resources, PUT updates resources completely, PATCH partially updates, DELETE removes resources. REST is stateless meaning each request contains all needed information. JSON is the most common response format.",
+  "what http methods does rest use?": "REST uses five main HTTP methods: GET for reading/retrieving data, POST for creating new resources, PUT for completely updating an existing resource, PATCH for partially updating a resource, and DELETE for removing a resource. These map to CRUD operations: Create (POST), Read (GET), Update (PUT/PATCH), Delete (DELETE).",
+  "what does http status code 404 mean?": "HTTP status code 404 means Not Found. It indicates the server cannot find the requested resource. The URL may be wrong, the resource may have been deleted, or it may never have existed. Other important codes: 200 (OK), 201 (Created), 400 (Bad Request), 401 (Unauthorized), 403 (Forbidden), 500 (Internal Server Error).",
+  "what is css flexbox?": "CSS Flexbox is a layout model that makes it easy to arrange items in a container. Activated by display: flex on the container. Key properties: flex-direction (row or column), justify-content (main axis alignment: flex-start, center, space-between), align-items (cross axis: stretch, center, flex-start), flex-wrap (allow wrapping). Items can have flex-grow to fill available space.",
+  "what is a javascript promise?": "A JavaScript Promise represents an asynchronous operation with three states: pending (initial), fulfilled (success), rejected (failure). Created with new Promise((resolve, reject) => {}). Consumed with .then() for success and .catch() for errors. async/await is syntactic sugar that makes promises look synchronous. Promise.all() runs multiple promises in parallel.",
+  "what is inorder traversal?": "Inorder traversal visits tree nodes in Left, Root, Right order. For a Binary Search Tree, inorder traversal produces nodes in ascending sorted order. Algorithm: recursively visit left subtree, then process current node, then recursively visit right subtree. Time complexity: O(n) where n is number of nodes.",
+  "what is the difference between bfs and dfs?": "BFS (Breadth-First Search) explores level by level using a queue data structure. It finds shortest path in unweighted graphs. DFS (Depth-First Search) explores as deep as possible using a stack or recursion. BFS is better for shortest path problems; DFS is better for cycle detection, topological sort, and finding connected components. Both have O(V+E) time complexity.",
+  "what is memoization?": "Memoization is a top-down dynamic programming technique that caches results of expensive function calls and returns the cached result when the same inputs occur again. It uses recursion combined with a hash map or array to store computed values. For example, Fibonacci with memoization runs in O(n) instead of O(2^n). It trades space for time.",
+  "what is flex-grow in css?": "flex-grow is a CSS Flexbox property that defines how much a flex item grows relative to other items when extra space is available in the container. A value of 0 means the item will not grow. A value of 1 means the item takes equal share of available space. If one item has flex-grow: 2 and another has flex-grow: 1, the first gets twice as much extra space."
+}

demo.py

@@ -1,6 +1,9 @@
 import os
+import json
+import numpy as np
 import streamlit as st
 import streamlit.components.v1 as components
+import plotly.graph_objects as go
 from system1_baseline import ask_baseline
 from system2_rag import ask_rag, load_vectorstore, build_vectorstore
 
@@ -16,6 +19,7 @@ if "last_results"     not in st.session_state: st.session_state.last_results
 if "rating_submitted" not in st.session_state: st.session_state.rating_submitted = False
 if "input_question"   not in st.session_state: st.session_state.input_question   = ""
 if "auto_run"         not in st.session_state: st.session_state.auto_run         = False
+if "auto_metrics"     not in st.session_state: st.session_state.auto_metrics     = {}
 
 # ── CSS ───────────────────────────────────────────────────────────────────────
 st.markdown("""<style>
@@ -281,6 +285,20 @@ div[data-testid="stTextInput"] label {
 .badge-grey   { background: rgba(100,116,139,.1); color: #64748b; border: 1px solid rgba(100,116,139,.25); }
 .metric-note  { color: #334155; font-size: 11px; font-weight: 400; }
 
+/* ── Auto-metric chips ───────────────────────────────────────── */
+.rouge-chip { font-size:11px; font-weight:600; padding:3px 9px; border-radius:99px;
+    background:rgba(251,191,36,.08); color:#fbbf24; border:1px solid rgba(251,191,36,.2); }
+.sim-chip   { font-size:11px; font-weight:600; padding:3px 9px; border-radius:99px;
+    background:rgba(244,114,182,.08); color:#f472b6; border:1px solid rgba(244,114,182,.2); }
+
+/* ── Tabs ────────────────────────────────────────────────────── */
+div[data-testid="stTabs"] button[data-baseweb="tab"] {
+    font-weight: 600; font-size: 13px; color: #475569;
+}
+div[data-testid="stTabs"] button[data-baseweb="tab"][aria-selected="true"] {
+    color: #e2e8f0;
+}
+
 /* ── Scrollbar ───────────────────────────────────────────────── */
 ::-webkit-scrollbar { width: 5px; height: 5px; }
 ::-webkit-scrollbar-track { background: #080d1a; }
@@ -320,6 +338,41 @@ def get_vectorstore():
 
 vs = get_vectorstore()
 
+# ── Reference answers + auto-metrics ─────────────────────────────────────────
+@st.cache_data
+def load_reference_answers():
+    path = "data/reference_answers.json"
+    if os.path.exists(path):
+        with open(path, encoding="utf-8") as f:
+            return json.load(f)
+    return {}
+
+ref_answers = load_reference_answers()
+
+def compute_auto_metrics(answer: str, question: str) -> dict:
+    ref = ref_answers.get(question.strip().lower())
+    if not ref or not answer:
+        return {}
+    try:
+        from rouge_score import rouge_scorer as rs
+        scorer = rs.RougeScorer(["rougeL"], use_stemmer=True)
+        rouge_l = round(scorer.score(ref, answer)["rougeL"].fmeasure, 3)
+        r_emb = np.array(vs.embeddings.embed_query(ref))
+        a_emb = np.array(vs.embeddings.embed_query(answer))
+        norm = np.linalg.norm(r_emb) * np.linalg.norm(a_emb)
+        sem_sim = round(float(np.dot(r_emb, a_emb) / (norm + 1e-8)), 3)
+        return {"rouge_l": rouge_l, "sem_sim": max(0.0, sem_sim)}
+    except Exception:
+        return {}
+
+def metric_chips(m: dict) -> str:
+    if not m:
+        return ""
+    return (
+        f'<span class="rouge-chip">ROUGE-L {m["rouge_l"]:.3f}</span>'
+        f'<span class="sim-chip">Sem.Sim {m["sem_sim"]:.3f}</span>'
+    )
+
 # ── 3D Particle background ────────────────────────────────────────────────────
 components.html("""
 <script>
@@ -480,6 +533,11 @@ if (run_clicked or st.session_state.auto_run) and question:
         "question": question,
         "r1": r1, "r2": r2, "r3": r3,
     }
+    st.session_state.auto_metrics = {
+        "r1": compute_auto_metrics(r1["answer"], question),
+        "r2": compute_auto_metrics(r2["answer"], question),
+        "r3": compute_auto_metrics(r3["answer"], question),
+    }
 
 # ── Render answers ────────────────────────────────────────────────────────────
 if st.session_state.last_results:
@@ -495,6 +553,8 @@ if st.session_state.last_results:
 
     col1, col2, col3 = st.columns(3)
 
+    m = st.session_state.auto_metrics
+
     with col1:
         st.markdown(f"""
         <div class="card card-blue">
@@ -509,6 +569,7 @@ if st.session_state.last_results:
                 <div class="card-answer">{r1['answer']}</div>
                 <div class="card-meta">
                     <span class="time-chip">⏱ {r1['response_time']}s</span>
+                    {metric_chips(m.get('r1', {}))}
                 </div>
             </div>
         </div>
@@ -532,6 +593,7 @@ if st.session_state.last_results:
                 </div>
                 <div class="card-meta">
                     <span class="time-chip">⏱ {r2['response_time']}s</span>
+                    {metric_chips(m.get('r2', {}))}
                 </div>
             </div>
         </div>
@@ -545,7 +607,7 @@ if st.session_state.last_results:
                     <span class="card-icon">🧠</span>
                     <div>
                         <div class="card-title">System 3 — Fine-Tuned Model</div>
-                        <div class="card-subtitle">TinyLlama · LoRA fine-tuning</div>
+                        <div class="card-subtitle">Qwen2.5-1.5B · LoRA fine-tuning</div>
                     </div>
                 </div>
                 <div class="card-body">
@@ -564,13 +626,14 @@ if st.session_state.last_results:
                     <span class="card-icon">🧠</span>
                     <div>
                         <div class="card-title">System 3 — Fine-Tuned Model</div>
-                        <div class="card-subtitle">TinyLlama 1.1B · LoRA on programming Q&A</div>
+                        <div class="card-subtitle">Qwen2.5-1.5B · LoRA on programming Q&A</div>
                     </div>
                 </div>
                 <div class="card-body">
                     <div class="card-answer">{r3['answer']}</div>
                     <div class="card-meta">
                         <span class="time-chip">⏱ {r3['response_time']}s</span>
+                        {metric_chips(m.get('r3', {}))}
                     </div>
                 </div>
             </div>
@@ -624,11 +687,11 @@ if st.session_state.last_results and not st.session_state.rating_submitted:
         st.success(f"Rating saved! Total questions rated: {len(st.session_state.ratings)}")
         st.rerun()
 
-# ── Metrics table ──────────────────────────────────────────────────────────────
+# ── Analytics tab ─────────────────────────────────────────────────────────────
 st.divider()
 st.markdown("""
 <div class="section-header">
-    <span class="section-header-text">Evaluation metrics</span>
+    <span class="section-header-text">📊 Analytics &amp; Evaluation</span>
     <div class="section-header-line"></div>
 </div>
 """, unsafe_allow_html=True)
@@ -668,6 +731,57 @@ else:
     row_complete = (badge(f"{v_p1}/5", color_complete(v_p1)), badge(f"{v_p2}/5", color_complete(v_p2)), badge(f"{v_p3}/5", color_complete(v_p3)))
     row_time     = (badge(f"{v_t1}s",  color_time(v_t1)),     badge(f"{v_t2}s",  color_time(v_t2)),     badge(f"{v_t3}s",  color_time(v_t3)))
 
+    # ── Plotly charts ──────────────────────────────────────────────────────────
+    _systems = ["Baseline LLM", "RAG Chatbot", "Fine-Tuned"]
+    _colors  = ["#3b82f6", "#10b981", "#8b5cf6"]
+    _layout  = dict(
+        paper_bgcolor="rgba(0,0,0,0)", plot_bgcolor="rgba(0,0,0,0)",
+        font=dict(color="#94a3b8", size=12),
+        legend=dict(bgcolor="rgba(0,0,0,0)", bordercolor="#1e293b", borderwidth=1),
+        xaxis=dict(gridcolor="#1a2540", linecolor="#1a2540"),
+        yaxis=dict(gridcolor="#1a2540", linecolor="#1a2540"),
+        margin=dict(l=20, r=20, t=48, b=20),
+    )
+
+    ch1, ch2 = st.columns(2)
+
+    with ch1:
+        fig1 = go.Figure()
+        fig1.add_trace(go.Bar(
+            name="Correctness",  x=_systems, y=[v_c1, v_c2, v_c3],
+            marker_color=_colors, opacity=1.0,
+            text=[f"{v}/5" for v in [v_c1, v_c2, v_c3]], textposition="auto",
+        ))
+        fig1.add_trace(go.Bar(
+            name="Completeness", x=_systems, y=[v_p1, v_p2, v_p3],
+            marker_color=_colors, opacity=0.55,
+            text=[f"{v}/5" for v in [v_p1, v_p2, v_p3]], textposition="auto",
+        ))
+        fig1.update_layout(
+            **_layout, barmode="group", height=300,
+            title=dict(text="Quality Scores (out of 5)", font=dict(color="#e2e8f0", size=14)),
+            yaxis=dict(range=[0, 5], gridcolor="#1a2540", linecolor="#1a2540"),
+        )
+        st.plotly_chart(fig1, use_container_width=True)
+
+    with ch2:
+        fig2 = go.Figure()
+        fig2.add_trace(go.Bar(
+            name="Response Time", x=_systems, y=[v_t1, v_t2, v_t3],
+            marker_color=_colors,
+            text=[f"{v}s" for v in [v_t1, v_t2, v_t3]], textposition="auto",
+        ))
+        fig2.add_trace(go.Bar(
+            name="Hallucination %", x=_systems, y=[v_h1, v_h2, v_h3],
+            marker_color=_colors, opacity=0.55,
+            text=[f"{v}%" for v in [v_h1, v_h2, v_h3]], textposition="auto",
+        ))
+        fig2.update_layout(
+            **_layout, barmode="group", height=300,
+            title=dict(text="Response Time (s) &amp; Hallucination (%)", font=dict(color="#e2e8f0", size=14)),
+        )
+        st.plotly_chart(fig2, use_container_width=True)
+
 if n == 0:
     dash = '<span class="badge badge-grey">—</span>'
     row_correct = row_halluc = row_complete = row_time = (dash, dash, dash)
@@ -722,5 +836,7 @@ with st.expander("About CodeSage"):
 |--------|-------------|
 | Baseline | Llama 3.1 8B via Groq · system prompt only |
 | RAG | FAISS vector store + HuggingFace embeddings + Llama 3.1 8B via Groq |
-| Fine-tuned | TinyLlama 1.1B fine-tuned via LoRA on 20 programming Q&A pairs |
+| Fine-tuned | Qwen2.5-1.5B-Instruct fine-tuned via LoRA on 20 programming Q&A pairs |
+
+Drop any PDF into `data/pdfs/` and delete `data/faiss_index/` to rebuild the RAG knowledge base with it included.
 """)

requirements.txt

@@ -1,6 +1,7 @@
 openai>=1.0.0
 langchain>=0.2.0
 langchain-community>=0.2.0
+langchain-huggingface>=0.0.3
 faiss-cpu>=1.7.4
 sentence-transformers>=2.7.0
 transformers>=4.40.0
@@ -11,3 +12,7 @@ streamlit>=1.35.0
 pandas>=2.0.0
 python-dotenv>=1.0.0
 tqdm>=4.66.0
+plotly>=5.0.0
+rouge-score>=0.1.2
+pypdf>=3.0.0
+numpy>=1.24.0

system2_rag.py

@@ -1,7 +1,8 @@
 import os
+import glob as glob_module
 import time
 from dotenv import load_dotenv
-from langchain_community.document_loaders import DirectoryLoader, TextLoader
+from langchain_community.document_loaders import DirectoryLoader, TextLoader, PyPDFLoader
 from langchain_text_splitters import RecursiveCharacterTextSplitter
 from langchain_community.vectorstores import FAISS
 from langchain_huggingface import HuggingFaceEmbeddings
@@ -16,7 +17,8 @@ client = OpenAI(
 
 MODEL = "llama-3.1-8b-instant"
 
-DOCS_PATH = "data/docs"
+DOCS_PATH  = "data/docs"
+PDFS_PATH  = "data/pdfs"
 INDEX_PATH = "data/faiss_index"
 EMBED_MODEL = "all-MiniLM-L6-v2"
 
@@ -27,11 +29,24 @@ SYSTEM_PROMPT = (
 
 
 def build_vectorstore() -> FAISS:
-    print("Loading documents...")
+    # Load .txt documents
+    print("Loading text documents...")
     loader = DirectoryLoader(DOCS_PATH, glob="**/*.txt", loader_cls=TextLoader,
                              loader_kwargs={"encoding": "utf-8"})
     documents = loader.load()
-    print(f"Loaded {len(documents)} documents.")
+    print(f"  Loaded {len(documents)} text files.")
+
+    # Load PDFs from data/pdfs/ if any exist
+    pdf_files = glob_module.glob(os.path.join(PDFS_PATH, "**/*.pdf"), recursive=True)
+    for pdf_path in pdf_files:
+        try:
+            pdf_loader = PyPDFLoader(pdf_path)
+            documents.extend(pdf_loader.load())
+            print(f"  Loaded PDF: {os.path.basename(pdf_path)}")
+        except Exception as e:
+            print(f"  Warning: Could not load {pdf_path}: {e}")
+
+    print(f"Total documents loaded: {len(documents)} ({len(pdf_files)} PDFs)")
 
     splitter = RecursiveCharacterTextSplitter(chunk_size=400, chunk_overlap=60)
     chunks = splitter.split_documents(documents)
@@ -52,7 +67,6 @@ def load_vectorstore() -> FAISS:
 def ask_rag(question: str, vectorstore: FAISS) -> dict:
     start = time.time()
 
-    # Retrieve top-3 relevant chunks
     docs = vectorstore.similarity_search(question, k=3)
     context = "\n\n".join([doc.page_content for doc in docs])
 

system3_finetune_colab.ipynb

@@ -35,25 +35,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "import json\n",
-    "import torch\n",
-    "from datasets import Dataset\n",
-    "from transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments, Trainer, DataCollatorForLanguageModeling\n",
-    "from peft import LoraConfig, get_peft_model, TaskType\n",
-    "\n",
-    "# Load dataset\n",
-    "data = []\n",
-    "with open('finetune_data.jsonl') as f:\n",
-    "    for line in f:\n",
-    "        data.append(json.loads(line))\n",
-    "\n",
-    "print(f'Loaded {len(data)} training examples')\n",
-    "\n",
-    "def format_prompt(example):\n",
-    "    return f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['output']}\"\n",
-    "\n",
-    "dataset = Dataset.from_list([{'text': format_prompt(d)} for d in data])\n",
-    "print(dataset[0]['text'][:200])"
+    "import json\nimport torch\nfrom datasets import Dataset\nfrom transformers import AutoTokenizer, AutoModelForCausalLM, TrainingArguments, Trainer, DataCollatorForLanguageModeling\nfrom peft import LoraConfig, get_peft_model, TaskType\n\n# Load dataset\ndata = []\nwith open('finetune_data.jsonl') as f:\n    for line in f:\n        data.append(json.loads(line))\n\nprint(f'Loaded {len(data)} training examples')\n\ndef format_prompt(example):\n    return f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['output']}\"\n\ndataset = Dataset.from_list([{'text': format_prompt(d)} for d in data])\nprint(dataset[0]['text'][:200])"
    ]
   },
   {
@@ -62,17 +44,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "MODEL_NAME = 'TinyLlama/TinyLlama-1.1B-Chat-v1.0'\n",
-    "\n",
-    "tokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\n",
-    "tokenizer.pad_token = tokenizer.eos_token\n",
-    "\n",
-    "model = AutoModelForCausalLM.from_pretrained(\n",
-    "    MODEL_NAME,\n",
-    "    torch_dtype=torch.float16,\n",
-    "    device_map='auto'\n",
-    ")\n",
-    "print('Model loaded.')"
+    "MODEL_NAME = 'Qwen/Qwen2.5-1.5B-Instruct'\n\ntokenizer = AutoTokenizer.from_pretrained(MODEL_NAME)\ntokenizer.pad_token = tokenizer.eos_token\n\nmodel = AutoModelForCausalLM.from_pretrained(\n    MODEL_NAME,\n    torch_dtype=torch.float16,\n    device_map='auto'\n)\nprint('Model loaded.')"
    ]
   },
   {
@@ -133,17 +105,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "# Test the fine-tuned model\n",
-    "def ask_finetuned(question):\n",
-    "    prompt = f'### Instruction:\\n{question}\\n\\n### Response:\\n'\n",
-    "    inputs = tokenizer(prompt, return_tensors='pt').to(model.device)\n",
-    "    with torch.no_grad():\n",
-    "        outputs = model.generate(**inputs, max_new_tokens=200, temperature=0.7,\n",
-    "                                  do_sample=True, pad_token_id=tokenizer.eos_token_id)\n",
-    "    full = tokenizer.decode(outputs[0], skip_special_tokens=True)\n",
-    "    return full.split('### Response:\\n')[-1].strip()\n",
-    "\n",
-    "print(ask_finetuned('What is binary search?'))"
+    "# Test the fine-tuned model\ndef ask_finetuned(question):\n    messages = [\n        {'role': 'system', 'content': 'You are a programming tutor specializing in Data Structures, Algorithms, and Web Development.'},\n        {'role': 'user', 'content': question}\n    ]\n    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n    inputs = tokenizer([text], return_tensors='pt').to(model.device)\n    with torch.no_grad():\n        outputs = model.generate(**inputs, max_new_tokens=200, do_sample=False,\n                                  pad_token_id=tokenizer.eos_token_id)\n    generated_ids = outputs[0][inputs.input_ids.shape[1]:]\n    return tokenizer.decode(generated_ids, skip_special_tokens=True).strip()\n\nprint(ask_finetuned('What is binary search?'))"
    ]
   },
   {
@@ -173,4 +135,4 @@
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}

system3_inference.py

@@ -4,6 +4,8 @@ After running system3_finetune_colab.ipynb on Google Colab:
   1. Download finetuned_model.zip from Colab
   2. Unzip into this project root as finetuned_model/
   3. Then run this file
+
+Base model: Qwen/Qwen2.5-1.5B-Instruct (upgraded from TinyLlama)
 """
 import os
 import time
@@ -11,7 +13,7 @@ import torch
 from transformers import AutoTokenizer, AutoModelForCausalLM
 from peft import PeftModel
 
-BASE_MODEL = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
+BASE_MODEL = "Qwen/Qwen2.5-1.5B-Instruct"
 FINETUNED_PATH = "./finetuned_model"
 
 _model = None
@@ -25,7 +27,10 @@ def load_model():
 
     print("Loading fine-tuned model (this takes ~30s first time)...")
     _tokenizer = AutoTokenizer.from_pretrained(FINETUNED_PATH)
-    base = AutoModelForCausalLM.from_pretrained(BASE_MODEL, torch_dtype=torch.float32)
+    base = AutoModelForCausalLM.from_pretrained(
+        BASE_MODEL,
+        torch_dtype=torch.float32,
+    )
     _model = PeftModel.from_pretrained(base, FINETUNED_PATH)
     _model.eval()
     print("Fine-tuned model ready.")
@@ -44,19 +49,32 @@ def ask_finetuned(question: str) -> dict:
     model, tokenizer = load_model()
     start = time.time()
 
-    prompt = f"### Instruction:\n{question}\n\n### Response:\n"
-    inputs = tokenizer(prompt, return_tensors="pt")
+    messages = [
+        {
+            "role": "system",
+            "content": (
+                "You are a programming tutor specializing in Data Structures, "
+                "Algorithms, and Web Development. Answer questions clearly and concisely."
+            ),
+        },
+        {"role": "user", "content": question},
+    ]
+
+    text = tokenizer.apply_chat_template(
+        messages, tokenize=False, add_generation_prompt=True
+    )
+    inputs = tokenizer([text], return_tensors="pt")
 
     with torch.no_grad():
         outputs = model.generate(
             **inputs,
-            max_new_tokens=100,
+            max_new_tokens=150,
             do_sample=False,
             pad_token_id=tokenizer.eos_token_id,
         )
 
-    full_output = tokenizer.decode(outputs[0], skip_special_tokens=True)
-    answer = full_output.split("### Response:\n")[-1].strip()
+    generated_ids = outputs[0][inputs.input_ids.shape[1]:]
+    answer = tokenizer.decode(generated_ids, skip_special_tokens=True).strip()
     elapsed = round(time.time() - start, 2)
 
     return {