Chat with PDF (RAG)

Upload a PDF, ask questions about it, and get answers grounded only in that document — with the page each answer comes from. Built by Noorullah Zamindar — original project at NoorullahZamindar-007/chat-with-pdf-rag.

This is a retrieval pipeline plus a thin LLM layer, not a model trained by Noorullah. There are no weights trained by him and no training data. The "model" is the pre-trained sentence-transformers/all-MiniLM-L6-v2 embedding model (Hugging Face, 384 dimensions) plus his RAG code: PDF extraction, chunking, ChromaDB storage, document-scoped retrieval, and a grounded prompt that tells the LLM to answer only from the retrieved context — or to say exactly "I could not find this information in the uploaded PDF."

How it works

  1. The user uploads a PDF. It is validated (real PDF magic bytes, size limit, password-protected and scanned PDFs rejected with a clear error).
  2. PyMuPDF extracts the text page by page; the text is cleaned (ligatures, hyphenated line breaks, whitespace).
  3. LangChain's RecursiveCharacterTextSplitter splits each page into overlapping chunks (900 characters, 180 overlap).
  4. Each chunk is embedded with all-MiniLM-L6-v2 and stored in ChromaDB with metadata: document ID, file name, page number, chunk index.
  5. A question is embedded the same way; the top 8 most similar chunks are retrieved, filtered to the current document only, so chunks from another uploaded PDF can never leak into the answer.
  6. The LLM (Groq llama-3.1-8b-instant, or a local Ollama model) receives only the retrieved context plus the question, at temperature 0, and answers with source page citations. If the answer is not in the context, it must reply with the not-found sentence.

How to use

pip install -r requirements.txt
cp .env.example .env   # then edit it
streamlit run app.py

Upload a text-based PDF, wait for indexing, and ask questions. The chat shows the answer, source page pills, and the retrieved chunks in an expander.

For Groq (cloud, fast):

LLM_PROVIDER=groq
GROQ_API_KEY=your_real_groq_key_here
GROQ_MODEL=llama-3.1-8b-instant

For local Ollama (private, no key):

ollama pull llama3.1
ollama pull nomic-embed-text
LLM_PROVIDER=ollama
EMBEDDING_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_LLM_MODEL=llama3.1
OLLAMA_EMBED_MODEL=nomic-embed-text

Without either, the app still indexes PDFs and retrieves chunks, but answering fails with a clear "GROQ_API_KEY is missing" error — the retrieval half needs no key at all. Other settings: CHUNK_SIZE (900), CHUNK_OVERLAP (180), TOP_K (8), MAX_PDF_MB (25).

Measured results

Measured 21 Nov 2026 with evaluate.py (included), Python 3.12.3, PyMuPDF 2.1, sentence-transformers 6.3.0, ChromaDB 1.5.9, torch 2.15.0. Full numbers in metrics.json. The test document is samples/sample_policy.pdf, a fictional 3-page "Aurora Office Supplies — Refund and Returns Policy" written for this evaluation (it is invented content, not a real company), with 6 questions in samples/sample_qa.jsonl.

Measure Value
Sample PDF 3 pages, 3,211 chars (1,162 / 1,199 / 850)
Chunks (900 chars, 180 overlap) 5 (2 on page 1, 2 on page 2, 1 on page 3); 446–866 chars, mean 707.8
Embedding model sentence-transformers/all-MiniLM-L6-v2, 384 dimensions
Embedding model load (cached) 32.3 s (51.7 s on the first run, including the ~90 MB download)
Embed + store 5 chunks 3.9 s
Retrieval hit rate, 5 answerable questions (top-k = 8) 5/5 = 1.000
Top-ranked chunk from an expected page 3/5
Top similarity score, answerable questions 0.37–0.60
Top similarity score, deliberately unanswerable question ("What is the company's policy on drone delivery?") 0.0835 — far below every answerable question
Retrieval time per question ~1.5 s (see note below)
Repo test suite (python -m pytest -q) 124 passed, 0 failed, 0 skipped (0.76 s)
Blank question Rejected: ValueError: question must not be blank.
Renamed non-PDF file (.pdf extension, text contents) Rejected: ValueError: The file does not look like a PDF.
Blank PDF with no extractable text (scanned-PDF case) Rejected: ValueError: No extractable text found. The PDF may be scanned and OCR is needed.
Retrieval scoped to a different document_id 0 chunks returned — no cross-document leak
Answer plumbing (stub LLM returning a fixed canned string) The returned dict carries the stub text and the correct source pages [1, 2, 3] — PASS
No documents at all Returns exactly "I could not find this information in the uploaded PDF." — PASS

Read the hit rate honestly: the sample PDF produces only 5 chunks and top-k is 8, so every question retrieves all 5 chunks — the 1.000 hit rate proves the pipeline works end to end, not that the ranking is discriminative. The ranking signal is the top-1 figure: the top-ranked chunk came from an expected page for 3 of the 5 questions. The two misses are instructive, not random: "What condition must a returned item be in?" ranked a page-2 chunk first (page 2 also discusses return conditions), and "Who is the Returns Manager?" ranked a page-1 chunk first (the question repeats "Aurora Office Supplies", which page 1 mentions several times) — the correct page-3 chunk ranked second. On a larger document with more chunks than k, hit rate would measure ranking properly; on this 3-page sample it cannot.

Per-question results (expected page → retrieved pages, top score):

Question Expected Retrieved Top chunk page Top score
How many days after delivery can I return an item? 1 1, 2, 3 1 0.3983
What condition must a returned item be in? 1 1, 2, 3 2 0.5397
How long does a refund to my original payment method take? 2 1, 2, 3 2 0.3749
Which items cannot be returned? 2 1, 2, 3 2 0.5968
Who is the Returns Manager at Aurora Office Supplies? 3 1, 2, 3 1 0.4988
What is the company's policy on drone delivery? (not in PDF) — 1, 2, 3 1 0.0835

The shipped evaluate.py reproduces every number above from a clean copy of this repo (verified: a second run in a fresh directory gave identical hit rate, per-question pages, and similarity scores). It needs internet on the first run to download the embedding model (~90 MB, cached afterwards).

What was NOT measured: end-to-end answer quality. There was no Groq API key and no Ollama server in the evaluation environment, so no real LLM answer was ever generated here — no answer accuracy, no hallucination rate, and no check that a real LLM actually says the not-found sentence for the drone-delivery question. Retrieval was measured locally; answer generation was not. The repo's run_basic_evaluation was deliberately not run for the same reason (it needs a real LLM). To measure answers, run the app with LLM_PROVIDER=groq and a GROQ_API_KEY, or a local Ollama, and score the answers against samples/sample_qa.jsonl.

Limitations

  • Scanned PDFs (images, no selectable text) are rejected — there is no OCR; run OCR first.
  • Single-PDF scope: one document is indexed at a time; uploading a new PDF replaces the old collection.
  • Retrieval is top-k cosine similarity, not exact search — near-duplicate wording on another page can outrank the right page (see the two top-1 misses above).
  • The grounded prompt instructs the LLM not to use outside knowledge, but that is a prompt instruction, not a guarantee — a real LLM can still hallucinate, which is why answer quality needs measuring with a real key (see above).
  • Retrieval re-instantiates the embedding model on every call in the retrieve_relevant_chunks path (~1.5 s per question here); the Streamlit app avoids this by caching the vector store in session state.
  • ChromaDB relevance scores can be negative (cosine similarity), which makes LangChain emit a UserWarning: Relevance scores must be between 0 and 1 — cosmetic; the ranking itself is unaffected.
  • The embedding model is Hugging Face's pre-trained all-MiniLM-L6-v2, not a model trained by Noorullah — it embeds text, it does not know this PDF.

Files

  • app.py — the Streamlit app (upload, index, chat, source citations), unchanged from the repo.
  • src/ — his RAG code: pdf_loader.py, text_cleaner.py, chunker.py, embeddings.py, vector_store.py, retriever.py, llm.py, rag_chain.py, evaluation.py, config.py, utils.py, logger.py.
  • evaluate.py — reproduces every measured number above; writes metrics.json and samples/eval_results.csv.
  • metrics.json — the measured results.
  • samples/ — the fictional 3-page test PDF, the 6-question QA file, and the per-question evaluation rows.
  • requirements.txt — pinned to the versions actually used for the measurement.
  • .env.example — copy to .env and add your Groq key (never commit .env).
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support