Instructions to use Noor-Khan/chat-with-pdf-rag with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sentence-transformers
How to use Noor-Khan/chat-with-pdf-rag with sentence-transformers:
from sentence_transformers import SentenceTransformer model = SentenceTransformer("Noor-Khan/chat-with-pdf-rag") sentences = [ "The weather is lovely today.", "It's so sunny outside!", "He drove to the stadium." ] embeddings = model.encode(sentences) similarities = model.similarity(embeddings, embeddings) print(similarities.shape) # [3, 3] - Notebooks
- Google Colab
- Kaggle
Chat with PDF (RAG)
Upload a PDF, ask questions about it, and get answers grounded only in that document — with the page each answer comes from. Built by Noorullah Zamindar — original project at NoorullahZamindar-007/chat-with-pdf-rag.
This is a retrieval pipeline plus a thin LLM layer, not a model
trained by Noorullah. There are no weights trained by him and no
training data. The "model" is the pre-trained
sentence-transformers/all-MiniLM-L6-v2 embedding model (Hugging Face,
384 dimensions) plus his RAG code: PDF extraction, chunking, ChromaDB
storage, document-scoped retrieval, and a grounded prompt that tells
the LLM to answer only from the retrieved context — or to say exactly
"I could not find this information in the uploaded PDF."
How it works
- The user uploads a PDF. It is validated (real PDF magic bytes, size limit, password-protected and scanned PDFs rejected with a clear error).
- PyMuPDF extracts the text page by page; the text is cleaned (ligatures, hyphenated line breaks, whitespace).
- LangChain's
RecursiveCharacterTextSplittersplits each page into overlapping chunks (900 characters, 180 overlap). - Each chunk is embedded with all-MiniLM-L6-v2 and stored in ChromaDB with metadata: document ID, file name, page number, chunk index.
- A question is embedded the same way; the top 8 most similar chunks are retrieved, filtered to the current document only, so chunks from another uploaded PDF can never leak into the answer.
- The LLM (Groq
llama-3.1-8b-instant, or a local Ollama model) receives only the retrieved context plus the question, at temperature 0, and answers with source page citations. If the answer is not in the context, it must reply with the not-found sentence.
How to use
pip install -r requirements.txt
cp .env.example .env # then edit it
streamlit run app.py
Upload a text-based PDF, wait for indexing, and ask questions. The chat shows the answer, source page pills, and the retrieved chunks in an expander.
For Groq (cloud, fast):
LLM_PROVIDER=groq
GROQ_API_KEY=your_real_groq_key_here
GROQ_MODEL=llama-3.1-8b-instant
For local Ollama (private, no key):
ollama pull llama3.1
ollama pull nomic-embed-text
LLM_PROVIDER=ollama
EMBEDDING_PROVIDER=ollama
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_LLM_MODEL=llama3.1
OLLAMA_EMBED_MODEL=nomic-embed-text
Without either, the app still indexes PDFs and retrieves chunks, but
answering fails with a clear "GROQ_API_KEY is missing" error — the
retrieval half needs no key at all. Other settings: CHUNK_SIZE
(900), CHUNK_OVERLAP (180), TOP_K (8), MAX_PDF_MB (25).
Measured results
Measured 21 Nov 2026 with evaluate.py (included), Python 3.12.3,
PyMuPDF 2.1, sentence-transformers 6.3.0, ChromaDB 1.5.9, torch
2.15.0. Full numbers in metrics.json. The test document is
samples/sample_policy.pdf, a fictional 3-page "Aurora Office
Supplies — Refund and Returns Policy" written for this evaluation
(it is invented content, not a real company), with 6 questions in
samples/sample_qa.jsonl.
| Measure | Value |
|---|---|
| Sample PDF | 3 pages, 3,211 chars (1,162 / 1,199 / 850) |
| Chunks (900 chars, 180 overlap) | 5 (2 on page 1, 2 on page 2, 1 on page 3); 446–866 chars, mean 707.8 |
| Embedding model | sentence-transformers/all-MiniLM-L6-v2, 384 dimensions |
| Embedding model load (cached) | 32.3 s (51.7 s on the first run, including the ~90 MB download) |
| Embed + store 5 chunks | 3.9 s |
| Retrieval hit rate, 5 answerable questions (top-k = 8) | 5/5 = 1.000 |
| Top-ranked chunk from an expected page | 3/5 |
| Top similarity score, answerable questions | 0.37–0.60 |
| Top similarity score, deliberately unanswerable question ("What is the company's policy on drone delivery?") | 0.0835 — far below every answerable question |
| Retrieval time per question | ~1.5 s (see note below) |
Repo test suite (python -m pytest -q) |
124 passed, 0 failed, 0 skipped (0.76 s) |
| Blank question | Rejected: ValueError: question must not be blank. |
Renamed non-PDF file (.pdf extension, text contents) |
Rejected: ValueError: The file does not look like a PDF. |
| Blank PDF with no extractable text (scanned-PDF case) | Rejected: ValueError: No extractable text found. The PDF may be scanned and OCR is needed. |
Retrieval scoped to a different document_id |
0 chunks returned — no cross-document leak |
| Answer plumbing (stub LLM returning a fixed canned string) | The returned dict carries the stub text and the correct source pages [1, 2, 3] — PASS |
| No documents at all | Returns exactly "I could not find this information in the uploaded PDF." — PASS |
Read the hit rate honestly: the sample PDF produces only 5 chunks and top-k is 8, so every question retrieves all 5 chunks — the 1.000 hit rate proves the pipeline works end to end, not that the ranking is discriminative. The ranking signal is the top-1 figure: the top-ranked chunk came from an expected page for 3 of the 5 questions. The two misses are instructive, not random: "What condition must a returned item be in?" ranked a page-2 chunk first (page 2 also discusses return conditions), and "Who is the Returns Manager?" ranked a page-1 chunk first (the question repeats "Aurora Office Supplies", which page 1 mentions several times) — the correct page-3 chunk ranked second. On a larger document with more chunks than k, hit rate would measure ranking properly; on this 3-page sample it cannot.
Per-question results (expected page → retrieved pages, top score):
| Question | Expected | Retrieved | Top chunk page | Top score |
|---|---|---|---|---|
| How many days after delivery can I return an item? | 1 | 1, 2, 3 | 1 | 0.3983 |
| What condition must a returned item be in? | 1 | 1, 2, 3 | 2 | 0.5397 |
| How long does a refund to my original payment method take? | 2 | 1, 2, 3 | 2 | 0.3749 |
| Which items cannot be returned? | 2 | 1, 2, 3 | 2 | 0.5968 |
| Who is the Returns Manager at Aurora Office Supplies? | 3 | 1, 2, 3 | 1 | 0.4988 |
| What is the company's policy on drone delivery? (not in PDF) | — | 1, 2, 3 | 1 | 0.0835 |
The shipped evaluate.py reproduces every number above from a clean
copy of this repo (verified: a second run in a fresh directory gave
identical hit rate, per-question pages, and similarity scores). It
needs internet on the first run to download the embedding model
(~90 MB, cached afterwards).
What was NOT measured: end-to-end answer quality. There was no
Groq API key and no Ollama server in the evaluation environment, so no
real LLM answer was ever generated here — no answer accuracy, no
hallucination rate, and no check that a real LLM actually says the
not-found sentence for the drone-delivery question. Retrieval was
measured locally; answer generation was not. The repo's
run_basic_evaluation was deliberately not run for the same reason
(it needs a real LLM). To measure answers, run the app with
LLM_PROVIDER=groq and a GROQ_API_KEY, or a local Ollama, and score
the answers against samples/sample_qa.jsonl.
Limitations
- Scanned PDFs (images, no selectable text) are rejected — there is no OCR; run OCR first.
- Single-PDF scope: one document is indexed at a time; uploading a new PDF replaces the old collection.
- Retrieval is top-k cosine similarity, not exact search — near-duplicate wording on another page can outrank the right page (see the two top-1 misses above).
- The grounded prompt instructs the LLM not to use outside knowledge, but that is a prompt instruction, not a guarantee — a real LLM can still hallucinate, which is why answer quality needs measuring with a real key (see above).
- Retrieval re-instantiates the embedding model on every call in the
retrieve_relevant_chunkspath (~1.5 s per question here); the Streamlit app avoids this by caching the vector store in session state. - ChromaDB relevance scores can be negative (cosine similarity), which
makes LangChain emit a
UserWarning: Relevance scores must be between 0 and 1— cosmetic; the ranking itself is unaffected. - The embedding model is Hugging Face's pre-trained all-MiniLM-L6-v2, not a model trained by Noorullah — it embeds text, it does not know this PDF.
Files
app.py— the Streamlit app (upload, index, chat, source citations), unchanged from the repo.src/— his RAG code:pdf_loader.py,text_cleaner.py,chunker.py,embeddings.py,vector_store.py,retriever.py,llm.py,rag_chain.py,evaluation.py,config.py,utils.py,logger.py.evaluate.py— reproduces every measured number above; writesmetrics.jsonandsamples/eval_results.csv.metrics.json— the measured results.samples/— the fictional 3-page test PDF, the 6-question QA file, and the per-question evaluation rows.requirements.txt— pinned to the versions actually used for the measurement..env.example— copy to.envand add your Groq key (never commit.env).