projects/document_rag/data/samples/evaluation_notes.md

Evaluation notes

These are project-authored demonstration notes for Document RAG Lab.

The included retrieval check uses questions written against the demonstration documents. Each question has an expected source file. Recall at one checks whether the first retrieved source is expected. Recall at three checks whether that source appears among the first three results.

This small development fixture is used to catch retrieval regressions. It does not estimate answer accuracy on customer documents. Its questions were authored together with the sample corpus.

The report saves per-question rankings, measured search times, source hashes, and the fixture hash. Re-run the check after changing the embedding model, chunking, or reranking code.

A separate answer-quality study would need an independently labeled question set, a fixed model version, repeated runs, and human checks of factual support. No such study is claimed by this demo.