Adityapande403/VisDoc-AI

VisDoc AI: Multimodal Visual Document Retrieval with Late Interaction & Quantization

VisDoc AI is a visual document retrieval engine built on Vision-Language Models (ColQwen2.5 / ColPali) and multi-vector late-interaction MaxSim scoring.

It indexes and searches complex visual documents—engineering datasheets, electrical schematics, PCB diagrams, financial balance sheets, commercial invoices, and stamped inspection certificates—without OCR-based text chunking.

• GitHub Repository: https://github.com/adityapande403/VisDoc-AI
• Base Architecture: vidore/colqwen2.5-v0.2 (Qwen/Qwen2.5-VL-3B-Instruct backbone)
• Output Representation: Multi-vector token embeddings with dimension $D = 128$ and unit L2-normalization.
• Scoring Function: MaxSim Late Interaction: $$S(Q, D) = \sum_{i=1}^{N_q} M_{q, i} \max_{j \in D} (q_i \cdot d_j)$$

---

Benchmark Results (Empirically Measured)

1. Retrieval Quality & Compression Retention (Held-Out Test Set)

Representation	NDCG@5	MRR@10	Recall@5	Relative NDCG@5 Retention	Embedding Payload	Compression vs FP16
FP16 Baseline	0.5132	0.4361	0.8333	100.0% (Baseline)	0.09 MB	1.0x
INT8 Scalar	0.5132	0.4361	0.8333	100.0%	0.05 MB	2.0x
1-Bit Binary	0.3747	0.2556	0.8333	73.0%	0.01 MB	16.0x (32x vs FP32)

2. 10,000-Page Scale Benchmark (5,000,000 Multi-Vectors)

Index Format	Total Indexed Pages	Embedding Memory (MB)	Compression vs FP16	Mean Latency (ms)	P95 Latency (ms)	Throughput (QPS)
FP16	10,000	1,220.7 MB	1.0x	2,028.90 ms	2,135.90 ms	0.5
INT8	10,000	610.4 MB	2.0x	3,013.15 ms	3,083.47 ms	0.3
BINARY	10,000	76.3 MB	16.0x	2,770.02 ms	2,890.07 ms	0.4

Two-stage candidate filtering reduces 1,000-page latency from 253.26 ms to 10.69 ms (23.7x acceleration).

---

Quickstart

from visdoc import VisualDocumentRetriever

# Initialize with INT8 compressed index (100% NDCG@5 retention, 2x memory reduction)
retriever = VisualDocumentRetriever(index_format="int8")

# Ingest and index PDF document pages
retriever.index_documents(paths=["./datasheet.pdf"])

# Run late-interaction visual search
results = retriever.search(
    query="Which pin provides the 5.0 V analog sensor supply on the TX-4000?",
    top_k=5,
)

for result in results:
    print(f"Document: {result.document_id} | Page: {result.page_number} | Score: {result.score:.4f}")

---

Citation & Reference

@software{visdoc_ai_2026,
  author = {Aditya Pande},
  title = {VisDoc AI: Multimodal Visual Document Retrieval with ColPali, ColQwen and Late Interaction},
  url = {https://github.com/adityapande403/VisDoc-AI},
  year = {2026}
}