| ---
|
| language:
|
| - en
|
| license: mit
|
| library_name: colpali-engine
|
| tags:
|
| - colpali
|
| - colqwen
|
| - multi-vector
|
| - late-interaction
|
| - visual-document-retrieval
|
| - document-ai
|
| - quantization
|
| - maxsim
|
| base_model: vidore/colqwen2.5-v0.2
|
| pipeline_tag: feature-extraction
|
| ---
|
|
|
| # VisDoc AI: Multimodal Visual Document Retrieval with Late Interaction & Quantization
|
|
|
| **VisDoc AI** is a visual document retrieval engine built on Vision-Language Models (`ColQwen2.5` / `ColPali`) and multi-vector late-interaction MaxSim scoring.
|
|
|
| It indexes and searches complex visual documents—**engineering datasheets, electrical schematics, PCB diagrams, financial balance sheets, commercial invoices, and stamped inspection certificates**—without OCR-based text chunking.
|
|
|
| - **GitHub Repository**: [https://github.com/adityapande403/VisDoc-AI](https://github.com/adityapande403/VisDoc-AI)
|
| - **Base Architecture**: `vidore/colqwen2.5-v0.2` (`Qwen/Qwen2.5-VL-3B-Instruct` backbone)
|
| - **Output Representation**: Multi-vector token embeddings with dimension $D = 128$ and unit L2-normalization.
|
| - **Scoring Function**: MaxSim Late Interaction:
|
| $$S(Q, D) = \sum_{i=1}^{N_q} M_{q, i} \max_{j \in D} (q_i \cdot d_j)$$
|
|
|
| ---
|
|
|
| ## Benchmark Results (Empirically Measured)
|
|
|
| ### 1. Retrieval Quality & Compression Retention (Held-Out Test Set)
|
|
|
| | Representation | NDCG@5 | MRR@10 | Recall@5 | Relative NDCG@5 Retention | Embedding Payload | Compression vs FP16 |
|
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
| | **FP16 Baseline** | 0.5132 | 0.4361 | 0.8333 | **100.0%** (Baseline) | 0.09 MB | 1.0x |
|
| | **INT8 Scalar** | 0.5132 | 0.4361 | 0.8333 | **100.0%** | 0.05 MB | **2.0x** |
|
| | **1-Bit Binary** | 0.3747 | 0.2556 | 0.8333 | **73.0%** | 0.01 MB | **16.0x** (32x vs FP32) |
|
|
|
| ### 2. 10,000-Page Scale Benchmark (5,000,000 Multi-Vectors)
|
|
|
| | Index Format | Total Indexed Pages | Embedding Memory (MB) | Compression vs FP16 | Mean Latency (ms) | P95 Latency (ms) | Throughput (QPS) |
|
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: |
|
| | **FP16** | 10,000 | 1,220.7 MB | 1.0x | 2,028.90 ms | 2,135.90 ms | 0.5 |
|
| | **INT8** | 10,000 | 610.4 MB | **2.0x** | 3,013.15 ms | 3,083.47 ms | 0.3 |
|
| | **BINARY** | 10,000 | **76.3 MB** | **16.0x** | 2,770.02 ms | 2,890.07 ms | 0.4 |
|
|
|
| *Two-stage candidate filtering reduces 1,000-page latency from 253.26 ms to **10.69 ms** (23.7x acceleration).*
|
|
|
| ---
|
|
|
| ## Quickstart
|
|
|
| ```python
|
| from visdoc import VisualDocumentRetriever
|
|
|
| # Initialize with INT8 compressed index (100% NDCG@5 retention, 2x memory reduction)
|
| retriever = VisualDocumentRetriever(index_format="int8")
|
|
|
| # Ingest and index PDF document pages
|
| retriever.index_documents(paths=["./datasheet.pdf"])
|
|
|
| # Run late-interaction visual search
|
| results = retriever.search(
|
| query="Which pin provides the 5.0 V analog sensor supply on the TX-4000?",
|
| top_k=5,
|
| )
|
|
|
| for result in results:
|
| print(f"Document: {result.document_id} | Page: {result.page_number} | Score: {result.score:.4f}")
|
| ```
|
|
|
| ---
|
|
|
| ## Citation & Reference
|
|
|
| ```bibtex
|
| @software{visdoc_ai_2026,
|
| author = {Aditya Pande},
|
| title = {VisDoc AI: Multimodal Visual Document Retrieval with ColPali, ColQwen and Late Interaction},
|
| url = {https://github.com/adityapande403/VisDoc-AI},
|
| year = {2026}
|
| }
|
| ```
|
|
|
