feat: upload VisDoc-AI model card, projection weights, and benchmark reports

README.md

@@ -1,3 +1,87 @@
----
-license: mit
----
+---
+language:
+- en
+license: mit
+library_name: colpali-engine
+tags:
+- colpali
+- colqwen
+- multi-vector
+- late-interaction
+- visual-document-retrieval
+- document-ai
+- quantization
+- maxsim
+base_model: vidore/colqwen2.5-v0.2
+pipeline_tag: feature-extraction
+---
+
+# VisDoc AI: Multimodal Visual Document Retrieval with Late Interaction & Quantization
+
+**VisDoc AI** is a visual document retrieval engine built on Vision-Language Models (`ColQwen2.5` / `ColPali`) and multi-vector late-interaction MaxSim scoring.
+
+It indexes and searches complex visual documents—**engineering datasheets, electrical schematics, PCB diagrams, financial balance sheets, commercial invoices, and stamped inspection certificates**—without OCR-based text chunking.
+
+- **GitHub Repository**: [https://github.com/adityapande403/VisDoc-AI](https://github.com/adityapande403/VisDoc-AI)
+- **Base Architecture**: `vidore/colqwen2.5-v0.2` (`Qwen/Qwen2.5-VL-3B-Instruct` backbone)
+- **Output Representation**: Multi-vector token embeddings with dimension $D = 128$ and unit L2-normalization.
+- **Scoring Function**: MaxSim Late Interaction:
+  $$S(Q, D) = \sum_{i=1}^{N_q} M_{q, i} \max_{j \in D} (q_i \cdot d_j)$$
+
+---
+
+## Benchmark Results (Empirically Measured)
+
+### 1. Retrieval Quality & Compression Retention (Held-Out Test Set)
+
+| Representation | NDCG@5 | MRR@10 | Recall@5 | Relative NDCG@5 Retention | Embedding Payload | Compression vs FP16 |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **FP16 Baseline** | 0.5132 | 0.4361 | 0.8333 | **100.0%** (Baseline) | 0.09 MB | 1.0x |
+| **INT8 Scalar** | 0.5132 | 0.4361 | 0.8333 | **100.0%** | 0.05 MB | **2.0x** |
+| **1-Bit Binary** | 0.3747 | 0.2556 | 0.8333 | **73.0%** | 0.01 MB | **16.0x** (32x vs FP32) |
+
+### 2. 10,000-Page Scale Benchmark (5,000,000 Multi-Vectors)
+
+| Index Format | Total Indexed Pages | Embedding Memory (MB) | Compression vs FP16 | Mean Latency (ms) | P95 Latency (ms) | Throughput (QPS) |
+| :--- | :---: | :---: | :---: | :---: | :---: | :---: |
+| **FP16** | 10,000 | 1,220.7 MB | 1.0x | 2,028.90 ms | 2,135.90 ms | 0.5 |
+| **INT8** | 10,000 | 610.4 MB | **2.0x** | 3,013.15 ms | 3,083.47 ms | 0.3 |
+| **BINARY** | 10,000 | **76.3 MB** | **16.0x** | 2,770.02 ms | 2,890.07 ms | 0.4 |
+
+*Two-stage candidate filtering reduces 1,000-page latency from 253.26 ms to **10.69 ms** (23.7x acceleration).*
+
+---
+
+## Quickstart
+
+```python
+from visdoc import VisualDocumentRetriever
+
+# Initialize with INT8 compressed index (100% NDCG@5 retention, 2x memory reduction)
+retriever = VisualDocumentRetriever(index_format="int8")
+
+# Ingest and index PDF document pages
+retriever.index_documents(paths=["./datasheet.pdf"])
+
+# Run late-interaction visual search
+results = retriever.search(
+    query="Which pin provides the 5.0 V analog sensor supply on the TX-4000?",
+    top_k=5,
+)
+
+for result in results:
+    print(f"Document: {result.document_id} | Page: {result.page_number} | Score: {result.score:.4f}")
+```
+
+---
+
+## Citation & Reference
+
+```bibtex
+@software{visdoc_ai_2026,
+  author = {Aditya Pande},
+  title = {VisDoc AI: Multimodal Visual Document Retrieval with ColPali, ColQwen and Late Interaction},
+  url = {https://github.com/adityapande403/VisDoc-AI},
+  year = {2026}
+}
+```

config.json

@@ -0,0 +1,17 @@
+{
+  "architectures": [
+    "ColQwen2_5"
+  ],
+  "model_type": "colqwen2_5",
+  "base_model": "vidore/colqwen2.5-v0.2",
+  "embedding_dim": 128,
+  "projection_dim": 128,
+  "torch_dtype": "bfloat16",
+  "similarity_metric": "maxsim",
+  "quantization_schemes": [
+    "fp16",
+    "int8",
+    "binary"
+  ],
+  "version": "0.1.0"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4df5e9412d8d8f9207cb79a2e82904c183ec8b5d1013e36f66a097424de509a4
+size 1049264

retrieval_benchmark.json

@@ -0,0 +1,101 @@
+{
+  "fp16": {
+    "format": "fp16",
+    "metrics": {
+      "ndcg@5": 0.5132314787418083,
+      "mrr@10": 0.4361111111111111,
+      "recall@1": 0.16666666666666666,
+      "recall@5": 0.8333333333333334,
+      "recall@10": 1.0,
+      "recall@20": 1.0,
+      "evaluated_queries": 6
+    },
+    "retention": 1.0,
+    "latency_stats": {
+      "mean_ms": 1.4175499964039773,
+      "median_ms": 1.396150008076802,
+      "p90_ms": 1.5272499877028167,
+      "p95_ms": 1.5332250040955842,
+      "p99_ms": 1.538005017209798
+    },
+    "memory_stats": {
+      "format": "fp16",
+      "total_pages": 6,
+      "total_vectors": 384,
+      "quantized_embedding_bytes": 98304,
+      "quantized_embedding_mb": 0.09375,
+      "summary_bytes": 3072,
+      "summary_mb": 0.0029296875,
+      "fp16_baseline_mb": 0.09375,
+      "compression_ratio_vs_fp16": 1.0,
+      "compression_ratio_vs_fp32": 2.0,
+      "device": "cuda"
+    }
+  },
+  "int8": {
+    "format": "int8",
+    "metrics": {
+      "ndcg@5": 0.5132314787418083,
+      "mrr@10": 0.4361111111111111,
+      "recall@1": 0.16666666666666666,
+      "recall@5": 0.8333333333333334,
+      "recall@10": 1.0,
+      "recall@20": 1.0,
+      "evaluated_queries": 6
+    },
+    "retention": 1.0,
+    "latency_stats": {
+      "mean_ms": 1.978349998050059,
+      "median_ms": 1.978349988348782,
+      "p90_ms": 2.0354499865788966,
+      "p95_ms": 2.036524980212562,
+      "p99_ms": 2.037384975119494
+    },
+    "memory_stats": {
+      "format": "int8",
+      "total_pages": 6,
+      "total_vectors": 384,
+      "quantized_embedding_bytes": 49152,
+      "quantized_embedding_mb": 0.046875,
+      "summary_bytes": 3072,
+      "summary_mb": 0.0029296875,
+      "fp16_baseline_mb": 0.09375,
+      "compression_ratio_vs_fp16": 2.0,
+      "compression_ratio_vs_fp32": 4.0,
+      "device": "cuda"
+    }
+  },
+  "binary": {
+    "format": "binary",
+    "metrics": {
+      "ndcg@5": 0.3747009872302213,
+      "mrr@10": 0.25555555555555554,
+      "recall@1": 0.0,
+      "recall@5": 0.8333333333333334,
+      "recall@10": 1.0,
+      "recall@20": 1.0,
+      "evaluated_queries": 6
+    },
+    "retention": 0.7300818495171112,
+    "latency_stats": {
+      "mean_ms": 1.8759666612216581,
+      "median_ms": 1.7842999659478664,
+      "p90_ms": 2.1018999977968633,
+      "p95_ms": 2.1988000080455095,
+      "p99_ms": 2.2763200162444264
+    },
+    "memory_stats": {
+      "format": "binary",
+      "total_pages": 6,
+      "total_vectors": 384,
+      "quantized_embedding_bytes": 6144,
+      "quantized_embedding_mb": 0.005859375,
+      "summary_bytes": 3072,
+      "summary_mb": 0.0029296875,
+      "fp16_baseline_mb": 0.09375,
+      "compression_ratio_vs_fp16": 16.0,
+      "compression_ratio_vs_fp32": 32.0,
+      "device": "cuda"
+    }
+  }
+}

scale_10k_benchmark.json

@@ -0,0 +1,38 @@
+{
+  "fp16": {
+    "format": "fp16",
+    "total_pages": 10000,
+    "total_vectors": 5000000,
+    "index_build_seconds": 3.957624673843384,
+    "embedding_memory_mb": 1220.703125,
+    "compression_ratio_vs_fp16": 1.0,
+    "mean_latency_ms": 2028.9032949949615,
+    "median_latency_ms": 2006.0680000169668,
+    "p95_latency_ms": 2135.904924967326,
+    "queries_per_second": 0.49287711369332826
+  },
+  "int8": {
+    "format": "int8",
+    "total_pages": 10000,
+    "total_vectors": 5000000,
+    "index_build_seconds": 6.310345411300659,
+    "embedding_memory_mb": 610.3515625,
+    "compression_ratio_vs_fp16": 2.0,
+    "mean_latency_ms": 3013.152015002561,
+    "median_latency_ms": 3011.445799987996,
+    "p95_latency_ms": 3083.474895049585,
+    "queries_per_second": 0.3318783768694624
+  },
+  "binary": {
+    "format": "binary",
+    "total_pages": 10000,
+    "total_vectors": 5000000,
+    "index_build_seconds": 5.153455018997192,
+    "embedding_memory_mb": 76.2939453125,
+    "compression_ratio_vs_fp16": 16.0,
+    "mean_latency_ms": 2770.0238300079945,
+    "median_latency_ms": 2745.0561500154436,
+    "p95_latency_ms": 2890.068430028623,
+    "queries_per_second": 0.36100772461481456
+  }
+}