NanoVDR

Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder
for Visual Document Retrieval

Code  |  Demo  |  Paper  |  Blog  |  Dataset

Paper: NanoVDR: Distilling a 2B Vision-Language Retriever into a 70M Text-Only Encoder for Visual Document Retrieval

NanoVDR distills a frozen 2B VLM teacher (Qwen3-VL-Embedding-2B) into tiny text-only query encoders (69–151M parameters) for visual document retrieval. Documents are indexed offline by the teacher; queries are encoded on CPU in 51 ms via a DistilBERT forward pass — no vision model at query time.

Queries and documents both map to the same 2048-dim single vector inherited from the teacher's embedding space, so retrieval is a plain dot product — FAISS-compatible with no MaxSim pooling. The doc index stores just 4 KB per page (float16), making NanoVDR 64× more storage-efficient than multi-vector retrievers like ColPali.

The org also hosts ColNanoVDR, which carries the same document-free distillation to multi-vector, late-interaction retrievers.

NanoVDR models

_{NDCG@5 (×100) on the ViDoRe benchmark (22 datasets). Retention = Student / Teacher. Teacher = Qwen3-VL-Embedding-2B (2.0B).}

Quick Start

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("nanovdr/NanoVDR-S-Multi")
query_emb = model.encode(["What was the revenue growth in Q3 2024?"])  # (1, 2048)

# Retrieve via cosine similarity against teacher-indexed document embeddings
# scores = query_emb @ doc_embeddings.T

Documents must be indexed offline with the teacher VLM. See the NanoVDR-S-Multi model page for a complete guide.

ColNanoVDR

ColNanoVDR extends the same idea from single-vector retrieval to multi-vector, late-interaction retrievers (ColPali-style models such as ColQwen3.5, Vultron, Tomoro and ColVec). A text-only query tower is distilled document-free — from cached teacher query token embeddings only, with no page image ever encoded and no relevance label — into the teacher's late-interaction space. An existing multi-vector page index is scored by MaxSim unchanged, and no vision-language model runs at query time.

_{NDCG@5 (×100) on ViDoRe (22 datasets), query side measured in isolation against teacher-encoded pages. Retention = Student / Teacher, averaged over v1–v3. † inherits the webAI Non-Commercial License from its teacher.}

from sentence_transformers import MultiVectorEncoder

model = MultiVectorEncoder(
    "nanovdr/ColNanoVDR-Q-Ettin150M-ColQwen35-320-ML", trust_remote_code=True
)
q = model.encode_query(["What was the revenue growth in Q3 2024?"])  # list of (n_tokens, 320)

scores = model.similarity(q, page_embeddings)   # meanMaxSim

Pages must be indexed offline with the matching teacher. A tower is only valid with the teacher it was distilled from: the teacher and the width must both match. See each model page for the indexing step.

Acknowledgements

This project has received funding from the Business Finland co-innovation programme under grant agreement No. 69/31/2025. It is supported by the AiWo: Human-centric AI-enabled Collaborative Fieldwork Operations project (2025–2027), which aims to revolutionize fieldwork operations and enhance human-AI collaboration across the manufacturing, construction, and industrial design sectors. The calculations presented in this project were performed using computer resources within the Aalto University School of Science "Science-IT" project.