Hugging Face's logo

Join the conversation

Join the community of Machine Learners and AI enthusiasts.

Sign Up
Back to feed
Ryenhails 
posted an update 1 day ago
Post
648
Quick follow-up on NanoVDR: we distilled an 8B visual document retriever down to a 457M document tower and a 70M query tower by simply reproducing the frozen Qwen3-VL-Embedding-8B page embeddings under cosine distance. No relevance labels, no negative mining, no contrastive loss.

Two document tower variants, each paired with the same 70M text-only query tower and averaged over all 22 ViDoRe datasets: HiRes keeps 86.9% of the teacher's NDCG@5 and indexes at 37 pages/s, Fast keeps 84.4% at 99 pages/s, against the teacher's 5.4. Both store 16.4 GB per million pages instead of the 256 GB their multi-vector peers need.

We released the training nanovdr/distilling-the-document-tower and models nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096, nanovdr/NanoVDR-D-HiRes-Qwen3VL8B-4096, nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML on our NanoVDR space.

Feel free to explore!
Yusufchy

Reproducing page embeddings to get a 527M document tower from an 8B teacher is an interesting efficiency tradeoff. The 84–87% NDCG@5 retention across 22 ViDoRe datasets makes the memory claim easier to evaluate, but deployment teams will also need query/document tower compatibility and domain-shift slices. Are those comparisons planned for the release?

·
Ryenhails

Thanks, and one clarification first: the 84-87% was the range across our two document tower variants, not across datasets. HiRes averages 86.9% of the teacher at 37 pages/s, Fast 84.4% at 99. I have edited the post to say that directly, since it was easy to read the other way.

On domain slices, you are asking for the right thing, and the spread is much wider than the headline. I have just added the full per-dataset breakdown to the write-up, section named Where the Student Holds Up, and Where It Does Not.

On tower compatibility, the rule is that the teacher and the output width must both match, which is why both now appear in the model names. The three 4096-width towers pair with each other e.g.(https://huggingface.co/nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096 for Doc tower with https://huggingface.co/nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML for Query tower); the 2048-d query towers from our earlier release do not pair with new document towers. Since both towers land in the same frozen teacher space, all four student and teacher combinations is usable, like following (number is avg. NDCG@5 for vidore v1, v2, v3):

teacher docs student docs
teacher queries 71.05 65.02
student queries 66.36 61.74
In this post