Polish model card

README.md

@@ -9,57 +9,85 @@ tags:
 - missing-modalities
 ---
 
-# HyVol++: Hypergraph Fusion for Multimodal Retrieval
+<div align="center">
 
-Pretrained weights for the paper
-**Linking Modalities, Learning Across Samples: Hypergraph Fusion for Multimodal Retrieval**
-by Anindya Nag, Ambuj Mehrish and Sebastiano Vascon.
+# HyVol++
 
-This work extends HyVol ([CVML-CFU/HyVol](https://huggingface.co/CVML-CFU/HyVol), WACV 2027)
-into a general training-time hypergraph fusion framework. Observed modalities (video, audio,
-subtitles) are vertices, and caption-guided mutual top-k neighbourhoods connect related
-training clips, so a clip can draw on evidence that related clips have observed.
-Two new configurations are released here:
+### Hypergraph Fusion for Multimodal Retrieval
 
-- **wHyVol** (weighted HyVol): a learned reliability weight for each view and a symmetric
-  caption affinity for each pair of related clips, with additive hypergraph propagation.
-- **wRHG** (weighted relational hypergraph): keeps both weights and replaces untyped links
-  with typed relations (document, same-modality, cross-modality and joint-candidate
-  hyperedges), with relation-conditioned message passing.
+**Anindya Nag · Ambuj Mehrish · Sebastiano Vascon**
 
-Missing modalities are omitted rather than imputed. The hypergraph is discarded after
-training, so the backbone, scoring function and inference cost are unchanged.
+Ca' Foscari University of Venice
 
-## Checkpoints
+[![HyVol](https://img.shields.io/badge/WACV%202027-HyVol-blue)](https://arxiv.org/abs/2609.15320)
+[![Code](https://img.shields.io/badge/GitHub-HyVol-black?logo=github)](https://github.com/CVML-CFU/HyVol)
+[![HyVol weights](https://img.shields.io/badge/%F0%9F%A4%97%20Weights-HyVol-yellow)](https://huggingface.co/CVML-CFU/HyVol)
+[![License](https://img.shields.io/badge/License-CC%20BY%204.0-lightgrey)](https://creativecommons.org/licenses/by/4.0/)
 
-| File | Backbone |
-|---|---|
-| `wHyVol/GRAM_wHyVol/wHyVol_GRAM.pt` | GRAM + wHyVol |
-| `wHyVol/HyperGRAM_wHyVol/wHyVol_HyperGRAM.pt` | HyperGRAM + wHyVol |
-| `wRHG/GRAM_wRHG/wRHG_GRAM.pt` | GRAM + wRHG |
-| `wRHG/HyperGRAM_wRHG/wRHG_HyperGRAM.pt` | HyperGRAM + wRHG |
+</div>
 
-All models are initialised from VAST and trained for one epoch on the 150K-clip subset
-of VAST-27M (136,694 clips recovered with both video and audio). The checkpoint is selected
-by text-to-video R@1 of the matching (ITM) score on MSR-VTT 1k in the T-VAS setting.
-Load each checkpoint with the codebase of its backbone
-([GRAM](https://github.com/ispamm/GRAM) or [HyperGRAM](https://github.com/uta-smile/HyperGram)).
+---
+
+Pretrained weights for **Linking Modalities, Learning Across Samples: Hypergraph Fusion for Multimodal Retrieval**.
+
+HyVol++ turns [HyVol](https://huggingface.co/CVML-CFU/HyVol) into a general **training-time hypergraph fusion** framework.
+Observed modalities (video, audio, subtitles) are vertices, and caption-guided mutual top-k
+neighbourhoods connect related training clips, so a clip can draw on evidence that related
+clips have observed. The hypergraph is **discarded after training**, so the backbone,
+scoring function and inference cost stay exactly the same.
+
+## ✨ Two new configurations
 
-## Results
+| | What it learns | Propagation |
+|---|---|---|
+| **wHyVol** (weighted HyVol) | a reliability weight per view + a symmetric caption affinity per pair of related clips | additive |
+| **wRHG** (weighted relational hypergraph) | both weights + typed relations: document, same-modality, cross-modality and joint-candidate hyperedges | relation-conditioned, multiplicative |
+
+Missing modalities get no vertex: they are **omitted, not imputed**.
 
-Zero-shot evaluation on eleven benchmarks: eight text-video (MSR-VTT, DiDeMo, ActivityNet,
-VATEX, Shot2Story, MSVD, FLARE, VALOR-32K) and three text-audio (AudioCaps, Clotho, SoundDescs).
+## 📦 Checkpoints
 
-| Backbone | Mean T2V R@1 (17 settings) | + wRHG |
+| Model | Backbone | File |
 |---|---|---|
-| GRAM | 63.4 | 68.2 |
-| HyperGRAM | 64.8 | 68.4 |
+| wHyVol | GRAM | [`wHyVol/GRAM_wHyVol/wHyVol_GRAM.pt`](wHyVol/GRAM_wHyVol/wHyVol_GRAM.pt) |
+| wHyVol | HyperGRAM | [`wHyVol/HyperGRAM_wHyVol/wHyVol_HyperGRAM.pt`](wHyVol/HyperGRAM_wHyVol/wHyVol_HyperGRAM.pt) |
+| wRHG | GRAM | [`wRHG/GRAM_wRHG/wRHG_GRAM.pt`](wRHG/GRAM_wRHG/wRHG_GRAM.pt) |
+| wRHG | HyperGRAM | [`wRHG/HyperGRAM_wRHG/wRHG_HyperGRAM.pt`](wRHG/HyperGRAM_wRHG/wRHG_HyperGRAM.pt) |
+
+- **Initialisation:** VAST (EVA-CLIP ViT-g/14 vision, BEATs audio, BERT-base text)
+- **Training data:** one epoch on the 150K-clip subset of VAST-27M (136,694 clips recovered with video and audio)
+- **Checkpoint selection:** best text-to-video R@1 of the matching (ITM) score on MSR-VTT 1k, T-VAS
+
+## 🚀 Usage
+
+```python
+from huggingface_hub import hf_hub_download
+
+ckpt = hf_hub_download(
+    repo_id="CVML-CFU/HyVol-PlusPlus",
+    filename="wRHG/GRAM_wRHG/wRHG_GRAM.pt",
+)
+```
+
+Load each checkpoint with the codebase of its backbone:
+[GRAM](https://github.com/ispamm/GRAM) or [HyperGRAM](https://github.com/uta-smile/HyperGram).
+No extra module is needed at test time.
+
+## 📊 Results
+
+Zero-shot on **eleven benchmarks**: eight text-video (MSR-VTT, DiDeMo, ActivityNet, VATEX,
+Shot2Story, MSVD, FLARE, VALOR-32K) and three text-audio (AudioCaps, Clotho, SoundDescs).
+
+| Backbone | Mean T2V R@1 (17 settings) | + wRHG | Gain |
+|---|:---:|:---:|:---:|
+| GRAM | 63.4 | **68.2** | +4.8 |
+| HyperGRAM | 64.8 | **68.4** | +3.6 |
 
-With GRAM, wRHG improves all 17 text-to-video and all 17 video-to-text settings.
-With HyperGRAM, it improves 16 of 17 and 17 of 17. The largest video-to-text gain is
-+13.3 R@1 on ActivityNet.
+- With GRAM, wRHG improves **all 34** text-to-video and video-to-text settings.
+- With HyperGRAM, it improves **33 of 34**.
+- The largest video-to-text gain is **+13.3 R@1** on ActivityNet.
 
-## Citation
+## 📝 Citation
 
 If you use these weights, please cite: