| --- |
| license: apache-2.0 |
| library_name: pytorch |
| --- |
| |
| <a id="top"></a> |
| <div align="center"> |
| <h1>DAAN: Mitigating Information Asymmetries for Point-Supervised Video Moment Retrieval</h1> |
|
|
| <p> |
| <b>Hao Liu</b><sup>1</sup> |
| <b>Yupeng Hu</b><sup>1✉</sup> |
| <b>Kun Wang</b><sup>1</sup> |
| <b>Junchao Wang</b><sup>1</sup> |
| <b>Zhiran Li</b><sup>1</sup> |
| <b>Liqiang Nie</b><sup>2</sup> |
| </p> |
| |
| <p> |
| <sup>1</sup>School of Software, Shandong University, Jinan, China<br> |
| <sup>2</sup>School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, China |
| </p> |
| </div> |
| |
| This repository hosts the official pretrained checkpoints for **DAAN**, a point-supervised Video Moment Retrieval (VMR) framework that mitigates Semantic-Positional Asymmetry and Instance-Duration Asymmetry through **Center-Guided Calibration (CGC)** and a **Duration-Aware Proposal Discriminator (DPD)**. |
|
|
| 🔗 **Paper:** [IEEE TPAMI 2026](https://doi.org/10.1109/TPAMI.2026.3742682) |
| 🔗 **GitHub Repository:** [iLearn-Lab/TPAMI26-DAAN](https://github.com/iLearn-Lab/TPAMI26-DAAN) |
|
|
| --- |
|
|
| ## Model Information |
|
|
| ### 1. Model Name |
|
|
| **DAAN** (Dual-Asymmetry Aligned Network). |
|
|
| ### 2. Task Type & Applicable Tasks |
|
|
| - **Task Type:** Point-Supervised Video Moment Retrieval (VMR) / Temporal Localization / Vision-Language Learning. |
| - **Applicable Tasks:** Locating the temporal interval in an untrimmed video that matches a natural language query, using only a single annotated frame per training instance instead of full temporal boundaries. |
|
|
| ### 3. Project Introduction |
|
|
| Point-supervised VMR substantially reduces annotation effort but introduces two inherent information asymmetries: |
|
|
| - **Semantic-Positional Asymmetry:** An annotated frame confirms the event semantics but does not reveal its relative position within the target moment. |
| - **Instance-Duration Asymmetry:** An annotated frame confirms the event's occurrence but provides no explicit information about its temporal duration. |
|
|
| DAAN addresses these challenges with two complementary components. **CGC** dynamically estimates a pseudo temporal center and recalibrates attention through local and global cross-modal cues, while **DPD** learns query-conditioned compatibility between proposal content and boundary representations to distinguish plausible temporal spans without boundary supervision. |
|
|
| ### 4. Training Data Source |
|
|
| DAAN is trained and evaluated on standard point-supervised VMR benchmarks: |
|
|
| - **Charades-STA** (I3D, C3D, and VGG visual features) |
| - **TACoS** (C3D visual features) |
| - **ActivityNet Captions** (C3D visual features) |
|
|
| --- |
|
|
| ## Usage & Basic Inference |
|
|
| This model repository provides pretrained weights and their corresponding configuration files. For the evaluation code and dataset preparation instructions, please refer to the [official GitHub repository](https://github.com/iLearn-Lab/TPAMI26-DAAN). |
|
|
| ### Step 1: Prepare the Environment |
|
|
| Clone the GitHub repository and install dependencies: |
|
|
| ```bash |
| git clone https://github.com/iLearn-Lab/TPAMI26-DAAN.git |
| cd TPAMI26-DAAN |
| |
| python3.10 -m venv .venv |
| source .venv/bin/activate # Linux / Mac |
| |
| pip install torch==2.3.0 torchtext==0.18.0 "numpy<2" scipy pyyaml tqdm matplotlib huggingface_hub |
| ``` |
|
|
| Choose the PyTorch build appropriate for your CUDA environment. |
|
|
| ### Step 2: Download Model Weights & Data |
|
|
| Download the pretrained checkpoints from this Hugging Face repository: |
|
|
| ```bash |
| hf download iLearn-Lab/TPAMI26-DAAN --local-dir checkpoints/TPAMI26-DAAN |
| ``` |
|
|
| The downloaded files are organized as follows: |
|
|
| ```text |
| checkpoints/TPAMI26-DAAN/ |
| ├── cha-i3d/ |
| │ ├── config.yaml |
| │ └── model_best.pt |
| ├── cha-c3d/ |
| │ ├── config.yaml |
| │ └── model_best.pt |
| ├── cha-vgg/ |
| │ ├── config.yaml |
| │ └── model_best.pt |
| └── tacos/ |
| ├── config.yaml |
| └── model_best.pt |
| ``` |
|
|
|
|
| Prepare video features and point annotations following [ViGA](https://github.com/r-cui/ViGA)'s dataset preparation protocol. **The benchmark datasets and extracted video features are not included in this model repository.** |
|
|
| Before evaluation, update the local paths in: |
|
|
| - `checkpoints/TPAMI26-DAAN/<checkpoint>/config.yaml`: `feature_dir` (and ensure `feature_dim` matches the extracted features). |
| - `src/utils/utils.py`: annotation directories for the corresponding datasets. |
|
|
|
|
| ### Step 3: Run Inference |
|
|
| Evaluate a downloaded checkpoint from the root of the GitHub repository: |
|
|
| ```bash |
| python -m src.experiment.eval --exp checkpoints/TPAMI26-DAAN/cha-c3d |
| ``` |
|
|
|
|
| --- |
|
|
| ## Limitations & Notes |
|
|
| **Disclaimer:** These checkpoints are provided for academic research purposes. |
|
|
| - The model requires access to the original benchmark datasets and extracted video features for evaluation. |
| - Some configuration files currently contain local path settings and should be updated before use. |
|
|
| --- |
|
|
| ## Citation |
|
|
| If you find our work useful in your research, please consider citing our paper: |
|
|
| ```bibtex |
| @article{liu2026daan, |
| title={DAAN: Mitigating Information Asymmetries for Point-Supervised Video Moment Retrieval}, |
| author={Liu, Hao and Hu, Yupeng and Wang, Kun and Wang, Junchao and Li, Zhiran and Nie, Liqiang}, |
| journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, |
| year={2026}, |
| doi={10.1109/TPAMI.2026.3742682} |
| } |
| ``` |
|
|
| --- |
|
|
| ## Contact |
|
|
| If you have any questions, feel free to contact me at **liuh90210@gmail.com**. |
|
|
