Blame for README.md

---
license: apache-2.0
library_name: pytorch
---

<a id="top"></a>
<div align="center">
  <h1>DAAN: Mitigating Information Asymmetries for Point-Supervised Video Moment Retrieval</h1>

  <p>
    <b>Hao Liu</b><sup>1</sup>&nbsp;
    <b>Yupeng Hu</b><sup>1✉</sup>&nbsp;
    <b>Kun Wang</b><sup>1</sup>&nbsp;
    <b>Junchao Wang</b><sup>1</sup>&nbsp;
    <b>Zhiran Li</b><sup>1</sup>&nbsp;
    <b>Liqiang Nie</b><sup>2</sup>
  </p>

  <p>
    <sup>1</sup>School of Software, Shandong University, Jinan, China<br>
    <sup>2</sup>School of Computer Science and Technology, Harbin Institute of Technology (Shenzhen), Shenzhen, China
  </p>
</div>

This repository hosts the official pretrained checkpoints for **DAAN**, a point-supervised Video Moment Retrieval (VMR) framework that mitigates Semantic-Positional Asymmetry and Instance-Duration Asymmetry through **Center-Guided Calibration (CGC)** and a **Duration-Aware Proposal Discriminator (DPD)**.

🔗 **Paper:** [IEEE TPAMI 2026](https://doi.org/10.1109/TPAMI.2026.3742682)  
🔗 **GitHub Repository:** [iLearn-Lab/TPAMI26-DAAN](https://github.com/iLearn-Lab/TPAMI26-DAAN)

---

## Model Information

### 1. Model Name

**DAAN** (Dual-Asymmetry Aligned Network).

### 2. Task Type & Applicable Tasks

- **Task Type:** Point-Supervised Video Moment Retrieval (VMR) / Temporal Localization / Vision-Language Learning.
- **Applicable Tasks:** Locating the temporal interval in an untrimmed video that matches a natural language query, using only a single annotated frame per training instance instead of full temporal boundaries.

### 3. Project Introduction

Point-supervised VMR substantially reduces annotation effort but introduces two inherent information asymmetries:

- **Semantic-Positional Asymmetry:** An annotated frame confirms the event semantics but does not reveal its relative position within the target moment.
- **Instance-Duration Asymmetry:** An annotated frame confirms the event's occurrence but provides no explicit information about its temporal duration.

DAAN addresses these challenges with two complementary components. **CGC** dynamically estimates a pseudo temporal center and recalibrates attention through local and global cross-modal cues, while **DPD** learns query-conditioned compatibility between proposal content and boundary representations to distinguish plausible temporal spans without boundary supervision.

### 4. Training Data Source

DAAN is trained and evaluated on standard point-supervised VMR benchmarks:

- **Charades-STA** (I3D, C3D, and VGG visual features)
- **TACoS** (C3D visual features)
- **ActivityNet Captions** (C3D visual features)

---

## Usage & Basic Inference

This model repository provides pretrained weights and their corresponding configuration files. For the evaluation code and dataset preparation instructions, please refer to the [official GitHub repository](https://github.com/iLearn-Lab/TPAMI26-DAAN).

### Step 1: Prepare the Environment

Clone the GitHub repository and install dependencies:

```bash
git clone https://github.com/iLearn-Lab/TPAMI26-DAAN.git
cd TPAMI26-DAAN

python3.10 -m venv .venv
source .venv/bin/activate  # Linux / Mac

pip install torch==2.3.0 torchtext==0.18.0 "numpy<2" scipy pyyaml tqdm matplotlib huggingface_hub
```

Choose the PyTorch build appropriate for your CUDA environment.

### Step 2: Download Model Weights & Data

Download the pretrained checkpoints from this Hugging Face repository:

```bash
hf download iLearn-Lab/TPAMI26-DAAN --local-dir checkpoints/TPAMI26-DAAN
```

The downloaded files are organized as follows:

```text
checkpoints/TPAMI26-DAAN/
├── cha-i3d/
│   ├── config.yaml
│   └── model_best.pt
├── cha-c3d/
│   ├── config.yaml
│   └── model_best.pt
├── cha-vgg/
│   ├── config.yaml
│   └── model_best.pt
└── tacos/
    ├── config.yaml
    └── model_best.pt
```


Prepare video features and point annotations following [ViGA](https://github.com/r-cui/ViGA)'s dataset preparation protocol. **The benchmark datasets and extracted video features are not included in this model repository.**

Before evaluation, update the local paths in:

- `checkpoints/TPAMI26-DAAN/<checkpoint>/config.yaml`: `feature_dir` (and ensure `feature_dim` matches the extracted features).
- `src/utils/utils.py`: annotation directories for the corresponding datasets.


### Step 3: Run Inference

Evaluate a downloaded checkpoint from the root of the GitHub repository:

```bash
python -m src.experiment.eval --exp checkpoints/TPAMI26-DAAN/cha-c3d
```


---

## Limitations & Notes

**Disclaimer:** These checkpoints are provided for academic research purposes.

- The model requires access to the original benchmark datasets and extracted video features for evaluation.
- Some configuration files currently contain local path settings and should be updated before use.

---

## Citation

If you find our work useful in your research, please consider citing our paper:

```bibtex
@article{liu2026daan,
  title={DAAN: Mitigating Information Asymmetries for Point-Supervised Video Moment Retrieval},
  author={Liu, Hao and Hu, Yupeng and Wang, Kun and Wang, Junchao and Li, Zhiran and Nie, Liqiang},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2026},
  doi={10.1109/TPAMI.2026.3742682}
}
```

---

## Contact

If you have any questions, feel free to contact me at **liuh90210@gmail.com**.