KD-OAD Encoders

Pretrained encoder and TeSTra head weights for a forthcoming paper on knowledge distillation for online action detection (OAD).

Model description

This release provides compact RGB-only student encoders for OAD, distilled from the original TeSTra feature extractors (ResNet-50 for appearance and BN-Inception for motion / optical flow). A two-stage procedure is used: (1) student encoders (X3D XS) are trained to match teacher features; (2) TeSTra is trained on the student features with additional logit distillation from the teacher model. The released X3D XS checkpoints take only RGB frames. The online detection head is based on TeSTra.

Checkpoints

File Dataset Role Distillation Params (encoder) Metric (best seed) Metric (10 runs)
thumos_x3d.pt THUMOS14 Student encoder Feature KD: ResNet-50 + BN-Inception β†’ X3D XS 3.0M β€” β€”
thumos_testra.pth THUMOS14 TeSTra head Logit KD on X3D features (teacher: TeSTra on ResNet-50 + BN-Inception) β€” 66.3% mAP 65.9 Β± 0.2% mAP
tvseries_x3d.pt TVSeries Student encoder Feature KD: ResNet-50 + BN-Inception β†’ X3D XS 3.0M β€” β€”
tvseries_testra.pth TVSeries TeSTra head Logit KD on X3D features (teacher: TeSTra on ResNet-50 + BN-Inception) β€” 86.2% mcAP 86.0 Β± 0.1% mcAP

Usage

git clone --recurse-submodules https://github.com/sag111/kd-oad-encoders.git
cd kd-oad-encoders
python3 -m venv .venv
source .venv/bin/activate
pip install -e kdoad
pip install -e testra

hf download elnmax/kd-oad-encoders --local-dir checkpoints --include "*.pth" "*.pt"

Evaluate on THUMOS14

# after video preprocessing and feature export (see GitHub README)
DATA_ROOT=../datasets/THUMOS14 \
GPU=0 \
./testra/scripts/test_checkpoint.sh \
  thumos \
  ../checkpoints/thumos_testra.pth

Evaluate on TVSeries

DATA_ROOT=../datasets/TVSeries \
GPU=0 \
./testra/scripts/test_checkpoint.sh \
  tvseries \
  ../checkpoints/tvseries_testra.pth

Full preprocessing and X3D feature-export steps: https://github.com/sag111/kd-oad-encoders

Data

Videos are not redistributed. Interval annotations (labels.csv) are provided in the GitHub repository under datasets/THUMOS14/ and datasets/TVSeries/ (derived from the official dataset annotations).

TeSTra per-frame targets (target_perframe/*.npy) are not included in this model repo:

  • THUMOS14: download the official TeSTra targets (see the GitHub README).
  • TVSeries: generate them from labels.csv via ./kdoad/scripts/generate_target_perframe.sh.

Original dataset pages:

License

Weights and original code are released under the MIT License. The TeSTra component is licensed under Apache 2.0.

Citation

@article{kd_oad_encoders,
  title   = {<Paper Title>},
  author  = {<Authors>},
  journal = {<Venue>},
  year    = {<Year>}
}
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support