KD-OAD Encoders
Pretrained encoder and TeSTra head weights for a forthcoming paper on knowledge distillation for online action detection (OAD).
- π» Code: https://github.com/sag111/kd-oad-encoders
- π Paper: forthcoming
Model description
This release provides compact RGB-only student encoders for OAD, distilled from the original TeSTra feature extractors (ResNet-50 for appearance and BN-Inception for motion / optical flow). A two-stage procedure is used: (1) student encoders (X3D XS) are trained to match teacher features; (2) TeSTra is trained on the student features with additional logit distillation from the teacher model. The released X3D XS checkpoints take only RGB frames. The online detection head is based on TeSTra.
Checkpoints
| File | Dataset | Role | Distillation | Params (encoder) | Metric (best seed) | Metric (10 runs) |
|---|---|---|---|---|---|---|
thumos_x3d.pt |
THUMOS14 | Student encoder | Feature KD: ResNet-50 + BN-Inception β X3D XS | 3.0M | β | β |
thumos_testra.pth |
THUMOS14 | TeSTra head | Logit KD on X3D features (teacher: TeSTra on ResNet-50 + BN-Inception) | β | 66.3% mAP | 65.9 Β± 0.2% mAP |
tvseries_x3d.pt |
TVSeries | Student encoder | Feature KD: ResNet-50 + BN-Inception β X3D XS | 3.0M | β | β |
tvseries_testra.pth |
TVSeries | TeSTra head | Logit KD on X3D features (teacher: TeSTra on ResNet-50 + BN-Inception) | β | 86.2% mcAP | 86.0 Β± 0.1% mcAP |
Usage
git clone --recurse-submodules https://github.com/sag111/kd-oad-encoders.git
cd kd-oad-encoders
python3 -m venv .venv
source .venv/bin/activate
pip install -e kdoad
pip install -e testra
hf download elnmax/kd-oad-encoders --local-dir checkpoints --include "*.pth" "*.pt"
Evaluate on THUMOS14
# after video preprocessing and feature export (see GitHub README)
DATA_ROOT=../datasets/THUMOS14 \
GPU=0 \
./testra/scripts/test_checkpoint.sh \
thumos \
../checkpoints/thumos_testra.pth
Evaluate on TVSeries
DATA_ROOT=../datasets/TVSeries \
GPU=0 \
./testra/scripts/test_checkpoint.sh \
tvseries \
../checkpoints/tvseries_testra.pth
Full preprocessing and X3D feature-export steps: https://github.com/sag111/kd-oad-encoders
Data
Videos are not redistributed. Interval annotations (labels.csv) are
provided in the GitHub repository under datasets/THUMOS14/ and
datasets/TVSeries/ (derived from the official dataset annotations).
TeSTra per-frame targets (target_perframe/*.npy) are not included
in this model repo:
- THUMOS14: download the official TeSTra targets (see the GitHub README).
- TVSeries: generate them from
labels.csvvia./kdoad/scripts/generate_target_perframe.sh.
Original dataset pages:
- THUMOS14: https://www.crcv.ucf.edu/THUMOS14/
- TVSeries: https://homes.esat.kuleuven.be/psi-archive/rdegeest/TVSeries.html
License
Weights and original code are released under the MIT License. The TeSTra component is licensed under Apache 2.0.
Citation
@article{kd_oad_encoders,
title = {<Paper Title>},
author = {<Authors>},
journal = {<Venue>},
year = {<Year>}
}