---
license: apache-2.0
pipeline_tag: image-classification
---

# Dyna-DINO: Efficient ViT Distillation Via Adaptive Representation Anchoring

[Paper](https://huggingface.co/papers/2606.19483) | [GitHub](https://github.com/KevinZ0217/Dyna-DINO)
<!-- [Project Page](https://kevinz0217.github.io/LEAP_page/) -->

This repository contains the ViT-Tiny and ViT-S checkpoints (No Register) distilled from ViT-G DINOv2 on ImageNet-100 and ImageNet-1K. The knowledge distillation process follows the procedure proposed in the paper **"Dyna-DINO: Efficient ViT Distillation Via Adaptive Representation Anchoring"**.

### Introduction

Vision Foundation Models (VFMs) with ViT backbones, such as DINOv2, are computationally demanding. LEAP (Layer-skipping Efficiency via Adaptive Progression) is a training curriculum for ViT feature-based knowledge distillation. Instead of supervising the student against a fixed teacher block, LEAP advances the supervisory target through the teacher's feature maps (shallow-to-deep) based on online CKA alignment. This allows the student to build a foundational representation before tackling higher-level abstractions.

### Use cases
The ViT models output feature maps that can be used for a variety of downstream tasks, including:
- Image Classification
- Instance Retrieval
- Semantic Segmentation

### Performance

#### ImageNet-100:

![Classification](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/OUZ0oNlU012QJvy2pVu_R.png)

![Image Retrieval](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/WXIzBsS1aHhnnA8kJtIdR.png)

![Segmentation](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/vpZ3PfXVZzqNtfjVA6EFo.png)


#### ImageNet-1K:

![Image Retrieval](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/G4IywFO07VdcRb-pIJkuC.png)

![Semantic Segmentation](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/_Yrsgkfsr352tPdf2uUjh.png)

![Classification](https://cdn-uploads.huggingface.co/production/uploads/63d84d163130cadcaf8a976a/MU3zRoNFRmYaARRx5rc-r.png)

### Citation

```bibtex
@article{DynaDINO,
  title={Dyna-DINO: Efficient ViT Distillation Via Adaptive Representation Anchoring},
  author={Zhang, Jiaqi and Lee, Ashton and Wong, Anthony and Zou, John and BuGhanem, Sami and Balestriero, Randall},
  journal={arXiv preprint arXiv:2606.19483},
  year={2026}
}
```