dinov2-small-droid

This is a DINOv2-small vision backbone adapted on robot-manipulation images from the DROID dataset. It starts from facebook/dinov2-small and preserves the standard Hugging Face Transformers DINOv2 architecture.

  • exported weights: teacher backbone
  • training step: 50,000
  • input size: 518
  • patch size: 14
  • hidden size: 384
  • training-only projection heads and optimizer state: not included

Usage

from PIL import Image
import torch
from transformers import AutoImageProcessor, AutoModel

model_id = "SakikoTogawa/dinov2-small-droid"
processor = AutoImageProcessor.from_pretrained(model_id)
model = AutoModel.from_pretrained(model_id).eval()

image = Image.open("example.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.inference_mode():
    outputs = model(**inputs)

cls_feature = outputs.last_hidden_state[:, 0]   # [batch, 384]
patch_features = outputs.last_hidden_state[:, 1:]

For cosine similarity or nearest-neighbor retrieval:

cls_feature = torch.nn.functional.normalize(cls_feature, dim=-1)

Intended use and limitations

The model is intended as an image-feature backbone for downstream robotics experiments such as frozen-feature evaluation, linear probing, retrieval, or further fine-tuning. The adaptation objective is self-supervised and does not guarantee improvement on every downstream task. Compare against the original facebook/dinov2-small using an episode-disjoint evaluation split.

Training data

DROID contains diverse robot-manipulation trajectories collected across multiple scenes and tasks. Consult the DROID project page for dataset details and terms.

Downloads last month
12
Safetensors
Model size
22.1M params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for SakikoTogawa/dinov2-small-droid

Finetuned
(42)
this model