File size: 1,037 Bytes
ec76e55 7e810ae ec76e55 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 | ---
license: cc
language:
- en
base_model:
- heyleandro/DINOcular
---
## Overview
Model weights and inference code for DINOcular - a self-supervised RGB-D encoder that learns joint visuospatial representations from RGB-D observations.
[project page](https://heyleadro.github.io/dinocular-project/) , [arxiv](https://arxiv.org/abs/2608.27226)
## Input
- `rgb`: image tensor of shape `(B, 3, H, W)`, normalized with ImageNet stats
- `depth`: depth map tensor of shape `(B, 1, H, W)` (same spatial size as the image), raw metric depth (i.e. in meters)
## Requirements
```
torch
timm
```
## Usage
```python
import torch
from DINOcular import DINOcular_S
chkpt = torch.load("dinocular_s.pth", map_location="cpu", weights_only=False)
model = DINOcular_S()
model.load_state_dict(chkpt, strict=False)
model.eval()
rgb = torch.randn(1, 3, 224, 224) # IMAGENET normalized RGB image
depth = torch.randn(1, 1, 224, 224) # raw metric depth (in meters)
with torch.no_grad():
features = model(rgb, depth) # output features
``` |