| --- |
| license: cc |
| language: |
| - en |
| base_model: |
| - heyleandro/DINOcular |
| --- |
| ## Overview |
|
|
| Model weights and inference code for DINOcular - a self-supervised RGB-D encoder that learns joint visuospatial representations from RGB-D observations. |
|
|
| [project page](https://heyleadro.github.io/dinocular-project/) , [arxiv](https://arxiv.org/abs/2608.27226) |
|
|
| ## Input |
| - `rgb`: image tensor of shape `(B, 3, H, W)`, normalized with ImageNet stats |
| - `depth`: depth map tensor of shape `(B, 1, H, W)` (same spatial size as the image), raw metric depth (i.e. in meters) |
|
|
| ## Requirements |
|
|
| ``` |
| torch |
| timm |
| ``` |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| from DINOcular import DINOcular_S |
| |
| chkpt = torch.load("dinocular_s.pth", map_location="cpu", weights_only=False) |
| model = DINOcular_S() |
| model.load_state_dict(chkpt, strict=False) |
| model.eval() |
| |
| rgb = torch.randn(1, 3, 224, 224) # IMAGENET normalized RGB image |
| depth = torch.randn(1, 1, 224, 224) # raw metric depth (in meters) |
| |
| with torch.no_grad(): |
| features = model(rgb, depth) # output features |
| ``` |
