|
Download README.md from dronefreak/dformer-tiny-sunrgbd: direct link, hf CLI and curl.
- Browser
- Download file 3.82 kB
-
https://huggingface.co/dronefreak/dformer-tiny-sunrgbd/resolve/main/README.md
- Command line
-
hf download hf://dronefreak/dformer-tiny-sunrgbd/README.md
-
curl -L -o README.md https://huggingface.co/dronefreak/dformer-tiny-sunrgbd/resolve/main/README.md
3.82 kB
| license: mit | |
| library_name: pytorch | |
| pipeline_tag: image-segmentation | |
| tags: | |
| - rgb-d | |
| - semantic-segmentation | |
| - indoor-scene-understanding | |
| - dformer | |
| - sunrgbd | |
| - community-mirror | |
| datasets: | |
| - dronefreak/SUN-RGBD | |
| # DFormer-T trained on SUN RGB-D | |
| <p align="center"><video controls autoplay loop muted playsinline width="900" poster="https://huggingface.co/dronefreak/dformer-tiny-sunrgbd/resolve/main/assets/demo_banner_poster.jpg" src="https://huggingface.co/dronefreak/dformer-tiny-sunrgbd/resolve/main/assets/demo_banner.mp4"><img src="https://huggingface.co/dronefreak/dformer-tiny-sunrgbd/resolve/main/assets/demo_banner_poster.jpg" alt="DFormer-T predictions on SUN RGB-D test scenes" width="900"></video></p> | |
| *Demo: predictions of this checkpoint on SUN RGB-D test scenes from our dataset mirror (RGB, depth input (8-bit), prediction overlay; per-frame pixel accuracy vs. ground truth), rendered with the official code.* | |
| > **Unofficial mirror** of the official DFormer weights from [VCIP-RGBD/DFormer](https://github.com/VCIP-RGBD/DFormer). The weights are **unchanged**; this repository adds a weights-only `model.safetensors` copy and a `config.json`. | |
| ## Model | |
| - **Architecture:** DFormer: DFormer-T | |
| - **Task:** RGB-D semantic segmentation | |
| - **Trained on:** [SUN RGB-D](https://huggingface.co/datasets/dronefreak/SUN-RGBD) (mirror on the Hub) | |
| - **Parameters:** 6.0M | |
| - **Paper:** [DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation](https://arxiv.org/abs/2309.09668) (ICLR 2024) | |
| - **Code:** https://github.com/VCIP-RGBD/DFormer | |
| - **License:** MIT (from the official repository) | |
| RGB-D pretrained encoder (ImageNet-1K with depth) plus a lightweight decoder head. The encoder used for initialisation is included under `pretrained/` (its `.pth.tar` stores training args, so load it with `weights_only=False`). | |
| ## Results (reported by the authors) | |
| | Metric | Value | | |
| |---|---| | |
| | mIoU (SUN RGB-D, single-scale) | 48.8 | | |
| See the [official repository](https://github.com/VCIP-RGBD/DFormer) for full tables and evaluation details. | |
| ## Files | |
| | File | Content | | |
| |---|---| | |
| | `SUNRGBD_DFormer_Tiny.pth` | original checkpoint, exactly as released | | |
| | `model.safetensors` | the model weights only (bit-exact copy of the checkpoint's `model` tensors; no optimizer state) | | |
| | `config.json` | architecture, dataset, results and provenance | | |
| | `pretrained/DFormer_Tiny.pth.tar` | DFormer-T encoder, RGB-D pretrained on ImageNet-1K | | |
| ## Usage | |
| ```python | |
| import json | |
| from huggingface_hub import hf_hub_download | |
| repo = "dronefreak/dformer-tiny-sunrgbd" | |
| config = json.load(open(hf_hub_download(repo, "config.json"))) | |
| ckpt = hf_hub_download(repo, "SUNRGBD_DFormer_Tiny.pth") # original format: use it with the official code | |
| print(config["architecture"], config["dataset"], ckpt) | |
| ``` | |
| Build the network with the official code (https://github.com/VCIP-RGBD/DFormer) using the matching config for this variant, then load | |
| `SUNRGBD_DFormer_Tiny.pth` exactly as you would the original download, or load `model.safetensors` with | |
| `safetensors.torch.load_file` into the same model. | |
| ## Citation | |
| If you use these weights, please cite the original work: | |
| ```bibtex | |
| @inproceedings{yin2024dformer, | |
| title = {DFormer: Rethinking RGBD Representation Learning for Semantic Segmentation}, | |
| author = {Yin, Bowen and Zhang, Xuying and Li, Zhong-Yu and Liu, Li and Cheng, Ming-Ming and Hou, Qibin}, | |
| booktitle = {ICLR}, | |
| year = {2024} | |
| } | |
| ``` | |
| ## Disclaimer | |
| This is not an official release. All credit goes to the original authors (Bowen Yin, Xuying Zhang, Zhong-Yu Li, Li Liu, Ming-Ming Cheng, Qibin Hou; Nankai University (VCIP)). The weights are | |
| redistributed under the MIT license of the official repository. If you are an author and want this mirror changed or | |
| removed, please open a discussion. | |