File size: 3,237 Bytes
162458f c4edd3f 162458f c4edd3f 162458f 7953329 162458f c4edd3f 162458f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | ---
language: en
license: other
license_name: rights-reserved-pending
license_link: https://huggingface.co/khamirdesai/RetViM/blob/main/LICENSE_WEIGHTS
library_name: pytorch
pipeline_tag: image-classification
tags:
- retinal-oct
- vision-transformer
- mamba
- research
base_model: timm/vit_base_patch16_224.augreg_in21k_ft_in1k
---
# RetViM
**Sequential Hybrid Vision Transformer with MedMamba for Retinal Disease Classification**
Accepted for presentation at **IEEE GSCon 2027**.
Khamir Desai 路 Mayuri A. Mehta 路 Sree Saicharan Vadapalli
[Code](https://github.com/khamirdesai/RetViM) 路 [Checkpoint details](https://github.com/khamirdesai/RetViM/blob/main/docs/checkpoint.md) 路 [Citation](https://github.com/khamirdesai/RetViM/blob/main/CITATION.cff)
## Model
| Property | Value |
| --- | --- |
| Task | Retinal OCT image classification |
| Labels, in order | CNV, DME, DRUSEN, NORMAL |
| Architecture | ViT-B/16 + 2 convolution/native-Mamba blocks |
| Parameters | 101,223,557 |
| Input | RGB, 224 脳 224; OpenCV linear resize; ImageNet normalization |
| Weights | Float32 safetensors, 404.95 MB |
The supplied checkpoint uses **native 1D Mamba**, with pooling fusion and classifier dimensions that differ from the manuscript's summarized SS2D configuration. The linked implementation preserves the trained tensors and loads them strictly. It is a custom PyTorch model, not a Transformers `AutoModel` checkpoint.
## Dataset and reported result
[Kermany retinal OCT dataset](https://www.kaggle.com/datasets/paultimothymooney/kermany2018), with CNV, DME, DRUSEN and NORMAL labels. [Original dataset record](https://data.mendeley.com/datasets/rscbjbr9sj/2).
The manuscript reports **99.90% accuracy (967/968 test images)**, the highest among the methods compared there. This is paper-reported, not an independently verified overall leaderboard ranking or a newly reproduced benchmark.
Standalone [training and evaluation instructions](https://github.com/khamirdesai/RetViM/blob/main/docs/training.md) are available with the source. New training runs use explicit, hashed patient-disjoint manifests and are not claimed to recreate the original split.
## Inference
```bash
git clone https://github.com/khamirdesai/RetViM.git
cd RetViM
pip install -e .
retvim predict --image /path/to/oct.png
```
The first run downloads the pinned checkpoint. For offline inference, add `--weights /path/to/model.safetensors`.
The package verifies the pinned checkpoint's SHA-256:
```text
45c2042229b00b8ac882722c092b5e99708b6ff1d8ab900ae7082972b01480fd
```
## Scope and license
Strict tensor loading, scan equations, and example inference were checked. The full paper benchmark was not independently rerun. Intended for research; scores are uncalibrated and no prospective clinical validation is claimed.
Original code: MIT. **Weights: license pending; rights reserved** under `LICENSE_WEIGHTS`. The manuscript, publisher PDF, and paper figures are not included. Final publication metadata will be added when available.
Mentor: **Prof. Mayuri A. Mehta**. Upstream research: [MedMamba](https://github.com/YubiaoYue/MedMamba), [VMamba](https://github.com/MzeroMiko/VMamba), and [Mamba](https://github.com/state-spaces/mamba).
|