Image Classification
Transformers
Safetensors
plankton
ecology
marine-science
long-tail
vision-transformer
environmental-monitoring
Instructions to use Noushineft/ViT-LASNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Noushineft/ViT-LASNet with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="Noushineft/ViT-LASNet") pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Noushineft/ViT-LASNet", device_map="auto") - Notebooks
- Google Colab
- Kaggle
File size: 4,293 Bytes
7a189b2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | ---
library_name: transformers
pipeline_tag: image-classification
tags:
- plankton
- ecology
- marine-science
- long-tail
- vision-transformer
- environmental-monitoring
---
# ViT-LASNet: plankton image classifiers
These are the trained weights for **ViT-LASNet**, from *Improving in situ real-time classification of long-tail marine plankton images for ecosystem studies*. The models classify images from the Plankton Imager (Pi-10). Plankton datasets are long-tailed: a few classes, such as detritus and copepods, dominate, and many classes are rare. ViT-LASNet fine-tunes pretrained Vision Transformers with label-aware smoothing to handle this.
- **Paper:** [Springer](https://link.springer.com/chapter/10.1007/978-3-031-92387-6_8)
- **Code:** [github.com/alan-turing-institute/ViT-LASNet](https://github.com/alan-turing-institute/ViT-LASNet)
- **Project page:** [alan-turing-institute.github.io/ViT-LASNet](https://alan-turing-institute.github.io/ViT-LASNet/)
## Models
Each model is in its own sub-folder.
| Sub-folder | Architecture | Classes | Training loss |
| --- | --- | --- | --- |
| `vit-mislas` | ViT-Base/16 (`google/vit-base-patch16-224`) | 18 | Dynamic label-aware smoothing (`MiSLAS_vit`), lr 5e-5, 30 epochs |
| `beit-balce` | BEiT-Base/16 (`microsoft/beit-base-patch16-224-pt22k-ft22k`) | 18 | Logit-adjusted cross-entropy (`Bal_CE`), lr 5e-5, 20 epochs |
| `vit-balce-3class` | ViT-Base/16 | 3 | Logit-adjusted cross-entropy (`Bal_CE`), lr 5e-5, 10 epochs |
| `resnet18-18class-rgb` | ResNet18 (torchvision) | 18 | Baseline |
| `resnet18-3class-rgb` | ResNet18 (torchvision) | 3 | Baseline |
**18 classes:** Detritus, Phyto_diatom, Phyto_diatom_chaetocerotanae_Chaetoceros, Phyto_diatom_rhisoleniales_Guinardia flaccida, Phyto_diatom_rhisoleniales_Rhizosolenia, Phyto_dinoflagellate_gonyaulacales_Tripos, Phyto_dinoflagellate_gonyaulacales_Tripos macroceros, Phyto_dinoflagellate_gonyaulacales_Tripos muelleri, Zoo_cnidaria, Zoo_crustacea_copepod, Zoo_crustacea_copepod_calanoida, Zoo_crustacea_copepod_calanoida_Acartia, Zoo_crustacea_copepod_calanoida_Centropages, Zoo_crustacea_copepod_cyclopoida, Zoo_crustacea_copepod_cyclopoida_Oithona, Zoo_crustacea_copepod_nauplii, Zoo_other, Zoo_tintinnidae.
**3 classes:** copepod, detritus, noncopepod.
## Usage
### ViT / BEiT (transformers)
```python
from PIL import Image
from transformers import AutoImageProcessor, ViTForImageClassification
repo = "Noushineft/ViT-LASNet"
processor = AutoImageProcessor.from_pretrained(repo, subfolder="vit-mislas")
model = ViTForImageClassification.from_pretrained(repo, subfolder="vit-mislas").eval()
image = Image.open("plankton.tif").convert("RGB")
logits = model(**processor(images=image, return_tensors="pt")).logits
print(model.config.id2label[logits.argmax(-1).item()])
```
For BEiT, use `BeitForImageClassification` with `subfolder="beit-balce"`.
### ResNet18 (torchvision + safetensors)
```python
import torch, torchvision
from huggingface_hub import hf_hub_download
from safetensors.torch import load_file
path = hf_hub_download("Noushineft/ViT-LASNet", "resnet18-18class-rgb/model.safetensors")
model = torchvision.models.resnet18()
model.fc = torch.nn.Linear(model.fc.in_features, 18)
model.load_state_dict(load_file(path))
model.eval()
```
The ResNet models take RGB images resized to 256×256 and scaled to [0, 1], without ImageNet normalisation. Each ResNet folder's `config.json` lists the class order.
### Command line
The `test/test.py` script in the code repository can load these models directly:
```bash
python test/test.py -m 2 -n 18 -w Noushineft/ViT-LASNet/vit-mislas -f path/to/images -o predictions.csv
```
## Training data
The models were trained on Pi-10 plankton images collected in situ, labelled into the classes above. The dataset is not distributed with these weights.
## Limitations
- The models were trained on images from one instrument (Pi-10) and its imaging conditions. Expect lower accuracy on other imagers or regions without fine-tuning.
- Rare classes have few training examples, so their predictions are less reliable than those for common classes.
- These are research models and have not been validated for operational monitoring.
## Citation
If you use these models, please cite the paper linked above.
|