Image Classification
Transformers
Safetensors
plankton
ecology
marine-science
long-tail
vision-transformer
environmental-monitoring
Instructions to use Noushineft/ViT-LASNet with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Noushineft/ViT-LASNet with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="Noushineft/ViT-LASNet") pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Noushineft/ViT-LASNet", device_map="auto") - Notebooks
- Google Colab
- Kaggle
|
Download README.md from Noushineft/ViT-LASNet: direct link, hf CLI and curl.
- Browser
- Download file 4.29 kB
-
https://huggingface.co/Noushineft/ViT-LASNet/resolve/main/README.md
- Command line
-
hf download hf://Noushineft/ViT-LASNet/README.md
-
curl -L -o README.md https://huggingface.co/Noushineft/ViT-LASNet/resolve/main/README.md
4.29 kB
| library_name: transformers | |
| pipeline_tag: image-classification | |
| tags: | |
| - plankton | |
| - ecology | |
| - marine-science | |
| - long-tail | |
| - vision-transformer | |
| - environmental-monitoring | |
| # ViT-LASNet: plankton image classifiers | |
| These are the trained weights for **ViT-LASNet**, from *Improving in situ real-time classification of long-tail marine plankton images for ecosystem studies*. The models classify images from the Plankton Imager (Pi-10). Plankton datasets are long-tailed: a few classes, such as detritus and copepods, dominate, and many classes are rare. ViT-LASNet fine-tunes pretrained Vision Transformers with label-aware smoothing to handle this. | |
| - **Paper:** [Springer](https://link.springer.com/chapter/10.1007/978-3-031-92387-6_8) | |
| - **Code:** [github.com/alan-turing-institute/ViT-LASNet](https://github.com/alan-turing-institute/ViT-LASNet) | |
| - **Project page:** [alan-turing-institute.github.io/ViT-LASNet](https://alan-turing-institute.github.io/ViT-LASNet/) | |
| ## Models | |
| Each model is in its own sub-folder. | |
| | Sub-folder | Architecture | Classes | Training loss | | |
| | --- | --- | --- | --- | | |
| | `vit-mislas` | ViT-Base/16 (`google/vit-base-patch16-224`) | 18 | Dynamic label-aware smoothing (`MiSLAS_vit`), lr 5e-5, 30 epochs | | |
| | `beit-balce` | BEiT-Base/16 (`microsoft/beit-base-patch16-224-pt22k-ft22k`) | 18 | Logit-adjusted cross-entropy (`Bal_CE`), lr 5e-5, 20 epochs | | |
| | `vit-balce-3class` | ViT-Base/16 | 3 | Logit-adjusted cross-entropy (`Bal_CE`), lr 5e-5, 10 epochs | | |
| | `resnet18-18class-rgb` | ResNet18 (torchvision) | 18 | Baseline | | |
| | `resnet18-3class-rgb` | ResNet18 (torchvision) | 3 | Baseline | | |
| **18 classes:** Detritus, Phyto_diatom, Phyto_diatom_chaetocerotanae_Chaetoceros, Phyto_diatom_rhisoleniales_Guinardia flaccida, Phyto_diatom_rhisoleniales_Rhizosolenia, Phyto_dinoflagellate_gonyaulacales_Tripos, Phyto_dinoflagellate_gonyaulacales_Tripos macroceros, Phyto_dinoflagellate_gonyaulacales_Tripos muelleri, Zoo_cnidaria, Zoo_crustacea_copepod, Zoo_crustacea_copepod_calanoida, Zoo_crustacea_copepod_calanoida_Acartia, Zoo_crustacea_copepod_calanoida_Centropages, Zoo_crustacea_copepod_cyclopoida, Zoo_crustacea_copepod_cyclopoida_Oithona, Zoo_crustacea_copepod_nauplii, Zoo_other, Zoo_tintinnidae. | |
| **3 classes:** copepod, detritus, noncopepod. | |
| ## Usage | |
| ### ViT / BEiT (transformers) | |
| ```python | |
| from PIL import Image | |
| from transformers import AutoImageProcessor, ViTForImageClassification | |
| repo = "Noushineft/ViT-LASNet" | |
| processor = AutoImageProcessor.from_pretrained(repo, subfolder="vit-mislas") | |
| model = ViTForImageClassification.from_pretrained(repo, subfolder="vit-mislas").eval() | |
| image = Image.open("plankton.tif").convert("RGB") | |
| logits = model(**processor(images=image, return_tensors="pt")).logits | |
| print(model.config.id2label[logits.argmax(-1).item()]) | |
| ``` | |
| For BEiT, use `BeitForImageClassification` with `subfolder="beit-balce"`. | |
| ### ResNet18 (torchvision + safetensors) | |
| ```python | |
| import torch, torchvision | |
| from huggingface_hub import hf_hub_download | |
| from safetensors.torch import load_file | |
| path = hf_hub_download("Noushineft/ViT-LASNet", "resnet18-18class-rgb/model.safetensors") | |
| model = torchvision.models.resnet18() | |
| model.fc = torch.nn.Linear(model.fc.in_features, 18) | |
| model.load_state_dict(load_file(path)) | |
| model.eval() | |
| ``` | |
| The ResNet models take RGB images resized to 256×256 and scaled to [0, 1], without ImageNet normalisation. Each ResNet folder's `config.json` lists the class order. | |
| ### Command line | |
| The `test/test.py` script in the code repository can load these models directly: | |
| ```bash | |
| python test/test.py -m 2 -n 18 -w Noushineft/ViT-LASNet/vit-mislas -f path/to/images -o predictions.csv | |
| ``` | |
| ## Training data | |
| The models were trained on Pi-10 plankton images collected in situ, labelled into the classes above. The dataset is not distributed with these weights. | |
| ## Limitations | |
| - The models were trained on images from one instrument (Pi-10) and its imaging conditions. Expect lower accuracy on other imagers or regions without fine-tuning. | |
| - Rare classes have few training examples, so their predictions are less reliable than those for common classes. | |
| - These are research models and have not been validated for operational monitoring. | |
| ## Citation | |
| If you use these models, please cite the paper linked above. | |