---
tags: [image-classification, plankton, PISCO, vision-transformer]
license: mit
---
# M181_multiclass

Second stage of the PISCO dual-ViT: taxon classification of crops the binary
stage called living. Fine-tuned `google/vit-base-patch16-224-in21k` on
expert-validated crops from cruise **M181** (tropical Atlantic, April-May
2022). Test accuracy 96.17% (13 classes).

**Classes (13):** Appendicularia, Asteroidea larvae, Bacillariophyceae,
Chaetognatha, Cnidaria, Copepoda, Ctenophora, Eumalacostraca, Noctiluca sp.,
Pyrocystis, Rhizaria, Thaliacea, Trichodesmium

## Known limitations
* **Echinoderm pluteus larvae are systematically labelled `Rhizaria`** - the
  model has no pluteus class. On a North Sea dataset (ATAIR-BSH) 2,354 objects
  called Rhizaria were on validation almost entirely pluteus.
* `Asteroidea larvae` and `Noctiluca sp.` are **deprecated taxa in EcoTaxa**;
  the pipeline remaps them to `Asteroidea` / `Noctiluca` at export so imports
  remain validatable.
* Trained on tropical Atlantic plankton; expect a domain gap elsewhere.

## Important: this model is coupled to its preprocessing
Trained on crops produced by the **231204 LUCYD deconvolution**
(`lucyd-edof-plankton_231204.pth`, the pipeline default) with **non-isolated**
crops (neighbouring particles present). Running it on crops from a different
deconvolution model, or on neighbour-isolated crops, is an out-of-domain shift:
measured on ATAIR-BSH, swapping the deconvolution alone collapsed the living
rate by 83-93% at unchanged confidence. Treat deconvolution + crop isolation +
classifier as one versioned unit.

Preprocessing: resize longest edge to 224, centre-pad to 224x224 with white
(255), normalise mean=std=0.5. No rotation at inference. Crops must carry no
scale bar (or have it removed with `utils.strip_scale_bar`).

## Use in the pipeline
These are the pipeline defaults (`--binary-model-dir` / `--living-model-dir`),
or from any machine:
```
process_pisco_profiles.py --binary-model-hf Veit/M181_binary --living-model-hf Veit/M181_multiclass
process_pisco_profiles.py --dualvit-model M181     # resolves both from the Hub when not local
```