| --- |
| tags: [image-classification, plankton, PISCO, vision-transformer] |
| license: mit |
| --- |
| # M181_multiclass |
| |
| Second stage of the PISCO dual-ViT: taxon classification of crops the binary |
| stage called living. Fine-tuned `google/vit-base-patch16-224-in21k` on |
| expert-validated crops from cruise **M181** (tropical Atlantic, April-May |
| 2022). Test accuracy 96.17% (13 classes). |
| |
| **Classes (13):** Appendicularia, Asteroidea larvae, Bacillariophyceae, |
| Chaetognatha, Cnidaria, Copepoda, Ctenophora, Eumalacostraca, Noctiluca sp., |
| Pyrocystis, Rhizaria, Thaliacea, Trichodesmium |
| |
| ## Known limitations |
| * **Echinoderm pluteus larvae are systematically labelled `Rhizaria`** - the |
| model has no pluteus class. On a North Sea dataset (ATAIR-BSH) 2,354 objects |
| called Rhizaria were on validation almost entirely pluteus. |
| * `Asteroidea larvae` and `Noctiluca sp.` are **deprecated taxa in EcoTaxa**; |
| the pipeline remaps them to `Asteroidea` / `Noctiluca` at export so imports |
| remain validatable. |
| * Trained on tropical Atlantic plankton; expect a domain gap elsewhere. |
| |
| ## Important: this model is coupled to its preprocessing |
| Trained on crops produced by the **231204 LUCYD deconvolution** |
| (`lucyd-edof-plankton_231204.pth`, the pipeline default) with **non-isolated** |
| crops (neighbouring particles present). Running it on crops from a different |
| deconvolution model, or on neighbour-isolated crops, is an out-of-domain shift: |
| measured on ATAIR-BSH, swapping the deconvolution alone collapsed the living |
| rate by 83-93% at unchanged confidence. Treat deconvolution + crop isolation + |
| classifier as one versioned unit. |
|
|
| Preprocessing: resize longest edge to 224, centre-pad to 224x224 with white |
| (255), normalise mean=std=0.5. No rotation at inference. Crops must carry no |
| scale bar (or have it removed with `utils.strip_scale_bar`). |
|
|
| ## Use in the pipeline |
| These are the pipeline defaults (`--binary-model-dir` / `--living-model-dir`), |
| or from any machine: |
| ``` |
| process_pisco_profiles.py --binary-model-hf Veit/M181_binary --living-model-hf Veit/M181_multiclass |
| process_pisco_profiles.py --dualvit-model M181 # resolves both from the Hub when not local |
| ``` |
|
|
