OceanGuard AI: BioDex species pack
Files that the OceanGuard AI Android app downloads on demand to identify marine species on the device, without a connection. The photo is encoded with BioCLIP 2 and compared with reference embeddings of 232 species, mostly Mediterranean plus 30 tropical species often seen by divers: prototypes of real photos plus the embedding of each species name. Below the calibrated confidence thresholds the app answers "not identified" or shows the closest candidates as "possible"; since pack v4 an unidentified photo can still get its group (for example "sharks and rays"). Only confident identifications are saved.
Packs
| Pack | Manifest | Used by | Encoder | Species |
|---|---|---|---|---|
| v5 (current, manifest version 7) | species_pack_v5.json |
app builds after 1.2.0-rc6 | v4 pack plus species sprites (version 6 redraws 14 sprites whose source photo cut part of the animal; version 7 redraws 2 more) | 232, all with a sprite |
| v4 | species_pack_v4.json |
app 1.2.0-rc5 and later | same encoder file as v3 | 232 (227 with reference photos, 5 by name only) |
| v3 | species_pack_v3.json |
app 1.2.0-rc3 / rc4 | BioCLIP 2 ViT-L/14, fp16 weights with fp32 compute (609 MB) | 202 (197 with reference photos, 5 by name only) |
| v2 | species_pack.json |
app 1.2.0-rc1 / rc2 | BioCLIP ViT-B/16 fp32 (345 MB) | 198 |
| v1 | (none) | app up to 1.1.1 | OpenCLIP ViT-B/32 (352 MB) | 203 |
v4 files (the encoder is the v3 file, unchanged)
| File | Description |
|---|---|
species_pack_v4.json |
Encoder contract (224 px, CLIP mean/std, 768 dimensions), scoring (image prototypes + 4 x species-name embedding), softmax temperature and thresholds calibrated on a validation split, session optimisation level, input crop mode, group threshold (groupFallback), SHA-256 of every file and a numeric self-test (the app refuses the encoder if the device result has cosine < 0.99). |
biodex_bioclip2_v3.onnx |
BioCLIP 2 image encoder. Input pixel_values [1, 3, 224, 224], output image_features [1, 768]. |
species_index_v4.bin |
Reference index: 6 prototypes per species (k-means) from 23,668 photos, plus one text row per species (SPEX format). |
species_catalog_v3.json |
Catalogue: verified common names in 6 languages with their source, AphiaID checked against WoRMS, group, short descriptions built only from WoRMS, IUCN and distribution data, MEOW ecoregions. |
reference_provenance_v4.jsonl |
Observation, photo, licence and attribution of every reference photo. |
meow_raster_v1.bin and ecoregion_hierarchy_v1.json (Marine Ecoregions of the World) are shared by all packs for
the geographic prior.
v5 files (everything else is the v4 pack, unchanged)
| File | Description |
|---|---|
species_pack_v5.json |
The v4 manifest plus sprites and spriteCredits, with the SHA-256 of each new file. Updating from v4 downloads only these files (about 0.8 MB). |
sprites/<speciesKey>.webp |
Pixel-art sprite of each of the 232 species (256 px, lossless WebP), shown in the BioDex. |
sprite_credits.json |
Source image of each sprite: attribution, licence and page. |
Each sprite is drawn from one image of the live, whole animal: 217 iNaturalist photos (199 CC BY, 18 CC0) and 15 Wikimedia Commons images (9 public domain, mostly 19th-century naturalist plates, and 6 CC BY). The organism is cut out, redrawn with Stable Diffusion XL (img2img with an edge ControlNet and a style reference) and reduced to a 128-pixel grid.
Evaluation (pack v4)
Through the app's own chain on photos that are not in the reference index. Thresholds chosen on validation for precision >= 0.95 of the confident answers.
| Test set | Top-1 | Top-5 | Confident answers: share / precision | Out-of-catalogue photos accepted as confident |
|---|---|---|---|---|
| iNaturalist, observers disjoint from the references (3,550 photos; 318 out-of-catalogue) | 0.893 | 0.980 | 63 % / 98.2 % | 8.8 % |
| Other sources: GBIF datasets other than iNaturalist and Wikimedia Commons (4,912 photos; 186 out-of-catalogue) | 0.814 | 0.937 | 57 % / 97.7 % | 3.8 % |
The out-of-catalogue photos of v4 include look-alike relatives of catalogue species, so its confident share is lower than v3 (69 % / 62 % on the v3 sets) at the same precision. The 30 new species: top-1 0.89 on both sets, confident answers 99-100 % right. When no species is confident, the group answer covers 59 % of those photos and is right 93 % of the time (threshold chosen on validation for 95 %). "Possible" answers (shown, not saved) are right about 73 % of the time. The labels of the second set have not been checked by an expert. On a PC CPU (4 threads) one photo takes about 0.5 s with about 0.8 GB of memory; phone figures are not measured yet.
Licences
- Reference index and provenance (
species_index_*.bin,reference_provenance*.jsonl): CC BY 4.0. Derived from iNaturalist research-grade photos licensed CC0 or CC BY only; the photos are not redistributed. Attribution of each photo is in the provenance file. - Sprites (
sprites/,sprite_credits.json): CC BY 4.0. Each one is derived from a CC0, CC BY or public-domain image; its author, licence and page are insprite_credits.jsonand the app shows them on the species page. - Encoders: BioCLIP 2 (imageomics/bioclip-2), BioCLIP (imageomics/bioclip) and OpenCLIP ViT-B/32: MIT licence.
- Catalogue: names and classification from WoRMS (CC BY 4.0), FAO ASFIS and GBIF; species ranges from OBIS and GBIF (CC BY 4.0) and AquaMaps (CC BY-NC; check its terms before any commercial use).
- MEOW (Spalding et al., 2007, The Nature Conservancy): free use with attribution.
Citation
@inproceedings{gu2025bioclip2,
title = {{BioCLIP 2}: Emergent Properties from Scaling Hierarchical Contrastive Learning},
author = {Gu, Jianyang and Stevens, Samuel and Campolongo, Elizabeth G. and Thompson, Matthew J. and Zhang, Net and Wu, Jiaman and Kopanev, Andrei and Mai, Zheda and White, Alexander E. and Balhoff, James and Dahdul, Wasila and Rubenstein, Daniel and Lapp, Hilmar and Berger-Wolf, Tanya and Chao, Wei-Lun and Su, Yu},
booktitle = {NeurIPS},
year = {2025}
}
@inproceedings{stevens2024bioclip,
title = {{BioCLIP}: A Vision Foundation Model for the Tree of Life},
author = {Stevens, Samuel and Wu, Jiaman and Thompson, Matthew J. and Campolongo, Elizabeth G. and Song, Chan Hee and Carlyn, David Edward and Dong, Li and Dahdul, Wasila M. and Stewart, Charles and Berger-Wolf, Tanya and Chao, Wei-Lun and Su, Yu},
booktitle = {CVPR},
year = {2024}
}
Code to rebuild and evaluate the packs: tools/species/ and docs/SPECIES_ID.md in
asferrer/oceanguard-app.