| --- |
| language: |
| - en |
| tags: |
| - biology |
| - dna |
| - biodiversity |
| - hyperbolic-embeddings |
| - pytorch |
| --- |
| |
| # Model Card for SSEL |
|
|
| Checkpoints and extracted DNA embeddings for **Hierarchical Embedding of DNA Barcode Taxonomy with Hyperbolic Entailment Cones**. This repository contains the nine 768D experiments presented in the iBOL poster; eight also appear in the UniReps submission. |
|
|
| ## Model Details |
|
|
| ### Model Description |
|
|
| **Structured Stacked Entailment Loss (SSEL)** extends **Stacked Entailment Loss (SEL)** with minimum-distance, full-cone nesting, sibling-cone separation and descendant-based cone allocation constraints. DNA barcodes and taxonomic labels are embedded in a shared hyperbolic space. |
|
|
| The DNA branch starts from [BarcodeBERT](https://huggingface.co/bioscan-ml/clibd/tree/main/ckpt/BarcodeBERT), followed by a 768 → 1536 → 768 MLP without a residual connection. Taxonomic labels use learned embedding tables. These runs use DNA and four taxonomic ranks, without an image branch. |
|
|
| ### Model Sources |
|
|
| - [Project page](https://bioscan-ml.org/ssel/) |
| - [Code preparation repository](https://github.com/zmgong/SSEL_temp_release) (available soon) |
| - [Paper](#) |
|
|
| ### Model Checkpoints |
|
|
| | Method | Checkpoint (`last.pth`) | Test embeddings | Saved configuration | UniReps | |
| | --- | --- | --- | --- | --- | |
| | EL | [el](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
| | SEL | [sel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
| | Positive EL + CL | [positive-el-cl](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/config.yaml) | No | |
| | SSEL | [ssel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/config.yaml) | Yes | |
| | No direct placement | [no-direct-placement](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/config.yaml) | Yes | |
| | Frozen labels | [frozen-labels](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
| | SEL + nesting | [sel-nesting](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
| | SEL + separation | [sel-separation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
| | SEL + allocation | [sel-allocation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | |
|
|
| Each selected checkpoint is the exact **last.pth** used for its reported result. All nine runs use 768 dimensions, 15 epochs and seed 42. EL and SEL omit minimum distance; the other listed variants include it. Positive EL + CL is our DNA–taxonomy adaptation, not an exact reproduction of image–text MERU. |
|
|
| ## Files and Usage |
|
|
| All versioned artifacts are under **Ver_1_0/**. The layout follows CLIBD's `ckpt/` and `extracted_embedding/` structure, with the version directory placed at the outside: |
|
|
| ```text |
| Ver_1_0/ |
| ckpt/ |
| bioscan_clip/bioscan_5m/<model_output_name>/ |
| last.pth |
| config.yaml |
| BarcodeBERT/old_checkpoints/trained_with_canada_1_5M/model_41.pth |
| extracted_embedding/ |
| bioscan_5m/<model_output_name>/ |
| extracted_feature_from_test_split.hdf5 |
| labels_test.json |
| processed_id_test.json |
| dna_to_label.csv |
| species_detection.csv |
| evaluation_complete.json |
| training_seen_taxonomy_768d_weighted_simplex_beta025_margin05_gap05_placement/ |
| data/BIOSCAN_5M/global_label_to_id_mapping.json |
| metadata/experiments.json |
| SHA256SUMS.txt |
| ``` |
|
|
| The checkpoints are native PyTorch state dictionaries and use the SSEL codebase's loader. They are not Transformers `AutoModel` packages. Configuration copies retain the scientific settings; local absolute roots become `${oc.env:SSEL_ARTIFACT_ROOT,.}`, and W&B is disabled by default. Set `SSEL_ARTIFACT_ROOT` to the downloaded `Ver_1_0` directory. The processed BIOSCAN-5M HDF5 is downloaded separately into `data/BIOSCAN_5M/` within that root. The current loader additionally uses `prajjwal1/bert-small` from Hugging Face or its local cache. |
|
|
| ### Embedding Format |
|
|
| Each HDF5 contains **test DNA embeddings only**, in the saved checkpoint's hyperbolic space: |
|
|
| | HDF5 dataset | Shape | Type | |
| | --- | --- | --- | |
| | `seen/encoded_dna_feature` | 39,373 × 768 | float32 | |
| | `unseen/encoded_dna_feature` | 7,887 × 768 | float32 | |
|
|
| The companion JSON files provide labels and process IDs in exactly the same row order. These are spatial coordinates of Lorentz-model hyperbolic embeddings; they are not unit-normalized Euclidean vectors. Recover the time coordinate using the checkpoint curvature when needed. |
|
|
| Final trained taxonomic label tables, learned scales and curvature are included **inside each checkpoint**. The shared placement artifact supplies initial coordinates, candidate labels, global IDs and parent relationships; **initial coordinates are not final trained label embeddings**. Separate final-label vector files, train/validation embeddings and specimen reference-key caches are not included in this version. |
|
|
| Classification uses the nearest label by exterior angle at each rank. Unseen-species detection uses cone checks against the seen-species label candidates. These are separate measurements; the accompanying metric CSVs and evaluation records preserve that distinction. |
|
|
| ## Training Data |
|
|
| The experiments use [BIOSCAN-5M](https://huggingface.co/datasets/bioscan-ml/BIOSCAN-5M). The [processed HDF5](https://huggingface.co/datasets/bioscan-ml/clibd) has 39,373 seen-species and 7,887 unseen-species test queries. Label candidates are training-derived: 55 orders, 929 families, 6,819 genera and 11,846 species. The raw dataset is not stored in this model repository. |
|
|
| ## Acknowledgement |
|
|
| This work was supported by the Government of Canada's New Frontiers in Research Fund (NFRF) [NFRFT-2020-00073] and the Canada CIFAR AI Chairs Program. We acknowledge BIOSCAN-5M, BarcodeBERT and the CLIBD codebase. This model card and artifact layout follow [CLIBD](https://huggingface.co/bioscan-ml/clibd). |
|
|
