--- language: - en tags: - biology - dna - biodiversity - hyperbolic-embeddings - pytorch --- # Model Card for SSEL Checkpoints and extracted DNA embeddings for **Hierarchical Embedding of DNA Barcode Taxonomy with Hyperbolic Entailment Cones**. This repository contains the nine 768D experiments presented in the iBOL poster; eight also appear in the UniReps submission. ## Model Details ### Model Description **Structured Stacked Entailment Loss (SSEL)** extends **Stacked Entailment Loss (SEL)** with minimum-distance, full-cone nesting, sibling-cone separation and descendant-based cone allocation constraints. DNA barcodes and taxonomic labels are embedded in a shared hyperbolic space. The DNA branch starts from [BarcodeBERT](https://huggingface.co/bioscan-ml/clibd/tree/main/ckpt/BarcodeBERT), followed by a 768 → 1536 → 768 MLP without a residual connection. Taxonomic labels use learned embedding tables. These runs use DNA and four taxonomic ranks, without an image branch. ### Model Sources - [Project page](https://bioscan-ml.org/ssel/) - [Code preparation repository](https://github.com/zmgong/SSEL_temp_release) (available soon) - [Paper](#) ### Model Checkpoints | Method | Checkpoint (`last.pth`) | Test embeddings | Saved configuration | UniReps | | --- | --- | --- | --- | --- | | EL | [el](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | | SEL | [sel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | | Positive EL + CL | [positive-el-cl](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/config.yaml) | No | | SSEL | [ssel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/config.yaml) | Yes | | No direct placement | [no-direct-placement](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/config.yaml) | Yes | | Frozen labels | [frozen-labels](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | | SEL + nesting | [sel-nesting](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | | SEL + separation | [sel-separation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | | SEL + allocation | [sel-allocation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes | Each selected checkpoint is the exact **last.pth** used for its reported result. All nine runs use 768 dimensions, 15 epochs and seed 42. EL and SEL omit minimum distance; the other listed variants include it. Positive EL + CL is our DNA–taxonomy adaptation, not an exact reproduction of image–text MERU. ## Files and Usage All versioned artifacts are under **Ver_1_0/**. The layout follows CLIBD's `ckpt/` and `extracted_embedding/` structure, with the version directory placed at the outside: ```text Ver_1_0/ ckpt/ bioscan_clip/bioscan_5m// last.pth config.yaml BarcodeBERT/old_checkpoints/trained_with_canada_1_5M/model_41.pth extracted_embedding/ bioscan_5m// extracted_feature_from_test_split.hdf5 labels_test.json processed_id_test.json dna_to_label.csv species_detection.csv evaluation_complete.json training_seen_taxonomy_768d_weighted_simplex_beta025_margin05_gap05_placement/ data/BIOSCAN_5M/global_label_to_id_mapping.json metadata/experiments.json SHA256SUMS.txt ``` The checkpoints are native PyTorch state dictionaries and use the SSEL codebase's loader. They are not Transformers `AutoModel` packages. Configuration copies retain the scientific settings; local absolute roots become `${oc.env:SSEL_ARTIFACT_ROOT,.}`, and W&B is disabled by default. Set `SSEL_ARTIFACT_ROOT` to the downloaded `Ver_1_0` directory. The processed BIOSCAN-5M HDF5 is downloaded separately into `data/BIOSCAN_5M/` within that root. The current loader additionally uses `prajjwal1/bert-small` from Hugging Face or its local cache. ### Embedding Format Each HDF5 contains **test DNA embeddings only**, in the saved checkpoint's hyperbolic space: | HDF5 dataset | Shape | Type | | --- | --- | --- | | `seen/encoded_dna_feature` | 39,373 × 768 | float32 | | `unseen/encoded_dna_feature` | 7,887 × 768 | float32 | The companion JSON files provide labels and process IDs in exactly the same row order. These are spatial coordinates of Lorentz-model hyperbolic embeddings; they are not unit-normalized Euclidean vectors. Recover the time coordinate using the checkpoint curvature when needed. Final trained taxonomic label tables, learned scales and curvature are included **inside each checkpoint**. The shared placement artifact supplies initial coordinates, candidate labels, global IDs and parent relationships; **initial coordinates are not final trained label embeddings**. Separate final-label vector files, train/validation embeddings and specimen reference-key caches are not included in this version. Classification uses the nearest label by exterior angle at each rank. Unseen-species detection uses cone checks against the seen-species label candidates. These are separate measurements; the accompanying metric CSVs and evaluation records preserve that distinction. ## Training Data The experiments use [BIOSCAN-5M](https://huggingface.co/datasets/bioscan-ml/BIOSCAN-5M). The [processed HDF5](https://huggingface.co/datasets/bioscan-ml/clibd) has 39,373 seen-species and 7,887 unseen-species test queries. Label candidates are training-derived: 55 orders, 929 families, 6,819 genera and 11,846 species. The raw dataset is not stored in this model repository. ## Acknowledgement This work was supported by the Government of Canada's New Frontiers in Research Fund (NFRF) [NFRFT-2020-00073] and the Canada CIFAR AI Chairs Program. We acknowledge BIOSCAN-5M, BarcodeBERT and the CLIBD codebase. This model card and artifact layout follow [CLIBD](https://huggingface.co/bioscan-ml/clibd).