File size: 10,062 Bytes

f7cceee
 
 
 
 
 
 
 
 
 
 
 
 
89cf0f9
f7cceee
 
 
 
 
 
 
 
 
 
 
 
f9840e6
f7cceee
 
 
 
 
 
 
 
 
89cf0f9
 
f7cceee
 
 
 
 
 
89cf0f9
f7cceee
 
 
89cf0f9
f7cceee

---
language:
- en
tags:
- biology
- dna
- biodiversity
- hyperbolic-embeddings
- pytorch
---

# Model Card for SSEL

Checkpoints and extracted DNA embeddings for **Hierarchical Embedding of DNA Barcode Taxonomy with Hyperbolic Entailment Cones**. This repository contains the nine 768D experiments presented in the iBOL poster (eight also appear in the UniReps submission), plus an additional 40-epoch DNA-only SSEL checkpoint.

## Model Details

### Model Description

**Structured Stacked Entailment Loss (SSEL)** extends **Stacked Entailment Loss (SEL)** with minimum-distance, full-cone nesting, sibling-cone separation and descendant-based cone allocation constraints. DNA barcodes and taxonomic labels are embedded in a shared hyperbolic space.

The DNA branch starts from [BarcodeBERT](https://huggingface.co/bioscan-ml/clibd/tree/main/ckpt/BarcodeBERT), followed by a 768 → 1536 → 768 MLP without a residual connection. Taxonomic labels use learned embedding tables. These runs use DNA and four taxonomic ranks, without an image branch.

### Model Sources

- [Project page](https://bioscan-ml.org/ssel/)
- [Code preparation repository](https://github.com/zmgong/SSEL_temp_release) (available soon)
- [Paper](#)

### Model Checkpoints

| Method | Checkpoint (`last.pth`) | Test embeddings | Saved configuration | UniReps |
| --- | --- | --- | --- | --- |
| EL | [el](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_EL_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |
| SEL | [sel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_no_radius_barrier_4rank_posneg_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |
| Positive EL + CL | [positive-el-cl](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_MERU_style_4rank_posEL02_CL1_768d_nonres_gap05_15epochs_seed42/config.yaml) | No |
| SSEL (40 epochs) | [ssel-40epochs](Ver_1_1/ckpt/bioscan_clip/bioscan_5m/SSEL_granularity_comparison_768d_dna_only_nonres_40epochs_seed42/last.pth) | — | [Inference config](Ver_1_1/ckpt/bioscan_clip/bioscan_5m/SSEL_granularity_comparison_768d_dna_only_nonres_40epochs_seed42/inference_config.json) | No (additional checkpoint) |
| SSEL (15 epochs, UniReps) | [ssel](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/SEL_structure_v2_768d_dna_nonres_margin05_gap05_weight10_15epochs_seed42/config.yaml) | Yes |
| No direct placement | [no-direct-placement](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_no_direct_placement_768d_nonres_gap05_15epochs_seed42_retry_20261008/config.yaml) | Yes |
| Frozen labels | [frozen-labels](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_full_frozen_labels_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |
| SEL + nesting | [sel-nesting](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_nesting_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |
| SEL + separation | [sel-separation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_sibling_separation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |
| SEL + allocation | [sel-allocation](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/last.pth) | [DNA](Ver_1_0/extracted_embedding/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/extracted_feature_from_test_split.hdf5) | [Config](Ver_1_0/ckpt/bioscan_clip/bioscan_5m/UniReps_SEL_basic_plus_allocation_768d_nonres_gap05_15epochs_seed42/config.yaml) | Yes |

Each selected checkpoint is the exact **last.pth** used for its reported result. The nine original `Ver_1_0` runs use 768 dimensions, 15 epochs and seed 42. The additional `ssel-40epochs` checkpoint uses 768 dimensions, 40 epochs and seed 42; its [full validation summary](Ver_1_1/ckpt/bioscan_clip/bioscan_5m/SSEL_granularity_comparison_768d_dna_only_nonres_40epochs_seed42/validation_summary.json) reports 99.56% seen Species nearest-label accuracy, 82.78% unseen-species detection and 2.52% seen false rejection on 14,757 seen and 8,819 unseen validation queries. It is separate from the original 15-epoch UniReps checkpoint. EL and SEL omit minimum distance; the other listed variants include it. Positive EL + CL is our DNA–taxonomy adaptation, not an exact reproduction of image–text MERU.

## Files and Usage

The original poster artifacts are under **Ver_1_0/**. The added 40-epoch checkpoint and its inference-only configuration are under **Ver_1_1/**; no test embeddings are published for this checkpoint. The layout follows CLIBD's `ckpt/` and `extracted_embedding/` structure, with the version directory placed at the outside:

```text
Ver_1_0/
  ckpt/
    bioscan_clip/bioscan_5m/<model_output_name>/
      last.pth
      config.yaml
    BarcodeBERT/old_checkpoints/trained_with_canada_1_5M/model_41.pth
  extracted_embedding/
    bioscan_5m/<model_output_name>/
      extracted_feature_from_test_split.hdf5
      labels_test.json
      processed_id_test.json
      dna_to_label.csv
      species_detection.csv
      evaluation_complete.json
    training_seen_taxonomy_768d_weighted_simplex_beta025_margin05_gap05_placement/
  data/BIOSCAN_5M/global_label_to_id_mapping.json
  metadata/experiments.json
  SHA256SUMS.txt
```

The checkpoints are native PyTorch state dictionaries and use the SSEL codebase's loader. They are not Transformers `AutoModel` packages. Configuration copies retain the scientific settings; local absolute roots become `${oc.env:SSEL_ARTIFACT_ROOT,.}`, and W&B is disabled by default. Set `SSEL_ARTIFACT_ROOT` to the downloaded `Ver_1_0` directory. The processed BIOSCAN-5M HDF5 is downloaded separately into `data/BIOSCAN_5M/` within that root. The current loader additionally uses `prajjwal1/bert-small` from Hugging Face or its local cache.

### Embedding Format

Each HDF5 contains **test DNA embeddings only**, in the saved checkpoint's hyperbolic space:

| HDF5 dataset | Shape | Type |
| --- | --- | --- |
| `seen/encoded_dna_feature` | 39,373 × 768 | float32 |
| `unseen/encoded_dna_feature` | 7,887 × 768 | float32 |

The companion JSON files provide labels and process IDs in exactly the same row order. These are spatial coordinates of Lorentz-model hyperbolic embeddings; they are not unit-normalized Euclidean vectors. Recover the time coordinate using the checkpoint curvature when needed.

Final trained taxonomic label tables, learned scales and curvature are included **inside each checkpoint**. The shared placement artifact supplies initial coordinates, candidate labels, global IDs and parent relationships; **initial coordinates are not final trained label embeddings**. Separate final-label vector files, train/validation embeddings and specimen reference-key caches are not included in this version.

Classification uses the nearest label by exterior angle at each rank. Unseen-species detection uses cone checks against the seen-species label candidates. These are separate measurements; the accompanying metric CSVs and evaluation records preserve that distinction.

## Training Data

The experiments use [BIOSCAN-5M](https://huggingface.co/datasets/bioscan-ml/BIOSCAN-5M). The [processed HDF5](https://huggingface.co/datasets/bioscan-ml/clibd) has 39,373 seen-species and 7,887 unseen-species test queries. Label candidates are training-derived: 55 orders, 929 families, 6,819 genera and 11,846 species. The raw dataset is not stored in this model repository.

## Acknowledgement

This work was supported by the Government of Canada's New Frontiers in Research Fund (NFRF) [NFRFT-2020-00073] and the Canada CIFAR AI Chairs Program. We acknowledge BIOSCAN-5M, BarcodeBERT and the CLIBD codebase. This model card and artifact layout follow [CLIBD](https://huggingface.co/bioscan-ml/clibd).