| --- |
| license: apache-2.0 |
| tags: |
| - self-supervised-learning |
| - vicreg |
| - joint-embedding |
| - marine-biology |
| - metagenomics |
| - environmental-modeling |
| - protein-domains |
| - tara-oceans |
| - pfam |
| - pytorch |
| pipeline_tag: tabular-regression |
| --- |
| |
| # TARA-WorldModel-VICReg |
|
|
| Joint environment-proteome embedding model using VICReg (Variance-Invariance-Covariance Regularization) self-supervised learning, applied to the TARA Oceans metagenomic dataset. This model aligns environmental and Pfam protein domain representations in a shared 32-dimensional latent space. |
|
|
| This model represents an exploratory methodological approach deposited for transparency and reproducibility. The XGBoost bidirectional framework ([TARA-XGBoost-Bidirectional](https://huggingface.co/GreenGenomicsLab/TARA-XGBoost-Bidirectional)) was retained as the primary modeling approach in the ELF-NET study. |
|
|
| ## Architecture |
|
|
| ``` |
| Environment branch: Input(env_dim) -> Linear(hidden) -> ReLU -> Dropout(0.3) -> Linear(32) |
| Pfam branch: Input(pfam_dim) -> Linear(hidden) -> ReLU -> Dropout(0.3) -> Linear(32) |
| ``` |
|
|
| | Property | Value | |
| |----------|-------| |
| | Latent dimension | 32 | |
| | Parameters | ~53K--64K (varies with Pfam input dimensionality) | |
| | VICReg loss weights | variance = 25.0, invariance = 25.0, covariance = 1.0 | |
| | Prediction head alpha | 1.0 | |
|
|
| ## Training Data |
|
|
| | Property | Value | |
| |----------|-------| |
| | Source | 1,151 samples with complete productivity data (Chl-a, POC, NFLH) from 1,810 total TARA Oceans samples | |
| | Environmental features | Google Earth Engine oceanographic variables | |
| | Pfam features | CLR-transformed domain abundances reduced via PCA to 20, 32, or 64 dimensions | |
|
|
| ## Performance |
|
|
| ### 6-Fold Leave-One-Basin-Out (LOBO) CV |
|
|
| | Target | Joint Model R² | Env-Only Baseline R² | Cohen's d | p-value | |
| |--------|----------------|----------------------|-----------|---------| |
| | POC | 0.532 | 0.422 | 0.026 | 0.38 | |
| | Chl-a | 0.516 | 0.561 | -- | -- | |
| | NFLH | 0.560 | 0.700 | -- | -- | |
|
|
| ### 9-Fold Spatial Block CV (matching primary XGBoost design) |
|
|
| | Pfam dim | XGB Baseline R² | VICReg R² | Delta R² | |
| |----------|-----------------|-----------|----------| |
| | pfam20 | 0.417 | -2.045 | -2.462 | |
| | pfam32 | 0.417 | -4.217 | -4.634 | |
| | pfam64 | 0.417 | -1.262 | -1.679 | |
|
|
| The negative R² under spatial CV reflects the MLP architecture's sensitivity to distribution shift on spatially distinctive held-out basins (Mediterranean, mid-Pacific), a known limitation of shallow neural networks on small tabular datasets (N ~ 1,100). This is an architecture confound, not evidence against the Pfam alignment signal itself. |
|
|
| ## Repository Contents |
|
|
| | Directory | Contents | |
| |-----------|----------| |
| | `checkpoints/` | 24 model checkpoints (4 hyperparameter configurations x 6 ocean basin folds) | |
| | `scripts/` | Core training code (`train_world_model.py`, `vicreg_loss.py`, `world_model.py`) | |
| | `results/` | Per-fold metrics, training curves, hyperparameter sweep results, permutation tests | |
| | `config/` | Best hyperparameter configuration | |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| |
| checkpoint = torch.load( |
| "checkpoints/20260127_111754/world_model_fold_Arctic_20260127_111754.pt", |
| map_location="cpu", |
| weights_only=False |
| ) |
| state_dict = checkpoint["model_state_dict"] |
| ``` |
|
|
| ## Related Resources |
|
|
| | Resource | Link | |
| |----------|------| |
| | ELF-NET analysis pipeline (371 scripts, 15 modules) | [github.com/olympus-terminal/ELF-NET](https://github.com/olympus-terminal/ELF-NET) | |
| | Bidirectional XGBoost models (primary approach) | [TARA-XGBoost-Bidirectional](https://huggingface.co/GreenGenomicsLab/TARA-XGBoost-Bidirectional) | |
| | algaGPT protein classifier | [GreenGenomicsLab/algaGPT](https://huggingface.co/GreenGenomicsLab/algaGPT) | |
| | Dark-whiteGPLM checkpoints | [SarahDaakour/dark-whiteGPLM](https://huggingface.co/SarahDaakour/dark-whiteGPLM) | |
|
|
| ## References |
|
|
| - Bardes, A., Ponce, J., & LeCun, Y. (2022). VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning. *ICLR 2022*. |
|
|
| ## Authors |
|
|
| David R. Nelson, Kourosh Salehi-Ashtiani |
|
|
| New York University Abu Dhabi |
|
|
| ## Citation |
|
|
| ```bibtex |
| @article{nelson2026elfnet, |
| title = {Coupling of oceanographic state to the dark proteome: a foundation for genome-informed marine productivity modeling}, |
| author = {Nelson, David Roy and Plouviez, Maxence and Daakour, Sarah and Jaiswal, Ashish and Fu, Weiqi and Amin, Shady A. and Salehi-Ashtiani, Kourosh}, |
| journal = {Forthcoming}, |
| year = {2026} |
| } |
| ``` |
|
|
| ## Contact |
|
|
| Kourosh Salehi-Ashtiani -- ksa3@nyu.edu |
|
|
