FISHER: Gradient-Decoupled Hierarchical Multi-Task Learning for Fine-Grained Aquatic Species Recognition

This repository contains the official implementation of FISHER, a novel framework designed for the fine-grained recognition of aquatic species, specifically addressing the challenges of long-tailed distributions where ultra-rare species are poorly represented.

FISHER aligns network optimization with the natural biological hierarchy of aquatic species by breaking down recognition into three interrelated sub-tasks: semantic segmentation of anatomical parts, morphological trait prediction, and species classification.

✨ Key Features

• Detached Hierarchical Architecture: Enforces a unidirectional information flow (Segmentation → Traits → Species) and applies gradient detachment at task boundaries to prevent high-level classification objectives from corrupting lower-level morphological representations.
• Prototype-Based Segmentation: Replaces conventional decoders with learnable prototypes and orthogonality regularization, enabling compact, disentangled, and interpretable delineations of subtle anatomical structures.
• Dynamic Task Balancing: Employs homoscedastic uncertainty weighting to dynamically balance the contributions of dense tasks (segmentation) with higher-level tasks during training.
• High Performance: Achieves state-of-the-art results on the large-scale Fish-Vista dataset, including 97.7% mAP for unseen trait identification and a 13.4% accuracy improvement for ultra-rare species compared to strong baselines.

⚙️ Installation

1. Clone the repository:

   git clone https://github.com/phucngvinuni/MTL
   cd MTL
   

2. Install dependencies:

   pip install torch torchvision pandas numpy tqdm scikit-learn torchmetrics opencv-python matplotlib
   

🚀 Usage

1. Training (Model)

To train the FISHER model from scratch:

python train_detached.py

Config: Batch size 32 (default evaluated in paper), Learning Rate 1e-4 using AdamW, 50 Epochs with Cosine Annealing.

• Output: Checkpoints will be automatically saved to the checkpoints_detached/ directory.

2. Evaluation

To evaluate the trained model across all three hierarchical tasks (Species Classification, Trait Identification, and Semantic Segmentation) and generate the final metrics JSON:

python evaluationdetached.py

• Note: Ensure you update the CHECKPOINT_PATH inside evaluationdetached.py to point to your best saved model (e.g., best_model.pth) before running.

💾 Checkpoints

Pre-trained model checkpoints can be found at: [Insert Link to HuggingFace / Google Drive / Release Assets Here]

📖 Citation

If you find this code or our research helpful in your work, please cite our paper:

@misc{nguyen2026fishergradientdecoupledhierarchicalmultitask,
      title={FISHER: Gradient-Decoupled Hierarchical Multi-Task Learning for Fine-Grained Aquatic Species Recognition}, 
      author={Phuc H. Nguyen and Ba Hung Ngo and Mai Phuong Tran and Cuong D. Do and Van-Dinh Nguyen},
      year={2026},
      eprint={2607.20523},
      archivePrefix={arXiv},
      primaryClass={q-bio.QM},
      url={https://arxiv.org/abs/2607.20523}, 
}

---