| --- |
| license: mit |
| library_name: pytorch |
| pipeline_tag: image-segmentation |
| tags: |
| - semantic-segmentation |
| - microstructure |
| - materials-science |
| - segment-anything |
| - lora |
| - matsam |
| --- |
| |
| # MatSAM Fine-Tuned Checkpoints for Microstructure Segmentation |
|
|
| ## Model Description |
|
|
| This repository contains dataset-specific fine-tuned parameter checkpoints for |
| MatSAM with a ViT-H image encoder. The files do not contain the full base |
| model. Each checkpoint must be loaded together with the official SAM ViT-H |
| checkpoint `sam_vit_h_4b8939.pth` using the matching test script. |
|
|
| Source code: |
| https://github.com/WUT-AI-AI4Mat/Segmentation-methods-evaluation-for-quantitative-microstructure-analysis |
|
|
| ## Fine-Tuning Routes and Checkpoints |
|
|
| | Dataset | Route | Classes | File | |
| | --- | --- | ---: | --- | |
| | Aachen-Heerlen | LoRA plus mask decoder | 2 | `checkpoints/Aachen-Heerlen/matsam_lora_decoder_best.pth` | |
| | EMPS | LoRA plus mask decoder | 2 | `checkpoints/EMPS/matsam_lora_decoder_best.pth` | |
| | Grain | LoRA plus mask decoder | 2 | `checkpoints/Grain/matsam_lora_decoder_best.pth` | |
| | EBC | LoRA, mask decoder, and class tokens | 3 | `checkpoints/EBC/semantic_matsam_best.pth` | |
| | Super | LoRA, mask decoder, and class tokens | 3 | `checkpoints/Super/semantic_matsam_best.pth` | |
| | MetalDAM | LoRA, mask decoder, and class tokens | 5 | `checkpoints/MetalDAM/semantic_matsam_best.pth` | |
| | UHCS | LoRA, mask decoder, and class tokens | 7 | `checkpoints/UHCS/semantic_matsam_best.pth` | |
|
|
| Class counts include background. |
|
|
| ## Base Model Requirement |
|
|
| Download the SAM ViT-H base checkpoint from Meta: |
| https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth |
|
|
| The fine-tuned files contain LoRA and mask-decoder parameters. Multiclass files |
| also contain learned class-token parameters. They are not standalone weights. |
|
|
| ## Training Details |
|
|
| - Base architecture: ViT-H |
| - SAM encoder input: longest side resized and padded to 1024 x 1024 |
| - Training batch size: 1 |
| - Epochs: 200 |
| - Optimizer: AdamW |
| - Learning rate: 0.0001 |
| - Weight decay: 0.0001 |
| - Scheduler: CosineAnnealingLR with minimum learning rate 0.000001 |
| - Early-stopping patience: 50 |
| - LoRA rank: 8 |
| - LoRA alpha: 16 |
| - LoRA dropout: 0.05 |
| - LoRA targets: `qkv` and `proj` |
| - Binary objective: BCE, Dice, and IoU MSE |
| - Multiclass objective: cross-entropy and Dice |
|
|
| ## Usage |
|
|
| Example for binary EMPS segmentation: |
|
|
| ```bash |
| hf download NAMESPACE/microstructure-matsam checkpoints/EMPS/matsam_lora_decoder_best.pth --local-dir weights/matsam |
| python matsam/test_lora_decoder.py \ |
| --dataset-root /path/to/EMPS \ |
| --checkpoint weights/pretrained/sam_vit_h_4b8939.pth \ |
| --finetuned-checkpoint weights/matsam/checkpoints/EMPS/matsam_lora_decoder_best.pth \ |
| --output-dir results/matsam_emps \ |
| --method-type 2 |
| ``` |
|
|
| For Grain, use `--method-type 1`. For other metallographic datasets, use |
| `--method-type 2`. For multiclass checkpoints, use |
| `matsam/test_semantic_matsam.py` and pass the class count from the table. |
|
|
| ## Inference Parameters |
|
|
| The original automatic-mask path uses `n_per_side_base=54`, |
| `pred_iou_thresh=0.88`, `stability_score_thresh=0.9`, |
| `box_nms_thresh=0.7`, `crop_nms_thresh=0.7`, and `crop_n_layers=0`. The binary |
| fine-tuned path uses point prompts and thresholds of 0.7 for predicted IoU and |
| stability score. |
|
|
| ## Evaluation, Intended Use, and Limitations |
|
|
| Evaluation restores masks to their original image dimensions and calculates |
| the common benchmark metrics through `Myutils/metrics.py`. These checkpoints |
| are intended for research reproduction on the named datasets. They require the |
| exact ViT-H base architecture and matching script. They may not generalize to |
| unseen materials or imaging conditions and are not validated for safety-critical |
| or industrial quality-control decisions. |
|
|
| ## License and Citation |
|
|
| The released experiment files are provided under the MIT license. The SAM base |
| checkpoint and MatSAM implementation remain subject to their upstream terms. |
| The base checkpoint is not redistributed here. A paper citation will be added |
| after publication. |
|
|
