Instructions to use Jim-myoenk/FDS-UNeXT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use Jim-myoenk/FDS-UNeXT with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("Jim-myoenk/FDS-UNeXT") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("Jim-myoenk/FDS-UNeXT") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
FDS-UNeXT
Checkpoints for "Boundary Fidelity in Frozen Foundation-Model Adaptation: When It Holds and When It Fails for Fine-Grained Rooftop Segmentation" (submitted to Remote Sensing).
FDS-UNeXT adapts two frozen foundation models, the SAM 2.1 Hiera-L encoder and DINOv2 ViT-L/14, to four-class rooftop material segmentation with 6.97 M trainable parameters (1.33 % of 523.1 M).
Code: https://github.com/Zhao-Yuancheng/fds-unext
Format: trainable tensors only
A full checkpoint is 2.0 GB, but 516.2 M of its 523.1 M parameters are the frozen trunks and are identical to the public SAM 2.1 and DINOv2 weights. Each file here stores only the 6.97 M tensors that training updated, which makes it about 28 MB. The round trip is exact: merging reproduces every tensor of the original file.
You still need the two public trunks:
wget -P weights https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_large.pt
wget -P weights https://dl.fbaipublicfiles.com/dinov2/dinov2_vitl14/dinov2_vitl14_pretrain.pth
Files
| File | Trained on | Used for |
|---|---|---|
lrmd-fold0.pth … lrmd-fold4.pth |
LRMD, 0.14 m, five folds | Table 3, Figures 2 and 5 |
potsdam-seed42.pth, potsdam-seed3407.pth |
ISPRS Potsdam, 0.05 m | Table 4, Figure 9a |
egcs-seed42.pth, egcs-seed3407.pth, egcs-seed666.pth |
E-GCS, 0.3 m | Table 4, Figures 3 and 9b |
Usage
hf download Jim-myoenk/FDS-UNeXT lrmd-fold0.pth --local-dir deltas
python tools/merge_trainable.py \
--delta deltas/lrmd-fold0.pth \
--hiera_path weights/sam2.1_hiera_large.pt \
--dinov2_path weights/dinov2_vitl14_pretrain.pth \
--out runs/fold0/FDS-UNeXT-best.pth
python eval.py \
--hiera_path weights/sam2.1_hiera_large.pt \
--dinov2_path weights/dinov2_vitl14_pretrain.pth \
--checkpoint runs/fold0/FDS-UNeXT-best.pth \
--split_path dataset/roof_dataset_v2/split_fold0.json --split test \
--num_classes 4 --bf_tolerance 8 --dy_edge_offset 6 \
--class_names background,high-albedo-steel,low-albedo-steel,tile
Classes: 0 background (concrete roofs, asphalt, soil, vegetation, roads,
water, vehicles), 1 high-albedo color-coated steel, 2 low-albedo
color-coated steel, 3 tile.
Results
| Dataset | Setting | mIoU (%) | Boundary F-score (%) |
|---|---|---|---|
| LRMD | five folds, 0.14 m | 70.00 ± 6.58 | 68.58 ± 5.08 |
| Potsdam | two seeds, 0.05 m | 75.56 ± 0.49 | 73.18 ± 0.65 |
| E-GCS | three seeds, 0.3 m | 95.15 ± 0.37 | 73.05 ± 1.30 |
The boundary F-score uses an 8-pixel tolerance and class-equal weighting (per-class mean over images, then mean over classes). Metrics skip classes that are absent from the reference tile.
For continuity with earlier releases, note that those releases used element-weighted BF (Potsdam seed 42: 73.14; E-GCS three-seed: 76.17 ± 0.59). The two conventions must not be mixed or subtracted.
Citation
@article{zhao2026fdsunext,
title = {Boundary Fidelity in Frozen Foundation-Model Adaptation: When It
Holds and When It Fails for Fine-Grained Rooftop Segmentation},
author = {Zhao, Yuancheng and Wang, Jie and Li, Faguo and Liu, Erte and
Meng, Xiangying and Chen, Yueyue and Yang, Jiapeng and
Zhou, Yibo and Li, Qingxue},
journal = {Remote Sensing},
year = {2026},
note = {Under review}
}
License: Apache-2.0 for this model and for the two upstream trunks.
- Downloads last month
- -