How to use from the
Use from the
sam2 library
# Gated model: Login with a HF token with gated access permission
hf auth login
# Use SAM2 with images
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor

predictor = SAM2ImagePredictor.from_pretrained("nycu-cplab/3AM")

with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    predictor.set_image(<your_image>)
    masks, _, _ = predictor.predict(<input_prompts>)
# Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor

predictor = SAM2VideoPredictor.from_pretrained("nycu-cplab/3AM")

with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
    state = predictor.init_state(<your_video>)

    # add new prompts and instantly get the output on the same frame
    frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>)

    # propagate the prompts to get masklets throughout the video
    for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
        ...

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

3AM

Weights for 3AM: Segment Anything with Adaptive Geometric Consistency in Videos: SAM2.1 (Hiera-L) whose memory attention fuses MUSt3R 3D features through routing tokens. It works on RGB video only, without camera poses or depth.

Demo: spaces/nycu-cplab/3AM

File Contents
sam2.1-3am-routing-fusion-v1-contiguous-step1486336.pt 3AM weights, selected by ScanNet++ validation pIoU (training step 1,486,336). model is the full SAM2.1 Hiera-L state dict, including the routing-fusion memory attention.
MUSt3R_512.pth MUSt3R-512 weights used for the 3D features (naver/must3r, NAVER non-commercial license).
examples/ Sampled frames, prompts, and tracking results for the demo examples.
import torch

ckpt = torch.load("sam2.1-3am-routing-fusion-v1-contiguous-step1486336.pt", map_location="cpu")
state_dict = ckpt["model"]  # load with strict=True into the 3AM predictor

The Space's engine.py (build_3am_predictor, run_must3r, track) contains the complete inference pipeline.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Space using nycu-cplab/3AM 1