| --- |
| library_name: pytorch |
| tags: |
| - 3d-segmentation |
| - point-cloud |
| - interactive-segmentation |
| - promptable-segmentation |
| - safetensors |
| --- |
| |
| # DynamicSAM |
|
|
| Interactive 3D point cloud segmentation with adaptive masking. Given a point cloud and click prompts (foreground/background), DynamicSAM produces a per-point foreground probability mask. |
|
|
| - **Input:** XYZ-only point cloud (no RGB) + click prompts (point coordinates + foreground/background labels) |
| - **Output:** per-point foreground probability, binarized at inference time by a tunable threshold |
| - **Architecture:** PointBERT-style point cloud encoder, a prompt encoder that shares its positional-embedding MLP with the point cloud encoder, and a two-way-transformer mask decoder with a PointNet++ upsampling head (adapted from the Segment Anything Model's decoder design, without an IoU-prediction head) |
|
|
| ## Files |
|
|
| - `model.safetensors` — inference weights (point cloud encoder, prompt encoder, mask decoder), including `group_size`/`num_group` in its metadata. |
|
|
| ## Training |
|
|
| - **Dataset:** ScanNet (ScanNet40) |
| - **Config:** `trans_dim=384`, `group_size=32`, `num_group=128`, batch size 128, AdamW + multi-step LR decay |
|
|
| ## Usage |
|
|
| Install [dynamic-sam](https://github.com/DESRLab/dynamic-sam.git) with the `inference` extra, then serve directly from this repo: |
|
|
| ```bash |
| pip install -e ".[inference]" |
| dynamic-sam serve --hf-repo-id Marali/dynamic-sam |
| ``` |
|
|
| This downloads `model.safetensors` through the local Hugging Face cache (`HF_HOME`) and serves it behind a FastAPI HTTP API (`/encode_pcd`, `/predict_mask`). |
|
|
| To load the weights directly in Python instead: |
|
|
| ```python |
| from src.serve.checkpoint import load_dynamic_sam |
| |
| model = load_dynamic_sam("model.safetensors", device=0) |
| ``` |
|
|
