File size: 1,748 Bytes
9312846 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 | ---
library_name: pytorch
tags:
- 3d-segmentation
- point-cloud
- interactive-segmentation
- promptable-segmentation
- safetensors
---
# DynamicSAM
Interactive 3D point cloud segmentation with adaptive masking. Given a point cloud and click prompts (foreground/background), DynamicSAM produces a per-point foreground probability mask.
- **Input:** XYZ-only point cloud (no RGB) + click prompts (point coordinates + foreground/background labels)
- **Output:** per-point foreground probability, binarized at inference time by a tunable threshold
- **Architecture:** PointBERT-style point cloud encoder, a prompt encoder that shares its positional-embedding MLP with the point cloud encoder, and a two-way-transformer mask decoder with a PointNet++ upsampling head (adapted from the Segment Anything Model's decoder design, without an IoU-prediction head)
## Files
- `model.safetensors` — inference weights (point cloud encoder, prompt encoder, mask decoder), including `group_size`/`num_group` in its metadata.
## Training
- **Dataset:** ScanNet (ScanNet40)
- **Config:** `trans_dim=384`, `group_size=32`, `num_group=128`, batch size 128, AdamW + multi-step LR decay
## Usage
Install [dynamic-sam](https://github.com/DESRLab/dynamic-sam.git) with the `inference` extra, then serve directly from this repo:
```bash
pip install -e ".[inference]"
dynamic-sam serve --hf-repo-id Marali/dynamic-sam
```
This downloads `model.safetensors` through the local Hugging Face cache (`HF_HOME`) and serves it behind a FastAPI HTTP API (`/encode_pcd`, `/predict_mask`).
To load the weights directly in Python instead:
```python
from src.serve.checkpoint import load_dynamic_sam
model = load_dynamic_sam("model.safetensors", device=0)
```
|