| --- |
| license: mit |
| library_name: torch-pointcloud |
| tags: |
| - point-cloud |
| - 3d |
| - pytorch |
| - torch-pointcloud |
| - act |
| - classification |
| datasets: |
| - scanobjectnn |
| base_model: torch-pointcloud/act.pretrain.runpei-dong |
| model-index: |
| - name: act.scanobjectnn-objonly.runpei-dong |
| results: |
| - task: |
| type: point-cloud-classification |
| dataset: |
| name: ScanObjectNN (OBJ_ONLY) |
| type: scanobjectnn |
| metrics: |
| - name: OA |
| type: accuracy |
| value: 91.22 |
| - name: mAcc |
| type: accuracy |
| value: 90.26 |
| --- |
| |
| # Model card for act.scanobjectnn-objonly.runpei-dong |
|
|
| An ACT point cloud classification model (masked point transformer distilled from a 2D image transformer). Trained on ScanObjectNN (OBJ_ONLY). |
| |
| ## Model Details |
| |
| - **Model Type:** Point cloud classification |
| - **Model Stats:** |
| - Params (M): 22.1 |
| - Classes: 15 |
| - Features: 768 |
| - **Dataset:** ScanObjectNN (OBJ_ONLY) |
| - **Metrics:** OA 91.22, mAcc 90.26 |
| - **Paper:** [Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?](https://arxiv.org/abs/2212.08320) |
| - **Converted from:** [RunpeiDong/ACT](https://github.com/RunpeiDong/ACT) (MIT) |
| - **Library:** [torch-pointcloud](https://github.com/arthurdjn/pytorch-pointcloud) |
|
|
| ## Install |
|
|
| ```bash |
| pip install torch-pointcloud |
| ``` |
|
|
| ## Usage |
|
|
| ```python |
| import torch |
| import torch_pointcloud as tp |
| from torch_pointcloud.utils.data import collate |
| |
| model, info = tp.create_model( |
| "act.scanobjectnn-objonly.runpei-dong", |
| pretrained=True, |
| return_info=True, |
| ) |
| model = model.eval() |
| |
| # synthetic sample with the keys a dataset provides |
| num_points = 8192 |
| sample = { |
| "pos": torch.randn(num_points, 3), |
| } |
| data = collate([sample]) |
| |
| with torch.no_grad(): |
| logits = model(data.get("x"), data["pos"], data["batch"]) |
| ``` |
|
|
| ## Feature extraction |
|
|
| ```python |
| with torch.no_grad(): |
| embeddings = model.forward_features(data.get("x"), data["pos"], data["batch"]) |
| |
| model.reset_classifier(num_classes=0) |
| with torch.no_grad(): |
| embeddings = model(data.get("x"), data["pos"], data["batch"]) # (B, 768) |
| ``` |
|
|
| ## Citation |
|
|
| ```bibtex |
| @inproceedings{dong2023act, |
| title = {Autoencoders as Cross-Modal Teachers: Can Pretrained {2D} Image Transformers Help {3D} Representation Learning?}, |
| author = {Dong, Runpei and Qi, Zekun and Zhang, Linfeng and Zhang, Junbo and Sun, Jianjian and Ge, Zheng and Yi, Li and Ma, Kaisheng}, |
| booktitle = {ICLR}, |
| year = {2023} |
| } |
| |
| @inproceedings{uy2019scanobjectnn, |
| title = {Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data}, |
| author = {Mikaela Angelina Uy and Quang-Hieu Pham and Binh-Son Hua and Duc Thanh Nguyen and Sai-Kit Yeung}, |
| booktitle = {ICCV}, |
| year = {2019} |
| } |
| |
| @software{dujardin2026pytorchpointcloud, |
| author = {Arthur Dujardin}, |
| title = {PyTorch PointCloud}, |
| year = {2026}, |
| doi = {10.5281/zenodo.22159632}, |
| url = {https://github.com/arthurdjn/pytorch-pointcloud}, |
| } |
| ``` |
|
|
