TimSchneider42 commited on
Commit
2ce06a8
·
verified ·
1 Parent(s): 533b16e

COD-VAE 64x32

Browse files
Files changed (1) hide show
  1. README.md +114 -0
README.md ADDED
@@ -0,0 +1,114 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ library_name: cod-vae
4
+ pipeline_tag: feature-extraction
5
+ tags:
6
+ - 3d
7
+ - shape-reconstruction
8
+ - autoencoder
9
+ - vae
10
+ - occupancy
11
+ ---
12
+
13
+ # COD-VAE 64 x 32
14
+
15
+ A [COD-VAE](https://arxiv.org/abs/2503.08737) that compresses a 3D shape into
16
+ **64 latent vectors of 32 dimensions = 2048 numbers**, and decodes them back
17
+ into an occupancy field.
18
+
19
+ Trained with [`cod-vae`](https://github.com/TimSchneider42/cod-vae), a PyTorch/JAX reimplementation of COD-VAE
20
+ (Cho et al., ICCV 2025). The weights are a self-contained npz and load with either
21
+ backend.
22
+
23
+ This repository is the **default model** of the `cod-vae` package: it holds the same
24
+ weights as [`TimSchneider42/cod-vae-64x32`](https://huggingface.co/TimSchneider42/cod-vae-64x32),
25
+ the largest configuration of the grid, under the short name `TimSchneider42/cod-vae`. The Hub
26
+ has no aliasing between repositories, so the two are independent copies of one model.
27
+
28
+ Stage 1 ran the full 100 epochs and stage 2 another 100, following the
29
+ reference schedule throughout.
30
+
31
+ ## Usage
32
+
33
+ ```python
34
+ import trimesh
35
+ from cod_vae import CODVAE
36
+
37
+ vae = CODVAE.from_pretrained("TimSchneider42/cod-vae")
38
+
39
+ mesh = trimesh.load("bunny.obj", force="mesh")
40
+ latent, transform = vae.encode_mesh(mesh, return_transform=True) # (64, 32)
41
+ reconstruction = vae.decode_mesh(latent, transform=transform) # trimesh.Trimesh
42
+ ```
43
+
44
+ Latents can also be computed from raw surface point clouds and decoded at arbitrary
45
+ query points:
46
+
47
+ ```python
48
+ latents = vae.encode(points) # (N, 3) in [-1, 1]^3
49
+ logits = vae.decode(latents, queries) # occupancy logits, positive inside
50
+ volume = vae.decode_volume(latents, resolution=128) # dense logit grid
51
+ ```
52
+
53
+ Install with `pip install cod-vae[torch,hub]` (or `cod-vae[jax,hub]`).
54
+
55
+ ## Training data
56
+
57
+ A merged dataset of 110,077 shapes, built with the `cod-vae-dataset` tool:
58
+
59
+ ```bash
60
+ cod-vae-dataset data/merged --vecset path/to/shapenet_vecset_root
61
+
62
+ cod-vae-dataset data/merged \
63
+ --hf abc=TimSchneider42/tactile-mnist-abc-dataset-small:0.24435897 --hf-split train \
64
+ --num-vol 500000 --num-surface 250000
65
+
66
+ cod-vae-dataset data/merged \
67
+ --hf mnist3d=TimSchneider42/tactile-mnist-mnist3d --hf-split train \
68
+ --num-vol 50000 --num-surface 25000
69
+ ```
70
+
71
+ | source | shapes | query pools per shape |
72
+ |---|---|---|
73
+ | ShapeNet (3DShape2VecSet, 55 synsets) | 48,597 | 500k volume + 500k near-surface |
74
+ | [tactile-mnist-abc-dataset-small](https://huggingface.co/datasets/TimSchneider42/tactile-mnist-abc-dataset-small) | 50,000 | 500k + 500k |
75
+ | [tactile-mnist-mnist3d](https://huggingface.co/datasets/TimSchneider42/tactile-mnist-mnist3d) | 11,480 | 50k + 50k |
76
+
77
+ Only the training splits are used; the ABC and MNIST3D pool sizes are scaled to the
78
+ geometric complexity of each source. Meshes are preprocessed with the original authors'
79
+ [sdf_gen](https://github.com/1zb/sdf_gen) recipe.
80
+
81
+ ## Training recipe
82
+
83
+ Both stages follow the reference implementation; see
84
+ [TRAINING.md](https://github.com/TimSchneider42/cod-vae/blob/main/TRAINING.md) for the full guide and the exact commands.
85
+
86
+ | | stage 1 (autoencoder) | stage 2 (latent VAE) |
87
+ |---|---|---|
88
+ | epochs | 100 | 100 |
89
+ | batch | 32 per GPU x 2 accumulation x 4 GPUs = 256 | 128 per GPU x 4 GPUs = 512 |
90
+ | learning rate | 1e-4, scaled by effective batch / 256 | same, halved at epochs 60/70/80/90 |
91
+ | dataset repeat | 8 per epoch | 8 per epoch |
92
+ | precision | float32 with TF32 matmuls | same |
93
+
94
+ ## Held-out reconstruction quality
95
+
96
+ _Not yet measured._
97
+
98
+ Measured on the test splits of ABC and MNIST3D, which are disjoint from training.
99
+ Volume IoU compares `decode(latents, queries) > 0` against ground-truth occupancy on
100
+ uniformly sampled query points; near-surface accuracy uses points sampled around the
101
+ surface.
102
+
103
+ ## Citation
104
+
105
+ The model architecture and training recipe are from:
106
+
107
+ ```bibtex
108
+ @inproceedings{cho2025cod,
109
+ author={Cho, In and Yoo, Youngbeom and Jeon, Subin and Kim, Seon Joo},
110
+ title={Representing 3D Shapes with 64 Latent Vectors for 3D Diffusion Models},
111
+ booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
112
+ year={2025}
113
+ }
114
+ ```