| --- |
| license: apache-2.0 |
| pipeline_tag: image-to-3d |
| base_model: Lucasgitguy-co/forma3d-tiny-3 |
| base_model_relation: finetune |
| model_name: Forma3D Tiny 4 |
| datasets: |
| - codepawl/abo-glb |
| tags: |
| - pytorch |
| - gaussian-splatting |
| - mesh-proxy-supervision |
| - research-prototype |
| --- |
| # Forma3D Tiny 4 |
|
|
| A 306,321-parameter image-to-Gaussian research model continued from |
| [Forma3D Tiny 3](https://huggingface.co/Lucasgitguy-co/forma3d-tiny-3) at |
| `3469f527c5f6833d654f62653ee92e0f73854b33`. The architecture remains a 16³ grid, |
| width 64 and at most 4,096 Gaussian slots. Trained locally on an M4 MacBook Air |
| with 16 GB unified memory using PyTorch MPS, float32 and batch size 8. |
|
|
| ## Use |
|
|
| Clone and install with Python 3.12. Git LFS is needed to fetch the checkpoint: |
|
|
| ```bash |
| git lfs install |
| git clone https://huggingface.co/Lucasgitguy-co/forma3d-tiny-4 |
| cd forma3d-tiny-4 |
| python3.12 -m venv .venv |
| source .venv/bin/activate |
| pip install . |
| image3d generate --checkpoint model.pt --image examples/abo-B071FJR479-v3-input.png --out object.ply --device mps --threshold 0.6 |
| image3d preview --ply object.ply --out preview.png |
| ``` |
|
|
| Use `--device cpu` on systems without Metal. Replace the sample input with an actual |
| image path to try other inputs. The recommended threshold is **0.6**; the inherited |
| CLI defaults to 0.5, so pass the flag explicitly. `generation_config.json` is |
| informational and is not automatically read by the CLI. Output is a Gaussian PLY, |
| not a textured mesh. No text prompting is supported. |
|
|
| ## Training and selection |
|
|
| The expanded dataset contains 1,024 ABO objects across all splits, not 1,024 training |
| objects. Training uses 677 objects with 12 views each (8,124 inputs): four azimuths at |
| three elevations. The original 512-object split assignments are preserved; duplicate |
| identities are grouped before new objects are assigned. The common validation set |
| has 87 objects / 348 views. An additional 100 objects / 400 views are reserved for |
| final confirmation. The historical test set is excluded from tuning. |
|
|
| Three variants start from the same v3 checkpoint: original data, 12-view data, |
| and expanded-object data. Each receives 6,000 AdamW steps at learning rate 5e-5. |
| Selection maximizes validation silhouette IoU subject to nonworsening occupancy IoU |
| and Chamfer distance, with thresholds 0.3, 0.4, 0.5 and 0.6 tuned on validation. |
| The expanded-object variant won. `model.pt` is seed 44, step 6000; `repeat/model.pt` |
| is its independently trained seed-45 confirmation. The initial winner was fixed |
| before inspecting final confirmation results. |
|
|
| ## Held-out confirmation results |
|
|
| All rows use the same 100 held-out objects / 400 proxy views. Higher IoU and lower |
| Chamfer distance are better. The first two rows isolate the checkpoint change at |
| threshold 0.5; the third includes the validation-selected threshold change. |
|
|
| | Checkpoint | Threshold | Silhouette IoU | Occupancy IoU | Chamfer distance | |
| |---|---:|---:|---:|---:| |
| | Original v3 | 0.5 | 0.463688 | 0.337398 | 0.069585 | |
| | Tiny 4, seed 44 | 0.5 | 0.512971 | 0.424836 | 0.053612 | |
| | Tiny 4, seed 44 (recommended) | 0.6 | 0.540550 | 0.434657 | 0.052380 | |
| | Tiny 4, seed 45 (repeat) | 0.6 | 0.546573 | 0.412526 | 0.054105 | |
|
|
| For recommended seed 44 versus original v3 at 0.5, paired object-cluster bootstrap |
| 95% intervals (2,000 resamples) for candidate-minus-baseline changes are: |
| silhouette IoU **+0.07686 [0.06621, 0.08926]**, occupancy IoU |
| **+0.09726 [0.07289, 0.12272]**, and Chamfer |
| **−0.01721 [−0.03762, −0.00622]**. These intervals include the threshold change. |
| Second-seed intervals also exclude zero in the favorable direction for all three |
| metrics. Detailed scores and per-example results are in `evaluation/` and |
| `evaluation.json`; splits and source metadata are in `splits/`. |
|
|
| The 15-minute batch-8 training benchmark measured 650.02 samples/s; this excludes |
| preparation and validation overhead. It recorded about 496 MiB peak process RSS |
| and 141 MiB MPS driver allocation; these overlap in unified memory and are not a |
| measurement of total system memory. CPU/MPS loss and gradient parity passed. |
|
|
| ## Examples and limitations |
|
|
| Each turntable shows the predicted Gaussian preview on the left and a reference |
| mesh silhouette on the right. The six examples include the two lowest, two median |
| and two highest silhouette-IoU views in the confirmation set; they are not random |
| samples. The reference image is a silhouette, not a photorealistic rendering. |
|
|
|  |
|
|
|  |
|
|
|  |
|
|
|  |
|
|
|  |
|
|
|  |
|
|
| Inputs are orthographic Gaussian-proxy renders on black. Natural photographs, |
| photorealistic rendering, arbitrary viewpoints, thin structures and hidden surfaces |
| remain unvalidated. Centering an ordinary photo does not make it an in-domain input. |
| Chamfer measures symmetric mean Euclidean distance between predicted Gaussian centers |
| and mesh-surface samples in normalized coordinates, not reconstructed mesh quality. |
| Silhouettes compare alpha>=0.5 splat rasterization with triangle rasterization at |
| 96 pixels from four fixed unseen viewpoints. Near duplicates may remain; ancestor |
| training exposure is not fully known. These measurements do not establish broad |
| real-world reconstruction quality. |
|
|
| ## Reproduction and attribution |
|
|
| `recipe/` contains the training/evaluation implementation and dependency lock. |
| Original source and weight lineage is Apache 2.0; retain LICENSE, NOTICE and |
| THIRD_PARTY.md. ABO assets and derived example imagery are credited to Amazon.com |
| and Amazon Berkeley Objects contributors under CC BY 4.0, with GLB conversion by |
| CodePawl. Geometry was normalized, sampled and rendered into proxy inputs and |
| silhouettes. See ATTRIBUTIONS.json, attribution/INHERITED.json, |
| licenses/LICENSE-ABO-CC-BY-4.0.txt and the |
| [dataset](https://huggingface.co/datasets/codepawl/abo-glb). |
| Raw GLB meshes and the full training image cache are not bundled. |
| |
