---
license: other
base_model: Qwen/Qwen3-4B
language:
- en
pipeline_tag: text-to-3d
tags:
- 3d
- mesh
- text-to-3d
- spatial-pe
- 3d-bpe
- qwen3
datasets:
- Jack04810/bpe-mix-dataset
model_name: SPAC-4B
---
SPAC-4B
Sparse Proximity-Aware Compression for 3D Understanding and Generation
Code & Demo ·
Training Data
**One language model, two directions:** describe a 3D mesh in words, or generate a mesh from a text description. SPAC combines a sparse SDF VQ-VAE, geometry-aware **3D-BPE**, and a Qwen3-4B backbone with explicit spatial positional encoding.
## Why sparse 3D tokens?
- **Compute near the surface.** Encode a narrow SDF band instead of an entire dense voxel volume. Under the paper's fixed voxel-band assumptions, active inputs grow approximately with surface area, **O(R²)**.
- **Compress in 3D.** Merge recurring face-adjacent patterns using token pairs and their relative offsets. 3D-BPE is reversible at the discrete VQ-token level.
- **Keep coordinates explicit.** Serialize geometry with Morton-coordinate tokens and Spatial-PE so the language model can consume and predict 3D structure.
## From text to meshes
*Qualitative examples from the manuscript. SPAC predicts SDF geometry; colored appearances in the paper are visualization/postprocessing, while the standalone demo exports an untextured mesh.*
Mesh understanding examples from the manuscript
## Paper highlights
At an equal merge budget, 3D-BPE reduces the leaf-token sequence by approximately **24.6%**:
| Tokenizer | Remaining sequence length ↓ | Bigram entropy ↓ |
|:--|--:|--:|
| **3D-BPE** | **0.754** | **0.972** |
| 1D Morton BPE | 0.803 | 1.092 |
| 1D Hilbert BPE | 0.820 | 1.220 |
| 1D Random BPE | 0.998 | 2.167 |
See the [merge-budget sweep](https://huggingface.co/Jack04810/SPAC-4B/resolve/main/assets/bpe_sweep.png) for additional compression and dependency measurements.
*At R = 256, the manuscript reports 137.98 GFLOPs and 85.04 ms for the SPAC encoder, versus 41.38 TFLOPs and 4227.84 ms for the dense ShapeLLM encoder, under its reported protocol.*
Text-to-3D results reported in the manuscript
| Method | CLIP ↑ | FD ↓ | KID ×100 ↓ |
|:--|--:|--:|--:|
| SAR3D | 23.9 | 27.2 | 0.28 |
| TRELLIS | **30.8** | **18.3** | **0.10** |
| ShapeLLM-Omni | 26.7 | 25.9 | 0.25 |
| SPAC | 24.4 | 23.7 | 0.22 |
These figures and tables describe the **paper's experimental models and protocols**. They are not fresh benchmark measurements of the checkpoint distributed on this page. In particular, the latest Toy4k training-overlap replay is a different experiment.
## Run SPAC-4B
Requires Linux x86_64, [CUDA Toolkit 12.4](https://developer.nvidia.com/cuda-12-4-0-download-archive) (`nvcc`), a C++ compiler, Google SparseHash headers, and a compatible NVIDIA driver. An A100 40 GB is recommended. Both the LLM and VQ-VAE run in FP32.
```bash
# Ubuntu build prerequisites
sudo apt-get install -y git build-essential libsparsehash-dev
export CUDA_HOME=/usr/local/cuda-12.4 # use your CUDA 12.4 installation
export PATH="$CUDA_HOME/bin:$PATH"
git clone https://github.com/yangjj007/SPAC-main.git
cd SPAC-main
bash install.sh --dependencies-only
.venv/bin/python download_weights.py
CUDA_VISIBLE_DEVICES=0 .venv/bin/python check_model_compatibility.py --check-vae
```
TorchSparse is built from source following [Direct3D-S2](https://github.com/DreamTechAI/Direct3D-S2#installation). The installer runs the upstream clone-and-install recipe automatically, pinned to TorchSparse 2.1.0; SPAC retains its tested PyTorch 2.6.0 / torchvision 0.21.0 / CUDA 12.4 stack.
```bash
git clone https://github.com/mit-han-lab/torchsparse.git .cache/torchsparse
git -C .cache/torchsparse checkout 385f5ce8718fcae93540511b7f5832f4e71fd835
.venv/bin/python -m pip install --no-build-isolation .cache/torchsparse
```
The downloader pins a compatible release and installs/verifies the LLM, VQ-VAE, BPE merge table, and Spatial-PE manifest at their explicit runtime paths. It disables proxy use. To download through a mirror, append `--endpoint https://hf-mirror.com`. Repository authentication may be required when cloning the code.
```bash
# Text → untextured mesh, token sequence, and runtime settings
.venv/bin/python infer.py generate "A wooden barrel with metal hoops." --output outputs/barrel
# Mesh → caption, using live mesh encoding
.venv/bin/python infer.py understand examples/barrel/reference.glb --encode-mesh
# Interactive interface
CUDA_VISIBLE_DEVICES=0 bash run.sh --host 127.0.0.1 --port 7860
```
The interface opens at `http://localhost:7860`. Packaged examples can reuse their saved input tokens; `--encode-mesh` runs the complete mesh-to-SDF-to-token path.
### Runtime file mapping
| Hugging Face path | Local runtime path |
|:--|:--|
| Four `pytorch_model-*.bin` shards, index, config and tokenizer | `models/llm/` |
| `merge_table.json` | `models/merge_table.json` |
| `spatial_pe_manifest.json` | `models/spatial_pe_manifest.json` |
| `vqvae/model.pt`, `vqvae/config.json` | `models/vqvae/` |
The downloader records these mappings and SHA-256 checksums in `models/asset_manifest.json`. The VQ-VAE filename change preserves the checkpoint bytes.
### Inference defaults
| Setting | Released demo |
|:--|:--|
| Precision / attention | FP32 / SDPA; TF32 and AMP disabled |
| Spatial-PE / RoPE theta | 0.08 / 1,000,000 |
| Generation | Greedy; temperature 0, top-k 1, top-p 1; up to 8192 new tokens |
| Understanding | Greedy; up to 256 new tokens |
| Geometry | SDF resolution 256; explicit coordinate grid 32³ |
The release config declares FP32, RoPE theta 1,000,000, and independent input/output embeddings. The loader also handles the original export's stale tied-embedding flag without overwriting the learned `lm_head`. Geometry inference requires the SPAC wrapper and matching spatial manifest / merge table; standard text-only `AutoModel.generate()` does not add Spatial-PE. An output that reaches the token limit can be incomplete; the demo records `ended_normally` in `result.json`.
## Checkpoint provenance
**SPAC-4B** uses the LLM checkpoint originally released as [`Jack04810/SPAC-4B-toy4k`](https://huggingface.co/Jack04810/SPAC-4B-toy4k), revision `de08d429d2910e784b7ca0a22731246e26649c06` (`checkpoint-20352`). It is a Toy4k-continuation checkpoint exported with FP32 weights and a 200,832-token vocabulary. Renaming the release does not change its training history. The repository also provides the matching VQ-VAE under `vqvae/`.
The method is described in **“From Dense to Sparse: Morton-Ordered Sparse Mapping for Efficient 3D Mesh Generation.”** The model generates geometry without native textures or materials. License metadata is retained from the source checkpoint.