OctLLM

Octrees as an Explicit 3D Language

Ran Dan¹, Si-Tong Wei¹, Pengfei Xiong², Wei Zhang², Yadong Mu¹, Peng-Shuai Wang^1,†

¹Peking University   ²Independent Researcher
^†Corresponding author

Official model weights for Octrees as an Explicit 3D Language.

OctLLM unifies text-to-3D, image-to-3D, and 3D understanding through an explicit 3D language. It represents shapes as Sparse Octree (S-Octree) sequences: occupancy tokens anchored to 3D coordinates and octree depth. This structured representation connects geometry generation and understanding within a single model.

To learn the 3D modality, OctLLM adds trainable branches alongside a frozen Qwen2.5-VL backbone. Mesh tokens and text/image tokens follow separate processing pathways and interact through shared self-attention. Generated octrees are completed by a 3D U-Net and decoded into textured meshes with pretrained TRELLIS decoders. The video below demonstrates generation, 3D understanding, and language interaction.

This repository contains the OctLLM checkpoint and occupancy completion weights. Use the official implementation for installation and inference; its loading code is required for the custom 3D components. More results are available on the project page.