| --- |
| license: apache-2.0 |
| pipeline_tag: image-to-3d |
| tags: |
| - 3d |
| - multimodal |
| - mllm |
| - qwen3-vl |
| - hunyuan3d |
| - mesh-generation |
| - 3d-captioning |
| - image-to-3d |
| - text-to-3d |
| - icml-2026 |
| language: |
| - en |
| papers: |
| - https://huggingface.co/papers/2601.21798 |
| --- |
| |
| # CG-MLLM (v0.1) |
|
|
| **CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models** (ICML 2026) |
|
|
| CG-MLLM is a 3D multimodal large language model built on a **Qwen3-VL** backbone and a Hunyuan3D-2.1 VAE, for 3D captioning and 3D generation. |
|
|
| This repository is the **v0.1 multi-task** checkpoint, on **Qwen3-VL-2B-Instruct**. It covers image-to-3D, text-to-3D, image understanding, and 3D understanding. |
|
|
| The **4B image-to-3D** checkpoint (Qwen3-VL-4B, specialized on HY3D-Bench) is at [JreamH/CGMLLM-4B-i2o](https://huggingface.co/JreamH/CGMLLM-4B-i2o). |
|
|
| ## Links |
|
|
| - **Code (how to run):** https://github.com/dreaming-huang/CG-MLLM |
| - **Paper (arXiv):** https://arxiv.org/abs/2601.21798 |
| - **Hugging Face Papers:** https://huggingface.co/papers/2601.21798 |
| - **Project page:** https://cv.jream.top/CG-MLLM-page/ |
| - **ICML poster:** https://icml.cc/virtual/2026/poster/63909 |
|
|
| ## Model Details |
|
|
| | Item | Value | |
| | --- | --- | |
| | Version | v0.1, multi-task | |
| | Vision-language backbone | Qwen3-VL-2B-Instruct | |
| | 3D latent tokenizer | Hunyuan3D-2.1 VAE | |
| | Tasks | Image-to-3D, text-to-3D, image understanding, 3D understanding | |
| | Checkpoint file | `ema.safetensors` | |
| | Venue | ICML 2026 | |
| | License | Apache-2.0 | |
| | Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu | |
| | Affiliations | Zhejiang University; LIGHTSPEED | |
|
|
| Usage is documented in the [GitHub repository](https://github.com/dreaming-huang/CG-MLLM). This card does not duplicate the inference commands. |
|
|
| ## Citation |
|
|
| If you find this work useful, please cite: |
|
|
| ```bibtex |
| @article{huang2026cg, |
| title={CG-MLLM: Captioning and Generating 3D content via Multi-modal Large Language Models}, |
| author={Huang, Junming and Wang, Chi and Li, Letian and Xu, Guangkai and Huang, Donglin and Chen, Hao and Dai, Qiang and Xu, Weiwei}, |
| journal={arXiv preprint arXiv:2601.21798}, |
| year={2026} |
| } |
| ``` |
|
|
| ## Acknowledgments |
|
|
| This work builds on [Qwen3-VL](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct), [BAGEL](https://github.com/ByteDance-Seed/Bagel), and the [Hunyuan3D-2.1](https://github.com/Tencent-Hunyuan/Hunyuan3D-2.1) shape VAE. Please follow their licenses when using those components. |
|
|
