Point the model card at the GitHub repo instead of local usage commands
Browse filesDrop the inference snippet so usage stays in the code repository. Mention the 4B image-to-3D checkpoint.
README.md
CHANGED
|
@@ -1,6 +1,5 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
-
library_name: transformers
|
| 4 |
pipeline_tag: image-to-3d
|
| 5 |
tags:
|
| 6 |
- 3d
|
|
@@ -23,62 +22,35 @@ papers:
|
|
| 23 |
|
| 24 |
**CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models** (ICML 2026)
|
| 25 |
|
| 26 |
-
CG-MLLM is a 3D multimodal large language model
|
| 27 |
|
| 28 |
-
|
|
|
|
|
|
|
| 29 |
|
| 30 |
## Links
|
| 31 |
|
|
|
|
| 32 |
- **Paper (arXiv):** https://arxiv.org/abs/2601.21798
|
| 33 |
- **Hugging Face Papers:** https://huggingface.co/papers/2601.21798
|
| 34 |
- **Project page:** https://cv.jream.top/CG-MLLM-page/
|
| 35 |
-
- **Code:** https://github.com/dreaming-huang/CG-MLLM
|
| 36 |
- **ICML poster:** https://icml.cc/virtual/2026/poster/63909
|
| 37 |
|
| 38 |
## Model Details
|
| 39 |
|
| 40 |
| Item | Value |
|
| 41 |
| --- | --- |
|
| 42 |
-
| Version | v0.1 |
|
| 43 |
-
| Architecture | Mixture-of-Transformer (TokenAR + BlockAR) |
|
| 44 |
| Vision-language backbone | Qwen3-VL-2B-Instruct |
|
| 45 |
| 3D latent tokenizer | Hunyuan3D-2.1 VAE |
|
| 46 |
| Tasks | Image-to-3D, text-to-3D, image understanding, 3D understanding |
|
|
|
|
| 47 |
| Venue | ICML 2026 |
|
| 48 |
| License | Apache-2.0 |
|
| 49 |
| Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu |
|
| 50 |
| Affiliations | Zhejiang University; LIGHTSPEED |
|
| 51 |
|
| 52 |
-
|
| 53 |
-
|
| 54 |
-
| File | Description |
|
| 55 |
-
| --- | --- |
|
| 56 |
-
| `ema.safetensors` | EMA checkpoint weights for CG-MLLM v0.1 |
|
| 57 |
-
|
| 58 |
-
## How to Use
|
| 59 |
-
|
| 60 |
-
Install and run inference from the [official code repository](https://github.com/dreaming-huang/CG-MLLM):
|
| 61 |
-
|
| 62 |
-
```bash
|
| 63 |
-
git clone https://github.com/dreaming-huang/CG-MLLM.git
|
| 64 |
-
cd CG-MLLM
|
| 65 |
-
# follow Installation in the GitHub README, then:
|
| 66 |
-
hf download JreamH/CGMLLM ema.safetensors --local-dir models/CGMLLM
|
| 67 |
-
```
|
| 68 |
-
|
| 69 |
-
Example (image-to-3D):
|
| 70 |
-
|
| 71 |
-
```bash
|
| 72 |
-
python inference.py \
|
| 73 |
-
--llm_base_path Qwen/Qwen3-VL-2B-Instruct \
|
| 74 |
-
--checkpoint models/CGMLLM \
|
| 75 |
-
--obj_vae_path tencent/Hunyuan3D-2.1 \
|
| 76 |
-
--obj_vae_len 4096 \
|
| 77 |
-
--use_qwen_vit --use_qwen_vl --qk_norm \
|
| 78 |
-
--mode i2obj --image examples/chairo.png
|
| 79 |
-
```
|
| 80 |
-
|
| 81 |
-
See the GitHub README for text-to-3D, image understanding, and 3D understanding examples.
|
| 82 |
|
| 83 |
## Citation
|
| 84 |
|
|
@@ -95,4 +67,4 @@ If you find this work useful, please cite:
|
|
| 95 |
|
| 96 |
## Acknowledgments
|
| 97 |
|
| 98 |
-
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
| 3 |
pipeline_tag: image-to-3d
|
| 4 |
tags:
|
| 5 |
- 3d
|
|
|
|
| 22 |
|
| 23 |
**CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models** (ICML 2026)
|
| 24 |
|
| 25 |
+
CG-MLLM is a 3D multimodal large language model built on a **Qwen3-VL** backbone and a Hunyuan3D-2.1 VAE, for 3D captioning and 3D generation.
|
| 26 |
|
| 27 |
+
This repository is the **v0.1 multi-task** checkpoint, on **Qwen3-VL-2B-Instruct**. It covers image-to-3D, text-to-3D, image understanding, and 3D understanding.
|
| 28 |
+
|
| 29 |
+
The **4B image-to-3D** checkpoint (Qwen3-VL-4B, specialized on HY3D-Bench) is at [JreamH/CGMLLM-4B-i2o](https://huggingface.co/JreamH/CGMLLM-4B-i2o).
|
| 30 |
|
| 31 |
## Links
|
| 32 |
|
| 33 |
+
- **Code (how to run):** https://github.com/dreaming-huang/CG-MLLM
|
| 34 |
- **Paper (arXiv):** https://arxiv.org/abs/2601.21798
|
| 35 |
- **Hugging Face Papers:** https://huggingface.co/papers/2601.21798
|
| 36 |
- **Project page:** https://cv.jream.top/CG-MLLM-page/
|
|
|
|
| 37 |
- **ICML poster:** https://icml.cc/virtual/2026/poster/63909
|
| 38 |
|
| 39 |
## Model Details
|
| 40 |
|
| 41 |
| Item | Value |
|
| 42 |
| --- | --- |
|
| 43 |
+
| Version | v0.1, multi-task |
|
|
|
|
| 44 |
| Vision-language backbone | Qwen3-VL-2B-Instruct |
|
| 45 |
| 3D latent tokenizer | Hunyuan3D-2.1 VAE |
|
| 46 |
| Tasks | Image-to-3D, text-to-3D, image understanding, 3D understanding |
|
| 47 |
+
| Checkpoint file | `ema.safetensors` |
|
| 48 |
| Venue | ICML 2026 |
|
| 49 |
| License | Apache-2.0 |
|
| 50 |
| Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu |
|
| 51 |
| Affiliations | Zhejiang University; LIGHTSPEED |
|
| 52 |
|
| 53 |
+
Usage is documented in the [GitHub repository](https://github.com/dreaming-huang/CG-MLLM). This card does not duplicate the inference commands.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 54 |
|
| 55 |
## Citation
|
| 56 |
|
|
|
|
| 67 |
|
| 68 |
## Acknowledgments
|
| 69 |
|
| 70 |
+
This work builds on [Qwen3-VL](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct), [BAGEL](https://github.com/ByteDance-Seed/Bagel), and the [Hunyuan3D-2.1](https://github.com/Tencent-Hunyuan/Hunyuan3D-2.1) shape VAE. Please follow their licenses when using those components.
|