JreamH commited on
Commit
ba45c47
·
verified ·
1 Parent(s): adbbf7a

Point the model card at the GitHub repo instead of local usage commands

Browse files

Drop the inference snippet so usage stays in the code repository. Mention the 4B image-to-3D checkpoint.

Files changed (1) hide show
  1. README.md +9 -37
README.md CHANGED
@@ -1,6 +1,5 @@
1
  ---
2
  license: apache-2.0
3
- library_name: transformers
4
  pipeline_tag: image-to-3d
5
  tags:
6
  - 3d
@@ -23,62 +22,35 @@ papers:
23
 
24
  **CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models** (ICML 2026)
25
 
26
- CG-MLLM is a 3D multimodal large language model (3D MLLM) built upon Qwen3-VL and Hunyuan3D-2.1 VAE for unified **3D understanding**, **3D captioning**, and **high-resolution 3D content generation**.
27
 
28
- It uses a Mixture-of-Transformer design: a TokenAR Transformer for token-level content and a BlockAR Transformer for block-level 3D latents, enabling long-context interaction between standard tokens and spatial blocks in one architecture.
 
 
29
 
30
  ## Links
31
 
 
32
  - **Paper (arXiv):** https://arxiv.org/abs/2601.21798
33
  - **Hugging Face Papers:** https://huggingface.co/papers/2601.21798
34
  - **Project page:** https://cv.jream.top/CG-MLLM-page/
35
- - **Code:** https://github.com/dreaming-huang/CG-MLLM
36
  - **ICML poster:** https://icml.cc/virtual/2026/poster/63909
37
 
38
  ## Model Details
39
 
40
  | Item | Value |
41
  | --- | --- |
42
- | Version | v0.1 |
43
- | Architecture | Mixture-of-Transformer (TokenAR + BlockAR) |
44
  | Vision-language backbone | Qwen3-VL-2B-Instruct |
45
  | 3D latent tokenizer | Hunyuan3D-2.1 VAE |
46
  | Tasks | Image-to-3D, text-to-3D, image understanding, 3D understanding |
 
47
  | Venue | ICML 2026 |
48
  | License | Apache-2.0 |
49
  | Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu |
50
  | Affiliations | Zhejiang University; LIGHTSPEED |
51
 
52
- ## Repository Contents
53
-
54
- | File | Description |
55
- | --- | --- |
56
- | `ema.safetensors` | EMA checkpoint weights for CG-MLLM v0.1 |
57
-
58
- ## How to Use
59
-
60
- Install and run inference from the [official code repository](https://github.com/dreaming-huang/CG-MLLM):
61
-
62
- ```bash
63
- git clone https://github.com/dreaming-huang/CG-MLLM.git
64
- cd CG-MLLM
65
- # follow Installation in the GitHub README, then:
66
- hf download JreamH/CGMLLM ema.safetensors --local-dir models/CGMLLM
67
- ```
68
-
69
- Example (image-to-3D):
70
-
71
- ```bash
72
- python inference.py \
73
- --llm_base_path Qwen/Qwen3-VL-2B-Instruct \
74
- --checkpoint models/CGMLLM \
75
- --obj_vae_path tencent/Hunyuan3D-2.1 \
76
- --obj_vae_len 4096 \
77
- --use_qwen_vit --use_qwen_vl --qk_norm \
78
- --mode i2obj --image examples/chairo.png
79
- ```
80
-
81
- See the GitHub README for text-to-3D, image understanding, and 3D understanding examples.
82
 
83
  ## Citation
84
 
@@ -95,4 +67,4 @@ If you find this work useful, please cite:
95
 
96
  ## Acknowledgments
97
 
98
- Inference code builds on [BAGEL](https://github.com/ByteDance-Seed/Bagel) and the [Hunyuan3D-2.1](https://github.com/Tencent-Hunyuan/Hunyuan3D-2.1) shape VAE. Please follow their licenses when using those components.
 
1
  ---
2
  license: apache-2.0
 
3
  pipeline_tag: image-to-3d
4
  tags:
5
  - 3d
 
22
 
23
  **CG-MLLM: Captioning and Generating 3D Content via Multi-modal Large Language Models** (ICML 2026)
24
 
25
+ CG-MLLM is a 3D multimodal large language model built on a **Qwen3-VL** backbone and a Hunyuan3D-2.1 VAE, for 3D captioning and 3D generation.
26
 
27
+ This repository is the **v0.1 multi-task** checkpoint, on **Qwen3-VL-2B-Instruct**. It covers image-to-3D, text-to-3D, image understanding, and 3D understanding.
28
+
29
+ The **4B image-to-3D** checkpoint (Qwen3-VL-4B, specialized on HY3D-Bench) is at [JreamH/CGMLLM-4B-i2o](https://huggingface.co/JreamH/CGMLLM-4B-i2o).
30
 
31
  ## Links
32
 
33
+ - **Code (how to run):** https://github.com/dreaming-huang/CG-MLLM
34
  - **Paper (arXiv):** https://arxiv.org/abs/2601.21798
35
  - **Hugging Face Papers:** https://huggingface.co/papers/2601.21798
36
  - **Project page:** https://cv.jream.top/CG-MLLM-page/
 
37
  - **ICML poster:** https://icml.cc/virtual/2026/poster/63909
38
 
39
  ## Model Details
40
 
41
  | Item | Value |
42
  | --- | --- |
43
+ | Version | v0.1, multi-task |
 
44
  | Vision-language backbone | Qwen3-VL-2B-Instruct |
45
  | 3D latent tokenizer | Hunyuan3D-2.1 VAE |
46
  | Tasks | Image-to-3D, text-to-3D, image understanding, 3D understanding |
47
+ | Checkpoint file | `ema.safetensors` |
48
  | Venue | ICML 2026 |
49
  | License | Apache-2.0 |
50
  | Authors | Junming Huang, Chi Wang, Letian Li, Guangkai Xu, Donglin Huang, Hao Chen, Qiang Dai, Weiwei Xu |
51
  | Affiliations | Zhejiang University; LIGHTSPEED |
52
 
53
+ Usage is documented in the [GitHub repository](https://github.com/dreaming-huang/CG-MLLM). This card does not duplicate the inference commands.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
54
 
55
  ## Citation
56
 
 
67
 
68
  ## Acknowledgments
69
 
70
+ This work builds on [Qwen3-VL](https://huggingface.co/Qwen/Qwen3-VL-2B-Instruct), [BAGEL](https://github.com/ByteDance-Seed/Bagel), and the [Hunyuan3D-2.1](https://github.com/Tencent-Hunyuan/Hunyuan3D-2.1) shape VAE. Please follow their licenses when using those components.