Instructions to use SUSSYEATING/GLM-Image-ConvRot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use SUSSYEATING/GLM-Image-ConvRot with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("SUSSYEATING/GLM-Image-ConvRot", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- Draw Things
- DiffusionBee
|
Download README.md from SUSSYEATING/GLM-Image-ConvRot: direct link, hf CLI and curl.
- Browser
- Download file 5.06 kB
-
https://huggingface.co/SUSSYEATING/GLM-Image-ConvRot/resolve/main/README.md
- Command line
-
hf download hf://SUSSYEATING/GLM-Image-ConvRot/README.md
-
curl -L -o README.md https://huggingface.co/SUSSYEATING/GLM-Image-ConvRot/resolve/main/README.md
5.06 kB
| license: mit | |
| language: | |
| - zh | |
| - en | |
| library_name: diffusers | |
| pipeline_tag: text-to-image | |
| base_model: zai-org/GLM-Image | |
| tags: | |
| - glm-image | |
| - convrot | |
| - quantization | |
| - int8 | |
| - int6 | |
| - int4 | |
| - comfy-kitchen | |
| # GLM-Image ConvRot (`INT8`, `INT6 w6w8`, `INT4 w4w8`) | |
| High-fidelity **ConvRot** quantized checkpoints for [**zai-org/GLM-Image**](https://huggingface.co/zai-org/GLM-Image) (hybrid **9B Autoregressive VLM** + **7B Diffusion Decoder DiT**), powered by [`comfy-kitchen`](https://github.com/Comfy-Org/comfy-kitchen) INT8 / W6A8 / W4A8 Tensor Core kernels. | |
| - **7B DiT (`transformer`)**: All `180` attention/FFN block linears quantized with ConvRot-256 (`int8_tensorwise`, `w6a8_int8`, `asym_w4a8_int8`) + `30` AdaLN modulation layers (`norm1.linear`) kept at `INT8 ConvRot-256` across all tiers; `prior_token_embedding`, `glyph_projector`, `image_projector`, `time_condition_embed`, `norm_out`, and `proj_out` remain BF16. | |
| - **9B AR VLM (`vision_language_encoder`)**: All `40` AR LLM layers quantized with SmoothQuant + ConvRot-256 (`q_proj`, `o_proj`, `mlp.gate_up_proj`) and **`INT8 ConvRot-64`** on `mlp.down_proj` (`K = 13696`, divisible by 64) for `w6w8` / `w4w8` compatibility; `40` ViT blocks quantized to `INT8 ConvRot-256`; `k_proj`, `v_proj` (`0.17 GB` total), `embed_tokens`, `lm_head`, and `vqmodel` remain BF16 to preserve KV-cache precision and discrete visual-token accuracy. | |
| - **Typography & VAE (`text_encoder` ByT5 + `vae`)**: Loaded unquantized directly from `zai-org/GLM-Image` (`1.68 GB` combined) for crisp text rendering and pixel fidelity. | |
| --- | |
| ## Verified Benchmarks (`1024x1024`, 50 DiT steps, `guidance_scale=1.5`) | |
| | Tier | DiT + VLM Size | Peak VRAM | DiT Warm Time | DiT-Only PSNR (`cabin / poster / watch`) | DiT-Only SSIM | VLM Teacher-Forced Top-1 / KL | | |
| | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | |
| | **BF16 Baseline** | `34.08 GB` (`13.85 + 20.23`) | `40.55 GB` | `21.60s` | Ref | `1.0000` | `100.0%` / `0.000` | | |
| | **INT8 ConvRot** | **`18.10 GB`** (`7.07 + 11.03`, `-46.9%`) | **`24.59 GB`** | **`17.21s`** (**`1.26x`**) | **`47.21 / 42.83 / 38.65 dB`** | **`0.9982 / 0.9965 / 0.9957`** | **`89.71%`** / `0.081` | | |
| | **INT6 (`w6w8`) ConvRot** | **`15.88 GB`** (`5.94 + 9.94`, `-53.4%`) | **`22.31 GB`** | **`17.95s`** (**`1.20x`**) | **`41.74 / 39.73 / 36.68 dB`** | **`0.9966 / 0.9941 / 0.9933`** | **`89.19%`** / `0.082` | | |
| | **INT4 (`w4w8`) ConvRot** | **`12.91 GB`** (`4.43 + 8.48`, `-62.1%`) | **`19.40 GB`** | **`17.80s`** (**`1.21x`**) | **`30.44 / 30.55 / 26.37 dB`** | **`0.9723 / 0.9650 / 0.9608`** | **`79.62%`** / `0.316` | | |
| > **Note:** In end-to-end sampling, GLM-Image's AR VLM autoregressively samples 4,096 discrete visual tokens, so small logit shifts produce different valid compositions of the same prompt. Both **DiT-isolated** reconstruction (identical BF16 prior tokens -> quantized DiT) and **VLM teacher-forced** visual-token agreement are reported above alongside full end-to-end visual comparisons below. | |
| --- | |
| ## Visual Comparisons (`BF16 | INT8 | INT6 w6w8 | INT4 w4w8`) | |
| ### End-to-End Photorealistic Scene (`t2i_cabin_s42`) | |
|  | |
| ### End-to-End Typography Poster (`t2i_poster_s1337`) | |
|  | |
| ### End-to-End Macro Detail (`t2i_watch_s777`) | |
|  | |
| ### DiT-Isolated Reconstruction Fidelity (`t2i_cabin_s42`, Same BF16 Prior Tokens) | |
|  | |
| --- | |
| ## Quickstart (`diffusers` + `comfy-kitchen`) | |
| ```bash | |
| pip install comfy-kitchen>=0.2.41 diffusers transformers safetensors accelerate | |
| ``` | |
| ```python | |
| import torch | |
| from huggingface_hub import hf_hub_download, snapshot_download | |
| from glm_image_convrot import load_glm_image_convrot | |
| base_model = snapshot_download("zai-org/GLM-Image") | |
| tier = "int6" # "int8", "int6", or "int4" | |
| transformer_path = hf_hub_download( | |
| "SUSSYEATING/GLM-Image-ConvRot", | |
| f"{tier}/glm-image-transformer-{tier}-convrot.safetensors", | |
| ) | |
| vlm_path = hf_hub_download( | |
| "SUSSYEATING/GLM-Image-ConvRot", | |
| f"{tier}/glm-image-vlm-{tier}-convrot.safetensors", | |
| ) | |
| pipe, info = load_glm_image_convrot( | |
| base_model=base_model, | |
| transformer_path=transformer_path, | |
| vlm_path=vlm_path, | |
| device="cuda", | |
| dtype=torch.bfloat16, | |
| ) | |
| image = pipe( | |
| prompt="A vintage travel poster for 'ALPS EXPRESS' with bold cream typography, a red steam train crossing a stone viaduct over snowy mountains, 'SWITZERLAND 1964'", | |
| height=1024, | |
| width=1024, | |
| num_inference_steps=50, | |
| guidance_scale=1.5, | |
| generator=torch.Generator(device="cuda").manual_seed(1337), | |
| ).images[0] | |
| image.save("glm_convrot_out.png") | |
| ``` | |
| --- | |
| ## License & Attribution | |
| - Derived from [**zai-org/GLM-Image**](https://huggingface.co/zai-org/GLM-Image), released under the **MIT License**. | |
| - As noted upstream, the incorporated VQ tokenizer and ViT weights from [**X-Omni/X-Omni-En**](https://huggingface.co/X-Omni/X-Omni-En) remain subject to the **Apache License, Version 2.0**. | |