Instructions to use SUSSYEATING/BAGEL-7B-MoT-ConvRot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusion Single File
How to use SUSSYEATING/BAGEL-7B-MoT-ConvRot with Diffusion Single File:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
BAGEL-7B-MoT ConvRot (INT8, W6/W8 Hybrid & INT6) for ComfyUI
Quantized single-file checkpoints of ByteDance-Seed/BAGEL-7B-MoT using ConvRot (block Hadamard rotation + row-wise symmetric quantization) for ComfyUI, cutting peak inference VRAM from 31.32 GB (BF16) down to 17.17β18.19 GB (fitting comfortably on a single 24 GB GPU) while preserving multimodal Text-to-Image, Thinking-guided Generation, Image Editing, and Image Understanding.
This repository also bundles the patched ComfyUI-BAGEL custom node (both extracted folder ComfyUI-BAGEL/ and reproducible archive ComfyUI-BAGEL-a1a19a40e619.tar.gz) plus a lossless unpack script scripts/unpack_int6.py for the bit-packed INT6 downloads.
Available Checkpoints
| File | Variant | Quantized Layers | File Size | ComfyUI Ready? | Notes |
|---|---|---|---|---|---|
bagel-7b-mot-int8-convrot.safetensors |
INT8 ConvRot (Recommended) | 392 / 392 INT8 | 16.18 GB | Yes (Direct) | Closest quality to BF16 across all tasks |
int6_experiments/bagel-7b-mot-w6w8-hybrid-convrot.safetensors |
W6/W8 Hybrid ConvRot | 288 INT6 + 104 INT8 | 16.18 GB | Yes (Direct) | Keeps layers.0,1,26,27 & o_proj in INT8; stored in INT8 container |
int6_experiments/bagel-7b-mot-w6w8-hybrid-convrot-packed.safetensors |
W6/W8 Hybrid ConvRot (Packed) | 288 INT6 + 104 INT8 | 13.53 GB | Unpack first | Smaller download; unpack via scripts/unpack_int6.py |
int6_experiments/bagel-7b-mot-w6a8-int6-convrot.safetensors |
W6A8 INT6 ConvRot | 392 / 392 INT6 ([-31, 31]) |
16.18 GB | Yes (Direct) | Experimental all-INT6 weights in ComfyUI INT8 container |
int6_experiments/bagel-7b-mot-int6-convrot-packed.safetensors |
INT6 ConvRot (Packed) | 392 / 392 INT6 | 12.91 GB | Unpack first | Smallest file (4Γ6-bit values bit-packed into 3 uint8 bytes) |
Why do
*-w6a8-*/*-w6w8-hybrid-*files still measure 16.18 GB on disk? Stock ComfyUI / PyTorchTensorCoreINT8Layoutstores 6-bit constrained weights ([-31, 31]) inside standardtorch.int8tensors so they load with zero custom CUDA kernel changes. Download the bit-packed*-packed.safetensorsfiles to save bandwidth and unpack them locally withscripts/unpack_int6.py.
Visual Comparisons (BF16 Baseline vs INT8 ConvRot vs W6/W8 Hybrid ConvRot vs W6A8 INT6 ConvRot)
All outputs below were rendered on the same ComfyUI v0.45.0 pipeline (1024Γ1024, 50 steps, cfg_text_scale=4.0, timestep_shift=3.0).
Text-to-Image (No Thinking)
Text-to-Image with <think> Chain-of-Thought
Honest Note on INT6 / Hybrid Thinking Mode: Notice in
t2i_think_s42:
- INT8 ConvRot closely matches the scene coherence of BF16 (
<think>rewrites the prompt slightly differently, producing a slightly different framing with crisp object structure).- All-INT6 (
W6A8) showed occasional composition artifacts under<think>(e.g., an extra small floating robot head). Keeping the 104 boundary/o_projlayers in INT8 (W6/W8 Hybrid ConvRot) eliminated the floating head artifact, though some composition merge still occurred between the watering pot and robot arm. For production multimodal generation and<think>mode, INT8 ConvRot is strongly recommended.
Image Editing (Using Matched BF16 Input Image)
Evaluation Metrics (G4 GPU, ComfyUI v0.45.0)
Measured via scripts/eval_bagel_g4.py comparing each quantized checkpoint directly against the unquantized bagel-7b-mot.safetensors (BF16) baseline (stats.json files archived in metrics/):
| Variant | Peak VRAM (T2I / Edit) | t2i_cabin_s42 (Time / Cos / SSIM / PSNR) |
t2i_watch_s1337 (Time / Cos / SSIM / PSNR) |
t2i_think_s42 (Time / Cos / SSIM / PSNR) |
edit_cabin_winter_s42 (Time / Cos / SSIM / PSNR) |
understand_cabin Time |
|---|---|---|---|---|---|---|
| BF16 Baseline | 30.30 / 31.32 GB | 36.38s / ref | 31.43s / ref | 34.55s / ref | 40.32s / ref | 1.58s |
| INT8 ConvRot (392/392) | 17.17 / 18.19 GB | 27.75s / 0.97786 / 0.89359 / 24.42 dB | 25.78s / 0.98396 / 0.92897 / 24.13 dB | 33.92s / 0.52064 / 0.49434 / 9.72 dB | 33.79s / 0.99582 / 0.95865 / 32.08 dB | 1.53s |
| W6/W8 Hybrid ConvRot | 17.18 / 18.19 GB | 28.20s / 0.95340 / 0.80723 / 21.93 dB | 25.77s / 0.96230 / 0.87035 / 20.82 dB | 34.13s / 0.51049 / 0.48816 / 9.59 dB | 33.78s / 0.99503 / 0.95232 / 31.08 dB | 1.85s |
| W6A8 INT6 ConvRot (392/392) | 17.18 / 18.19 GB | 28.16s / 0.95483 / 0.81805 / 22.08 dB | 25.77s / 0.93379 / 0.83128 / 18.12 dB | 35.90s / 0.54385 / 0.49905 / 10.03 dB | 33.77s / 0.99477 / 0.95100 / 30.62 dB | 1.85s |
(Note: For t2i_think_s42, <think> generates autoregressive reasoning tokens before image diffusion, so a slightly differently phrased <think> output naturally shifts pixel layout metrics while editing (edit_cabin_winter_s42) achieves 0.99582 Cosine / 0.95865 SSIM / 32.08 dB PSNR with INT8 ConvRot and 0.99503 / 0.95232 / 31.08 dB with W6/W8 Hybrid ConvRot.)
ComfyUI Setup & Usage
- Install the patched
ComfyUI-BAGELcustom node: CopyComfyUI-BAGEL/(or extractComfyUI-BAGEL-a1a19a40e619.tar.gz) into:ComfyUI/custom_nodes/ComfyUI-BAGEL/ - Download a checkpoint into
ComfyUI/models/bagel/:- Direct plug-and-play (recommended): Download
bagel-7b-mot-int8-convrot.safetensorsstraight intoComfyUI/models/bagel/. - Packed INT6 download: Download a
*-packed.safetensorsfile and unpack it losslessly before loading in ComfyUI:python scripts/unpack_int6.py bagel-7b-mot-w6w8-hybrid-convrot-packed.safetensors ComfyUI/models/bagel/bagel-7b-mot-w6w8-hybrid-convrot.safetensors
- Direct plug-and-play (recommended): Download
- Ensure BAGEL's VAE (
ae.safetensors) is placed inComfyUI/models/vae/. - Hardware note: Requires a CUDA GPU with native
bfloat16support (Ampere / Ada / Hopper / Blackwell, such as RTX 3090 / 4090 / L4 / G4 / A100) and PyTorch/ComfyUI INT8 quant ops support (ComfyUI >= v0.45.0).
License & Attribution
Licensed under Apache License 2.0, matching the upstream ByteDance-Seed/BAGEL-7B-MoT repository and ComfyUI-BAGEL. See NOTICE for full upstream attribution and a summary of modifications made to the weights and custom node.
- Downloads last month
- 55



