int8 text-encoder erroring out

by eepos - opened 24 days ago

Discussion

eepos

24 days ago

•

edited 24 days ago

While bf16 TE works fine and I can generate images succesfully, I get an error with int8 TE.

Running ComfyUI Portable on Win11. Python 3.13.12, torch 2.12.0+cu130.

error log:

[INFO] got prompt
[INFO] Found quantization metadata version 1
[INFO] Model storage policy: fast_disk=True paths=['G:\\ComfyUI_windows_portable\\ComfyUI\\models\\text_encoders\\ming_image_0.1_ling_mini_2.0_int8_convrot.safetensors']
[INFO] CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cpu, dtype: torch.float16
[INFO] Requested to load MingImageTEModel_
[INFO] Model MingImageTEModel_ prepared for dynamic VRAM loading. 16899MB Staged. 0 patches attached. Force pre-loaded 139 weights: 357 KB.
[ERROR] !!! Exception during processing !!! No backend can handle 'dequantize_int8_convrot_weight_dtype': cuda: q: shape [256, 1024, 2048] fails: exactly 2D; eager: q: shape [256, 1024, 2048] fails: exactly 2D
[ERROR] Traceback (most recent call last):
  File "G:\ComfyUI_windows_portable\ComfyUI\execution.py", line 547, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\execution.py", line 352, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\execution.py", line 326, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "G:\ComfyUI_windows_portable\ComfyUI\execution.py", line 314, in process_inputs
    result = f(**inputs)
  File "G:\ComfyUI_windows_portable\ComfyUI\nodes.py", line 77, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 346, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\sd.py", line 417, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 743, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\sd1_clip.py", line 306, in encode
    return self(tokens)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 328, in forward
    cap_feats, direct_context = self.transformer(embeds, attention_mask, embeds_info)
                                ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 261, in forward
    hidden, captured = self.thinker(embeds, attention_mask, start, size, self.capture_layers)
                       ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 208, in forward
    x = layer(x, mask, freqs_cis, optimized_attention, image_mask)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 161, in forward
    return x + self.mlp(h, image_mask)
               ~~~~~~~~^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 140, in forward
    out = self.experts(flat, topk_idx, topk_weight).view(batch, seq_len, hidden)
          ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\nn\modules\module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\ming_image.py", line 121, in forward
    return moe_experts_forward(x, topk_idx, topk_weight, self.num_experts, self.gate_up_proj, self.down_proj, swiglu)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\text_encoders\llama.py", line 567, in moe_experts_forward
    with gate_up_proj.bank_resident(x) as gate_up_bank, down_proj.bank_resident(x) as down_bank:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "contextlib.py", line 141, in __enter__
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 1566, in bank_resident
    with CastBiasWeightContext(self, input, offloadable=True) as self._resident_bank:
         ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 468, in __init__
    self.state = (None, None) if self.slf is None else cast_bias_weight(*args, **kwargs)
                                                       ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 382, in cast_bias_weight
    weight, bias = resolve_cast_module_with_vbar(s, dtype, device, bias_dtype, compute_dtype, want_requant)
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 329, in resolve_cast_module_with_vbar
    weight = post_cast(s, "weight", weight, dtype, prefetch["resident"], update_weight)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 310, in post_cast
    x = to_dequant(x, dtype)
  File "G:\ComfyUI_windows_portable\ComfyUI\comfy\ops.py", line 306, in to_dequant
    tensor = tensor.dequantize()
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfy_kitchen\tensor\base.py", line 291, in dequantize
    full = self.layout_cls.dequantize(qdata, self._params)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfy_kitchen\tensor\int8.py", line 172, in dequantize
    result = torch.ops.comfy_kitchen.dequantize_int8_convrot_weight_dtype(
        qdata, params.scale, params.convrot_groupsize, output_dtype_code
    )
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\_ops.py", line 1275, in __call__
    return self._op(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\_library\custom_ops.py", line 375, in backend_impl
    result = self._backend_fns[device_type](*args, **kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\_compile.py", line 54, in inner
    return disable_fn(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\_dynamo\eval_frame.py", line 1298, in _fn
    return fn(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\torch\_library\custom_ops.py", line 410, in wrapped_fn
    return fn(*args, **kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfy_kitchen\backends\eager\quantization.py", line 1162, in _op_dequantize_int8_convrot_weight_dtype
    impl = registry.get_implementation("dequantize_int8_convrot_weight_dtype", kwargs=kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfy_kitchen\registry.py", line 269, in get_implementation
    selected_backend = self.get_capable_backend(func_name, kwargs)
  File "G:\ComfyUI_windows_portable\python_embeded\Lib\site-packages\comfy_kitchen\registry.py", line 233, in get_capable_backend
    raise NoCapableBackendError(func_name, failures)
comfy_kitchen.exceptions.NoCapableBackendError: No backend can handle 'dequantize_int8_convrot_weight_dtype': cuda: q: shape [256, 1024, 2048] fails: exactly 2D; eager: q: shape [256, 1024, 2048] fails: exactly 2D

[INFO] Prompt executed in 1.14 seconds

itok-hf

24 days ago

•

edited 24 days ago

Mee too. Ubuntu 26.04，Python 3.13.7 ComfyUI: v0.37.0-15-g51c1e29f7 (2026-09-23), comfy-kitchen 0.2.35, torch 2.13.0+cu132

error log:

ComfyUI Error Report

Error Details

Node ID: 597
Node Type: CLIPTextEncode
Exception Type: comfy_kitchen.exceptions.NoCapableBackendError
Exception Message: comfy_kitchen.exceptions.NoCapableBackendError: No backend can handle 'dequantize_int8_convrot_weight_dtype': cuda: q: shape [256, 1024, 2048] fails: exactly 2D; eager: q: shape [256, 1024, 2048] fails: exactly 2D

Stack Trace

  File "/home/SD/ComfyUI/execution.py", line 547, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/execution.py", line 352, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/execution.py", line 326, in _async_map_node_over_list
    await process_inputs(input_dict, i)

  File "/home/SD/ComfyUI/execution.py", line 314, in process_inputs
    result = f(**inputs)

  File "/home/SD/ComfyUI/nodes.py", line 77, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^

  File "/home/SD/ComfyUI/comfy/sd.py", line 346, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)

  File "/home/SD/ComfyUI/comfy/sd.py", line 417, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)

  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 743, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)

  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)

  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 306, in encode
    return self(tokens)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 328, in forward
    cap_feats, direct_context = self.transformer(embeds, attention_mask, embeds_info)
                                ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 261, in forward
    hidden, captured = self.thinker(embeds, attention_mask, start, size, self.capture_layers)
                       ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 208, in forward
    x = layer(x, mask, freqs_cis, optimized_attention, image_mask)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 161, in forward
    return x + self.mlp(h, image_mask)
               ~~~~~~~~^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 140, in forward
    out = self.experts(flat, topk_idx, topk_weight).view(batch, seq_len, hidden)
          ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)

  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 121, in forward
    return moe_experts_forward(x, topk_idx, topk_weight, self.num_experts, self.gate_up_proj, self.down_proj, swiglu)

  File "/home/SD/ComfyUI/comfy/text_encoders/llama.py", line 567, in moe_experts_forward
    with gate_up_proj.bank_resident(x) as gate_up_bank, down_proj.bank_resident(x) as down_bank:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^

  File "/home/.pyenv/versions/3.13.7/lib/python3.13/contextlib.py", line 141, in __enter__
    return next(self.gen)

  File "/home/SD/ComfyUI/comfy/ops.py", line 1566, in bank_resident
    with CastBiasWeightContext(self, input, offloadable=True) as self._resident_bank:
         ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/comfy/ops.py", line 468, in __init__
    self.state = (None, None) if self.slf is None else cast_bias_weight(*args, **kwargs)
                                                       ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/comfy/ops.py", line 436, in cast_bias_weight
    weight = weight.dequantize()

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/tensor/base.py", line 291, in dequantize
    full = self.layout_cls.dequantize(qdata, self._params)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/tensor/int8.py", line 172, in dequantize
    result = torch.ops.comfy_kitchen.dequantize_int8_convrot_weight_dtype(
        qdata, params.scale, params.convrot_groupsize, output_dtype_code
    )

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_ops.py", line 1279, in __call__
    return self._op(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_library/custom_ops.py", line 441, in backend_impl
    result = self._backend_fns[device_type](*args, **kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_compile.py", line 54, in inner
    return disable_fn(*args, **kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_dynamo/eval_frame.py", line 1446, in _fn
    return fn(*args, **kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_library/custom_ops.py", line 502, in wrapped_fn
    return fn(*args, **kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/backends/eager/quantization.py", line 1162, in _op_dequantize_int8_convrot_weight_dtype
    impl = registry.get_implementation("dequantize_int8_convrot_weight_dtype", kwargs=kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/registry.py", line 269, in get_implementation
    selected_backend = self.get_capable_backend(func_name, kwargs)

  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/registry.py", line 233, in get_capable_backend
    raise NoCapableBackendError(func_name, failures)

System Information

ComfyUI Version: 0.37.0
Arguments: main.py --use-sage-attention --highvram --disable-all-custom-nodes
OS: linux
Python Version: 3.13.7 (main, Sep 15 2025, 14:35:55) [GCC 13.3.0]
Embedded Python: false
PyTorch Version: 2.13.0+cu132

Devices

Name: cuda:0 NVIDIA RTX PRO 6000 Blackwell Max-Q Workstation Edition : cudaMallocAsync
- Type: cuda
- VRAM Total: 102014189568
- VRAM Free: 100826349568
- Torch VRAM Total: 0
- Torch VRAM Free: 0
Name: cuda:1 NVIDIA GeForce RTX 4060 Ti : cudaMallocAsync
- Type: cuda
- VRAM Total: 16752508928
- VRAM Free: 16605446144
- Torch VRAM Total: 0
- Torch VRAM Free: 0

Logs

2026-09-23T20:53:52.637493 - [32m[INFO][0m got prompt
2026-09-23T20:53:52.650280 - [32m[INFO][0m Model storage policy: fast_disk=True paths=['/media/DATA/SD/ComfyUI/models/vae/ming_image_vae_bf16.safetensors']
2026-09-23T20:53:52.653436 - [32m[INFO][0m Using pytorch attention in VAE
2026-09-23T20:53:52.654205 - [32m[INFO][0m Using pytorch attention in VAE
2026-09-23T20:53:53.352669 - [32m[INFO][0m VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
2026-09-23T20:53:53.376975 - [32m[INFO][0m Found quantization metadata version 1
2026-09-23T20:53:53.377014 - [32m[INFO][0m Detected mixed precision quantization
2026-09-23T20:53:53.377140 - [32m[INFO][0m Using mixed precision operations
2026-09-23T20:53:53.377190 - [32m[INFO][0m Native ops: convrot_w4a4, nvfp4, float8_e4m3fn, int8_tensorwise, asym_w4a8_int8, mxfp8, float8_e5m2 
2026-09-23T20:53:53.386902 - [32m[INFO][0m model weight dtype torch.bfloat16, manual cast: torch.bfloat16
2026-09-23T20:53:53.388258 - [32m[INFO][0m model_type FLOW
2026-09-23T20:53:53.388462 - [32m[INFO][0m Model storage policy: fast_disk=True paths=['/media/DATA/SD/ComfyUI/models/diffusion_models/ming_image/ming_image_0.1_design_int8_convrot.safetensors']
2026-09-23T20:53:53.429386 - [32m[INFO][0m Found quantization metadata version 1
2026-09-23T20:53:53.782070 - [32m[INFO][0m Model storage policy: fast_disk=True paths=['/media/DATA/SD/ComfyUI/models/text_encoders/ming_image/ming_image_0.1_ling_mini_2.0_int8_convrot.safetensors']
2026-09-23T20:53:55.098399 - [32m[INFO][0m Requested to load MingImageTEModel_
2026-09-23T20:53:55.131106 - [32m[INFO][0m loaded completely;  16899.62 MB loaded, full load: True
2026-09-23T20:53:55.138186 - [32m[INFO][0m CLIP/text encoder model load device: cuda:0, offload device: cpu, current: cuda:0, dtype: torch.float16
2026-09-23T20:53:55.639542 - [1m[31m[ERROR][0m !!! Exception during processing !!! No backend can handle 'dequantize_int8_convrot_weight_dtype': cuda: q: shape [256, 1024, 2048] fails: exactly 2D; eager: q: shape [256, 1024, 2048] fails: exactly 2D
2026-09-23T20:53:55.643463 - [1m[31m[ERROR][0m Traceback (most recent call last):
  File "/home/SD/ComfyUI/execution.py", line 547, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                                                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/execution.py", line 352, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/execution.py", line 326, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "/home/SD/ComfyUI/execution.py", line 314, in process_inputs
    result = f(**inputs)
  File "/home/SD/ComfyUI/nodes.py", line 77, in encode
    return (clip.encode_from_tokens_scheduled(tokens), )
            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^
  File "/home/SD/ComfyUI/comfy/sd.py", line 346, in encode_from_tokens_scheduled
    pooled_dict = self.encode_from_tokens(tokens, return_pooled=return_pooled, return_dict=True)
  File "/home/SD/ComfyUI/comfy/sd.py", line 417, in encode_from_tokens
    o = self.cond_stage_model.encode_token_weights(tokens)
  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 743, in encode_token_weights
    out = getattr(self, self.clip).encode_token_weights(token_weight_pairs)
  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 45, in encode_token_weights
    o = self.encode(to_encode)
  File "/home/SD/ComfyUI/comfy/sd1_clip.py", line 306, in encode
    return self(tokens)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 328, in forward
    cap_feats, direct_context = self.transformer(embeds, attention_mask, embeds_info)
                                ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 261, in forward
    hidden, captured = self.thinker(embeds, attention_mask, start, size, self.capture_layers)
                       ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 208, in forward
    x = layer(x, mask, freqs_cis, optimized_attention, image_mask)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 161, in forward
    return x + self.mlp(h, image_mask)
               ~~~~~~~~^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 140, in forward
    out = self.experts(flat, topk_idx, topk_weight).view(batch, seq_len, hidden)
          ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1778, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1789, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/SD/ComfyUI/comfy/text_encoders/ming_image.py", line 121, in forward
    return moe_experts_forward(x, topk_idx, topk_weight, self.num_experts, self.gate_up_proj, self.down_proj, swiglu)
  File "/home/SD/ComfyUI/comfy/text_encoders/llama.py", line 567, in moe_experts_forward
    with gate_up_proj.bank_resident(x) as gate_up_bank, down_proj.bank_resident(x) as down_bank:
         ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/home/.pyenv/versions/3.13.7/lib/python3.13/contextlib.py", line 141, in __enter__
    return next(self.gen)
  File "/home/SD/ComfyUI/comfy/ops.py", line 1566, in bank_resident
    with CastBiasWeightContext(self, input, offloadable=True) as self._resident_bank:
         ~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/comfy/ops.py", line 468, in __init__
    self.state = (None, None) if self.slf is None else cast_bias_weight(*args, **kwargs)
                                                       ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/comfy/ops.py", line 436, in cast_bias_weight
    weight = weight.dequantize()
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/tensor/base.py", line 291, in dequantize
    full = self.layout_cls.dequantize(qdata, self._params)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/tensor/int8.py", line 172, in dequantize
    result = torch.ops.comfy_kitchen.dequantize_int8_convrot_weight_dtype(
        qdata, params.scale, params.convrot_groupsize, output_dtype_code
    )
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_ops.py", line 1279, in __call__
    return self._op(*args, **kwargs)
           ~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_library/custom_ops.py", line 441, in backend_impl
    result = self._backend_fns[device_type](*args, **kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_compile.py", line 54, in inner
    return disable_fn(*args, **kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_dynamo/eval_frame.py", line 1446, in _fn
    return fn(*args, **kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/torch/_library/custom_ops.py", line 502, in wrapped_fn
    return fn(*args, **kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/backends/eager/quantization.py", line 1162, in _op_dequantize_int8_convrot_weight_dtype
    impl = registry.get_implementation("dequantize_int8_convrot_weight_dtype", kwargs=kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/registry.py", line 269, in get_implementation
    selected_backend = self.get_capable_backend(func_name, kwargs)
  File "/home/SD/ComfyUI/venv/lib/python3.13/site-packages/comfy_kitchen/registry.py", line 233, in get_capable_backend
    raise NoCapableBackendError(func_name, failures)
comfy_kitchen.exceptions.NoCapableBackendError: No backend can handle 'dequantize_int8_convrot_weight_dtype': cuda: q: shape [256, 1024, 2048] fails: exactly 2D; eager: q: shape [256, 1024, 2048] fails: exactly 2D

2026-09-23T20:53:55.645838 - [32m[INFO][0m [32mPrompt executed in 3.00 seconds[0m

eepos

24 days ago

•

edited 24 days ago

Kijai fixed it, works for me now:

https://github.com/kijai/ComfyUI/commit/ec2af723496b13ed52c049ac5b5524ec6cc6a7cd

itok-hf

24 days ago

I also confirmed that the issue has been fixed.
Thank you eepos. Thank you Kijai.

eepos changed discussion status to closed 24 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment