Instructions to use mingyi456/SRPO-DF11-ComfyUI with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusion Single File
How to use mingyi456/SRPO-DF11-ComfyUI with Diffusion Single File:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -12,11 +12,13 @@ tags:
|
|
| 12 |
- comfyui
|
| 13 |
---
|
| 14 |
|
| 15 |
-
Thanks to @rootey for pointing out that my existing DF11 compressions do not work with ComfyUI.
|
|
|
|
|
|
|
| 16 |
|
| 17 |
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
|
| 18 |
|
| 19 |
-
Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be tricky for me.
|
| 20 |
|
| 21 |
This compressed model was made from [rockerBOO/flux.1-dev-SRPO](https://huggingface.co/rockerBOO/flux.1-dev-SRPO)'s BF16 quantization. Thanks to rockerBOO, without which I would not have been able to directly work with. (My PC only has 48 GB of system RAM, too little to work with a 12B model in FP32 precision)
|
| 22 |
|
|
@@ -28,4 +30,30 @@ Follow the instructions here: https://github.com/LeanModels/ComfyUI-DFloat11. Af
|
|
| 28 |

|
| 29 |
|
| 30 |
#### `diffusers`
|
| 31 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 12 |
- comfyui
|
| 13 |
---
|
| 14 |
|
| 15 |
+
Thanks to @rootey for pointing out that my existing DF11 compressions do not work with ComfyUI. After I realized the issue was with the different naming schemes of the tensors, I worked on trying to translate and remap each tensor in the `FluxTransformer2D` model (in the `diffusers` library), to the `Flux.diffusion_model` model (which ComfyUI, and I believe other platforms, use), only to realize that the tensors corresponding to the DF11 compressed weights expect to be decompressed into a specific structure at compression time, and trying to adapt for it will be extremely difficult for me, although it should be theoretically possible for an expert to do.
|
| 16 |
+
|
| 17 |
+
Instead, I figured out a way to manually load an instance of the `Flux.diffusion_model` model in ComfyUI, and carry out the compression in ComfyUI instead. Unfortunately, this means that 2 separate, mutrally-incompatible versions of DF11 models need to coexist, which I personally find to be bothersome. Hence, I am currently considering not to release the custom nodes/script for producing DF11 compressed models for ComfyUI, while I research on the feasibility of adapting the existing models. In the meantime, I plan to release ComfyUI compatible DF11 compressions for only a select handful of models, but feel free to request for any of the existing DF11 models to be remade into ComfyUI format.
|
| 18 |
|
| 19 |
For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
|
| 20 |
|
| 21 |
+
Feel free to request for other models for compression as well, although compressing models that do not use the Flux architecture might be slightly tricky for me.
|
| 22 |
|
| 23 |
This compressed model was made from [rockerBOO/flux.1-dev-SRPO](https://huggingface.co/rockerBOO/flux.1-dev-SRPO)'s BF16 quantization. Thanks to rockerBOO, without which I would not have been able to directly work with. (My PC only has 48 GB of system RAM, too little to work with a 12B model in FP32 precision)
|
| 24 |
|
|
|
|
| 30 |

|
| 31 |
|
| 32 |
#### `diffusers`
|
| 33 |
+
Refer to this [model](https://huggingface.co/mingyi456/SRPO-DF11) instead.
|
| 34 |
+
|
| 35 |
+
### Compression Details
|
| 36 |
+
|
| 37 |
+
This is the `pattern_dict` for compressing Flux-based models in ComfyUI:
|
| 38 |
+
|
| 39 |
+
```python
|
| 40 |
+
pattern_dict_comfyui = {
|
| 41 |
+
"double_blocks\.\d+": (
|
| 42 |
+
"img_mod.lin",
|
| 43 |
+
"img_attn.qkv",
|
| 44 |
+
"img_attn.proj",
|
| 45 |
+
"img_mlp.0",
|
| 46 |
+
"img_mlp.2",
|
| 47 |
+
"txt_mod.lin",
|
| 48 |
+
"txt_attn.qkv",
|
| 49 |
+
"txt_attn.proj",
|
| 50 |
+
"txt_mlp.0",
|
| 51 |
+
"txt_mlp.2",
|
| 52 |
+
),
|
| 53 |
+
"single_blocks\.\d+": (
|
| 54 |
+
"linear1",
|
| 55 |
+
"linear2",
|
| 56 |
+
"modulation.lin",
|
| 57 |
+
),
|
| 58 |
+
}
|
| 59 |
+
```
|