Text-to-Image
Diffusers
Safetensors
English
8-bit precision
mingyi456 commited on
Commit
fd3a586
·
verified ·
1 Parent(s): 7b983fb

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +76 -1
README.md CHANGED
@@ -9,4 +9,79 @@ base_model:
9
  - circlestone-labs/Anima-Base-v1.0-Diffusers
10
  base_model_relation: quantized
11
  library_name: diffusers
12
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
9
  - circlestone-labs/Anima-Base-v1.0-Diffusers
10
  base_model_relation: quantized
11
  library_name: diffusers
12
+ ---
13
+ For more information (including how to compress models yourself), check out https://huggingface.co/DFloat11 and https://github.com/LeanModels/DFloat11
14
+
15
+ Feel free to request for other models for compression as well (for either the `diffusers` library, ComfyUI, or any other model), although models that use architectures which are unfamiliar to me might be more difficult.
16
+
17
+ ### How to Use
18
+
19
+ #### `diffusers`
20
+
21
+ ```python
22
+ import torch
23
+ from diffusers import ModularPipeline, CosmosTransformer3DModel
24
+ from dfloat11 import DFloat11Model
25
+ from transformers.initialization import no_init_weights
26
+
27
+ with no_init_weights():
28
+ transformer = CosmosTransformer3DModel.from_config(
29
+ CosmosTransformer3DModel.load_config(
30
+ r"circlestone-labs/Anima-Base-v1.0-Diffusers",
31
+ subfolder="transformer"
32
+ ),
33
+ torch_dtype=torch.bfloat16
34
+ ).to(torch.bfloat16)
35
+
36
+ DFloat11Model.from_pretrained(r"mingyi456/Anima-Base-v1.0-DF11-Diffusers", device="cpu", bfloat16_model=transformer)
37
+
38
+ text_encoder = DFloat11Model.from_pretrained(r"mingyi456/Qwen3-0.6B-Base-DF11", device = "cpu")
39
+
40
+ pipe = ModularPipeline.from_pretrained(r"circlestone-labs/Anima-Base-v1.0-Diffusers")
41
+
42
+ pipe.update_components(text_encoder=text_encoder.model)
43
+ pipe.update_components(transformer=transformer)
44
+
45
+ pipe.load_components(dtype=torch.bfloat16)
46
+
47
+ pipe.to("cuda")
48
+
49
+ image = pipe(
50
+ prompt="masterpiece, best quality, 1girl, solo, city lights",
51
+ generator=torch.Generator("cpu").manual_seed(114514),
52
+ ).images[0]
53
+ ```
54
+
55
+ #### ComfyUI
56
+ Refer to this [model](https://huggingface.co/mingyi456/Anima-DF11-ComfyUI) instead.
57
+
58
+ ### Compression details
59
+
60
+ This is the `pattern_dict` for compression:
61
+
62
+ ```python
63
+ pattern_dict = {
64
+ r"transformer_blocks\.\d+": (
65
+ "norm1.linear_1",
66
+ "norm1.linear_2",
67
+ "attn1.to_q",
68
+ "attn1.to_k",
69
+ "attn1.to_v",
70
+ "attn1.to_out.0",
71
+ "norm2.linear_1",
72
+ "norm2.linear_2",
73
+ "attn2.to_q",
74
+ "attn2.to_k",
75
+ "attn2.to_v",
76
+ "attn2.to_out.0",
77
+ "norm3.linear_1",
78
+ "norm3.linear_2",
79
+ "ff.net.0.proj",
80
+ "ff.net.2"
81
+ ),
82
+ r"time_embed\.t_embedder": (
83
+ "linear_1",
84
+ "linear_2",
85
+ )
86
+ }
87
+ ```