madtune commited on
Commit
1b1e962
·
verified ·
1 Parent(s): c374d2c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +11 -73
README.md CHANGED
@@ -1,14 +1,13 @@
1
  ---
2
  license: other
3
  tags:
4
- - text-to-image
5
- - diffusion
6
- - pixeldit
7
- - nvidia
8
- - pixel-space
9
- - lora
10
  base_model: nvidia/PixelDiT-1300M-1024px
11
- library_name: diffusers
12
  ---
13
 
14
  ![FourNeuron-PixelDiT Banner](assets/banner.png)
@@ -60,12 +59,6 @@ python generate.py --prompt "a viking warrior on a cliff at sunset, cinematic"
60
  # Portrait mode
61
  python generate.py --height 1280 --width 768 --steps 60 --cfg 8.5 --prompt "your prompt"
62
 
63
- # Qwen encoder (creative/fantasy)
64
- python generate.py --encoder qwen --proj qwen_proj.pt --prompt "A giant hamster emperor in a battle fortress"
65
-
66
- # With LoRA
67
- python generate.py --lora lora_yarn_out/best --prompt "a dark anime woman in a field, yarn art style"
68
-
69
  # LCM fast mode (8 steps)
70
  python generate.py --scheduler lcm --steps 8 --cfg 2.0 --prompt "your prompt"
71
  ```
@@ -76,22 +69,10 @@ python generate.py --scheduler lcm --steps 8 --cfg 2.0 --prompt "your prompt"
76
 
77
  ```python
78
  import torch
79
- from transformers import AutoTokenizer, AutoModelForCausalLM
80
- from diffusers.pipelines.pixeldit import PixelDiTPipeline
81
-
82
- tokenizer = AutoTokenizer.from_pretrained("Efficient-Large-Model/gemma-2-2b-it")
83
- tokenizer.padding_side = "right"
84
- text_encoder = (
85
- AutoModelForCausalLM.from_pretrained("Efficient-Large-Model/gemma-2-2b-it", dtype=torch.bfloat16)
86
- .get_decoder().eval()
87
- )
88
-
89
- pipe = PixelDiTPipeline.from_pretrained(
90
- "madtune/pixeldit-diffusers",
91
- text_encoder=text_encoder,
92
- tokenizer=tokenizer,
93
- torch_dtype=torch.bfloat16,
94
- )
95
  pipe.enable_model_cpu_offload()
96
 
97
  image = pipe(
@@ -101,49 +82,11 @@ image = pipe(
101
  num_inference_steps=50,
102
  guidance_scale=7.5,
103
  ).images[0]
104
- image.save("out.jpg")
105
- ```
106
-
107
- ---
108
-
109
- ## LoRA
110
-
111
- ### Train a style LoRA
112
-
113
- ```bash
114
- # 1. Download images (Pexels API key required)
115
- python scripts/download_unsplash.py --query "yarn wool textile" --n 150 --out /data/lora_yarn
116
-
117
- # 2. Precompute embeddings
118
- python scripts/precompute_lora_data.py --images /data/lora_yarn --out /data/lora_yarn_cache --trigger "yarn art style" --recaption
119
 
120
- # 3. Train
121
- python scripts/train_lora.py --data /data/lora_yarn_cache --out lora_yarn_out/ --epochs 50 --batch 2
122
- ```
123
-
124
- ### Load LoRA in pipeline
125
-
126
- ```python
127
- pipe.load_lora_weights("lora_yarn_out/best")
128
- pipe.set_adapters(["default"], adapter_weights=[1.0])
129
-
130
- # merge multiple LoRAs
131
- pipe.load_lora_weights("lora_style/best", adapter_name="style")
132
- pipe.load_lora_weights("lora_char/best", adapter_name="char")
133
- pipe.set_adapters(["style", "char"], adapter_weights=[1.0, 0.7])
134
-
135
- # bake into weights
136
- pipe.fuse_lora()
137
  ```
138
 
139
  ---
140
-
141
- ## Qwen Encoder
142
-
143
- > **Coming soon.** Qwen3-2B integration (creative/fantasy prompts) is implemented in the pipeline but projection training scripts are not yet released. Watch this repo for updates.
144
-
145
- ---
146
-
147
  ## ComfyUI
148
 
149
  ```bash
@@ -163,13 +106,8 @@ Three nodes under **PixelDiT** category:
163
  |---|---|
164
  | `generate.py` | Main generation script |
165
  | `scripts/upscale_images.py` | RealESRGAN 4× upscale before LoRA precompute |
166
- | `scripts/precompute_lora_data.py` | Precompute image+caption pairs for LoRA training |
167
- | `scripts/train_lora.py` | LoRA fine-tuning |
168
- | `scripts/download_unsplash.py` | Download images from Pexels by search query |
169
  | `scripts/setup_diffusers_pixeldit.py` | Install pipeline into active venv's diffusers |
170
 
171
- See `howto_lora.md` for the full LoRA training walkthrough.
172
-
173
  ---
174
 
175
  ## Credits
 
1
  ---
2
  license: other
3
  tags:
4
+ - text-to-image
5
+ - diffusion
6
+ - pixeldit
7
+ - nvidia
8
+ - pixel-space
9
+ - lora
10
  base_model: nvidia/PixelDiT-1300M-1024px
 
11
  ---
12
 
13
  ![FourNeuron-PixelDiT Banner](assets/banner.png)
 
59
  # Portrait mode
60
  python generate.py --height 1280 --width 768 --steps 60 --cfg 8.5 --prompt "your prompt"
61
 
 
 
 
 
 
 
62
  # LCM fast mode (8 steps)
63
  python generate.py --scheduler lcm --steps 8 --cfg 2.0 --prompt "your prompt"
64
  ```
 
69
 
70
  ```python
71
  import torch
72
+ from diffusers import PixelDiTPipeline
73
+
74
+
75
+ pipe = PixelDiTPipeline.from_pretrained("madtune/pixeldit-diffusers", torch_dtype=torch.bfloat16)
 
 
 
 
 
 
 
 
 
 
 
 
76
  pipe.enable_model_cpu_offload()
77
 
78
  image = pipe(
 
82
  num_inference_steps=50,
83
  guidance_scale=7.5,
84
  ).images[0]
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
85
 
86
+ image.save("out.jpg")
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
87
  ```
88
 
89
  ---
 
 
 
 
 
 
 
90
  ## ComfyUI
91
 
92
  ```bash
 
106
  |---|---|
107
  | `generate.py` | Main generation script |
108
  | `scripts/upscale_images.py` | RealESRGAN 4× upscale before LoRA precompute |
 
 
 
109
  | `scripts/setup_diffusers_pixeldit.py` | Install pipeline into active venv's diffusers |
110
 
 
 
111
  ---
112
 
113
  ## Credits