Update README.md
Browse files
README.md
CHANGED
|
@@ -60,22 +60,21 @@ pipeline_tag: text-to-image
|
|
| 60 |
- OneTrainer: 910 steps, 1.12 s/it (`Local Batch Size: 3, LR: 0.0003`) → ~0.37s/img (17m 43s)
|
| 61 |
- Fizgig(Batch-Size: Not Supported): 2730 steps, 1.25 it/s (`Gradient Accumulation: 3, LR: 0.0001`) → ~0.83s/img (36m 17s)
|
| 62 |
- Fizgig-Advanced(Batch-Size: Supported): 2730 steps, 1.25 it/s (`Batch Size: 3, LR: 0.0003`) → ~0.83s/img (36m 17s)
|
| 63 |
-
- *Note on Speed & Formats: OneTrainer used the official sharded Diffusers format (bfloat16), while Fizgig used the ComfyUI single-file format (int8).*
|
| 64 |
- Showcase & Card Image Parameters:
|
| 65 |
- Model: -
|
| 66 |
- LoRA Weight: 0.8 ~ 1
|
| 67 |
|
| 68 |
-
|
| 69 |
-
|
| 70 |
-
|
| 71 |
-
|
| 72 |
-
|
| 73 |
-
|
| 74 |
-
|
| 75 |
-
|
| 76 |
-
|
| 77 |
-
|
| 78 |
-
|
| 79 |
- **Krea2**
|
| 80 |
- Training Tools: [Onetrainer](https://github.com/nerogar/OneTrainer)
|
| 81 |
- Dataset: Mostly 50+ images (averaging 100+)
|
|
|
|
| 60 |
- OneTrainer: 910 steps, 1.12 s/it (`Local Batch Size: 3, LR: 0.0003`) → ~0.37s/img (17m 43s)
|
| 61 |
- Fizgig(Batch-Size: Not Supported): 2730 steps, 1.25 it/s (`Gradient Accumulation: 3, LR: 0.0001`) → ~0.83s/img (36m 17s)
|
| 62 |
- Fizgig-Advanced(Batch-Size: Supported): 2730 steps, 1.25 it/s (`Batch Size: 3, LR: 0.0003`) → ~0.83s/img (36m 17s)
|
| 63 |
+
- *(Note on Speed & Formats: OneTrainer used the official sharded Diffusers format (bfloat16), while Fizgig used the ComfyUI single-file format (int8). Although int8/fp8 usually compute faster on modern GPUs, the benchmark shows Fizgig is slower here—likely due to software-level framework overheads. Fizgig also supports the original bf16 Diffusers shards.)*
|
| 64 |
- Showcase & Card Image Parameters:
|
| 65 |
- Model: -
|
| 66 |
- LoRA Weight: 0.8 ~ 1
|
| 67 |
|
| 68 |
+
> #### LR / Batch Size / Gradient Accumulation
|
| 69 |
+
>
|
| 70 |
+
> - **Effective Batch Size** = `Batch Size × Gradient Accumulation`. Gradient accumulation saves VRAM but is slower; a larger Batch Size is faster but uses more VRAM.
|
| 71 |
+
> - **LR Scaling**: Don't blindly apply the linear rule to LoRA. Validate with loss curves. (My baseline: `Batch 3` + `LR 0.0003`). *Note: The vanilla Fizgig benchmark used the default `LR 0.0001`, which is technically under-scaled for an effective batch of 3; however, it was sufficient for this speed test.*
|
| 72 |
+
>
|
| 73 |
+
> #### Paper TL;DR:
|
| 74 |
+
> - [*Accurate, Large Minibatch SGD* (2017)](https://arxiv.org/abs/1706.02677): The "Linear Scaling Rule" — if you multiply batch size by k, multiply LR by k. This was designed for full fine-tuning on ImageNet, not LoRA.
|
| 75 |
+
> - [*One weird trick for parallelizing CNNs* (2014)](https://arxiv.org/abs/1404.5997): The "Square Root Scaling Rule" — to keep gradient variance stable, scale LR by √k instead. More conservative when batch gets very large.
|
| 76 |
+
> - [*Beware of the Batch Size* (2026)](https://arxiv.org/abs/2602.09492): For LoRA specifically, batch size is not just a multiplier for LR — it's a **first-order design parameter**. Optimal batch depends on dataset size (most sensitive), rank, and model capacity. Vanilla LoRA matches fancy variants when batch is tuned properly.
|
| 77 |
+
|
|
|
|
| 78 |
- **Krea2**
|
| 79 |
- Training Tools: [Onetrainer](https://github.com/nerogar/OneTrainer)
|
| 80 |
- Dataset: Mostly 50+ images (averaging 100+)
|