|
Download docs/train_network_README-ja.md from trixlyaisolutions/onnx: direct link, hf CLI and curl.
- Browser
- Download file 29.3 kB
-
https://huggingface.co/trixlyaisolutions/onnx/resolve/main/docs/train_network_README-ja.md
- Command line
-
hf download hf://trixlyaisolutions/onnx/docs/train_network_README-ja.md
-
curl -L -o train_network_README-ja.md https://huggingface.co/trixlyaisolutions/onnx/resolve/main/docs/train_network_README-ja.md
29.3 kB
| # LoRAã®åŠç¿ã«ã€ã㊠| |
| [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685)ïŒarxivïŒã[LoRA](https://github.com/microsoft/LoRA)ïŒgithubïŒãStable Diffusionã«é©çšãããã®ã§ãã | |
| [cloneofsimoæ°ã®ãªããžããª](https://github.com/cloneofsimo/lora)ã倧ãã«åèã«ãããŠããã ããŸãããããããšãããããŸãã | |
| éåžžã®LoRA㯠Linear ãããŽã«ãŒãã«ãµã€ãº 1x1 ã® Conv2d ã«ã®ã¿é©çšãããŸãããã«ãŒãã«ãµã€ãº 3x3 ã®Conv2dã«é©çšãæ¡å€§ããããšãã§ããŸãã | |
| Conv2d 3x3ãžã®æ¡å€§ã¯ [cloneofsimoæ°](https://github.com/cloneofsimo/lora) ãæåã«ãªãªãŒã¹ããKohakuBlueleafæ°ã [LoCon](https://github.com/KohakuBlueleaf/LoCon) ã§ãã®æå¹æ§ãæããã«ãããã®ã§ããKohakuBlueleafæ°ã«æ·±ãæè¬ããŸãã | |
| 8GB VRAMã§ãããããåäœããããã§ãã | |
| [åŠç¿ã«ã€ããŠã®å ±éããã¥ã¡ã³ã](./train_README-ja.md) ãããããŠã芧ãã ããã | |
| # åŠç¿ã§ããLoRAã®çš®é¡ | |
| 以äžã®äºçš®é¡ããµããŒãããŸãã以äžã¯åœãªããžããªå ã®ç¬èªã®åç§°ã§ãã | |
| 1. __LoRA-LierLa__ : (LoRA for __Li__ n __e__ a __r__ __La__ yersããªãšã©ãšèªã¿ãŸã) | |
| Linear ããã³ã«ãŒãã«ãµã€ãº 1x1 ã® Conv2d ã«é©çšãããLoRA | |
| 2. __LoRA-C3Lier__ : (LoRA for __C__ olutional layers with __3__ x3 Kernel and __Li__ n __e__ a __r__ layersãã»ãªã¢ãšèªã¿ãŸã) | |
| 1.ã«å ããã«ãŒãã«ãµã€ãº 3x3 ã® Conv2d ã«é©çšãããLoRA | |
| LoRA-LierLaã«æ¯ã¹ãLoRA-C3Liarã¯é©çšãããå±€ãå¢ããåãé«ã粟床ãæåŸ ã§ãããããããŸããã | |
| ãŸãåŠç¿æã¯ __DyLoRA__ ã䜿çšããããšãã§ããŸãïŒåŸè¿°ããŸãïŒã | |
| ## åŠç¿ããã¢ãã«ã«é¢ããæ³šæ | |
| LoRA-LierLa ã¯ãAUTOMATIC1111æ°ã®Web UIã®LoRAæ©èœã§äœ¿çšããããšãã§ããŸãã | |
| LoRA-C3Liarã䜿ãWeb UIã§çæããã«ã¯ããã¡ãã®[WebUIçšextension](https://github.com/kohya-ss/sd-webui-additional-networks)ã䜿ã£ãŠãã ããã | |
| ããããåŠç¿ããLoRAã®ã¢ãã«ããStable Diffusionã®ã¢ãã«ã«ãã®ãªããžããªå ã®ã¹ã¯ãªããã§ãããããããŒãžããããšãã§ããŸãã | |
| cloneofsimoæ°ã®ãªããžããªãããã³d8ahazardæ°ã®[Dreambooth Extension for Stable-Diffusion-WebUI](https://github.com/d8ahazard/sd_dreambooth_extension)ãšã¯ãçŸæç¹ã§ã¯äºææ§ããããŸãããããã€ãã®æ©èœæ¡åŒµãè¡ã£ãŠããããã§ãïŒåŸè¿°ïŒã | |
| # åŠç¿ã®æé | |
| ããããããã®ãªããžããªã®READMEãåç §ããç°å¢æŽåãè¡ã£ãŠãã ããã | |
| ## ããŒã¿ã®æºå | |
| [åŠç¿ããŒã¿ã®æºåã«ã€ããŠ](./train_README-ja.md) ãåç §ããŠãã ããã | |
| ## åŠç¿ã®å®è¡ | |
| `train_network.py`ãçšããŸãã | |
| `train_network.py`ã§ã¯ `--network_module` ãªãã·ã§ã³ã«ãåŠç¿å¯Ÿè±¡ã®ã¢ãžã¥ãŒã«åãæå®ããŸããLoRAã«å¯Ÿå¿ããã®ã¯`network.lora`ãšãªããŸãã®ã§ããããæå®ããŠãã ããã | |
| ãªãåŠç¿çã¯éåžžã®DreamBoothãfine tuningãããé«ãã®ã`1e-4`ïœ`1e-3`çšåºŠãæå®ãããšããããã§ãã | |
| 以äžã¯ã³ãã³ãã©ã€ã³ã®äŸã§ãã | |
| ``` | |
| accelerate launch --num_cpu_threads_per_process 1 train_network.py | |
| --pretrained_model_name_or_path=<.ckptãŸãã¯.safetensordãŸãã¯Diffusersçã¢ãã«ã®ãã£ã¬ã¯ããª> | |
| --dataset_config=<ããŒã¿æºåã§äœæãã.tomlãã¡ã€ã«> | |
| --output_dir=<åŠç¿ããã¢ãã«ã®åºåå ãã©ã«ã> | |
| --output_name=<åŠç¿ããã¢ãã«åºåæã®ãã¡ã€ã«å> | |
| --save_model_as=safetensors | |
| --prior_loss_weight=1.0 | |
| --max_train_steps=400 | |
| --learning_rate=1e-4 | |
| --optimizer_type="AdamW8bit" | |
| --xformers | |
| --mixed_precision="fp16" | |
| --cache_latents | |
| --gradient_checkpointing | |
| --save_every_n_epochs=1 | |
| --network_module=networks.lora | |
| ``` | |
| ãã®ã³ãã³ãã©ã€ã³ã§ã¯ LoRA-LierLa ãåŠç¿ãããŸãã | |
| `--output_dir` ãªãã·ã§ã³ã§æå®ãããã©ã«ãã«ãLoRAã®ã¢ãã«ãä¿åãããŸããä»ã®ãªãã·ã§ã³ããªããã£ãã€ã¶çã«ã€ããŠã¯ [åŠç¿ã®å ±éããã¥ã¡ã³ã](./train_README-ja.md) ã®ããã䜿ããããªãã·ã§ã³ããåç §ããŠãã ããã | |
| ãã®ä»ã以äžã®ãªãã·ã§ã³ãæå®ã§ããŸãã | |
| * `--network_dim` | |
| * LoRAã®RANKãæå®ããŸãïŒ``--networkdim=4``ãªã©ïŒãçç¥æã¯4ã«ãªããŸããæ°ãå€ãã»ã©è¡šçŸåã¯å¢ããŸãããåŠç¿ã«å¿ èŠãªã¡ã¢ãªãæéã¯å¢ããŸãããŸãéé²ã«å¢ãããŠãè¯ããªãããã§ãã | |
| * `--network_alpha` | |
| * ã¢ã³ããŒãããŒãé²ãå®å®ããŠåŠç¿ããããã® ``alpha`` å€ãæå®ããŸããããã©ã«ãã¯1ã§ãã``network_dim``ãšåãå€ãæå®ãããšä»¥åã®ããŒãžã§ã³ãšåãåäœã«ãªããŸãã | |
| * `--persistent_data_loader_workers` | |
| * Windowsç°å¢ã§æå®ãããšãšããã¯éã®åŸ ã¡æéãå€§å¹ ã«ççž®ãããŸãã | |
| * `--max_data_loader_n_workers` | |
| * ããŒã¿èªã¿èŸŒã¿ã®ããã»ã¹æ°ãæå®ããŸããããã»ã¹æ°ãå€ããšããŒã¿èªã¿èŸŒã¿ãéããªãGPUãå¹ççã«å©çšã§ããŸãããã¡ã€ã³ã¡ã¢ãªãæ¶è²»ããŸããããã©ã«ãã¯ã`8` ãŸã㯠`CPUåæå®è¡ã¹ã¬ããæ°-1` ã®å°ããã»ãããªã®ã§ãã¡ã€ã³ã¡ã¢ãªã«äœè£ããªãå ŽåããGPU䜿çšçã90%çšåºŠä»¥äžãªãããããã®æ°å€ãèŠãªãã `2` ãŸã㯠`1` çšåºŠãŸã§äžããŠãã ããã | |
| * `--network_weights` | |
| * åŠç¿åã«åŠç¿æžã¿ã®LoRAã®éã¿ãèªã¿èŸŒã¿ããããã远å ã§åŠç¿ããŸãã | |
| * `--network_train_unet_only` | |
| * U-Netã«é¢é£ããLoRAã¢ãžã¥ãŒã«ã®ã¿æå¹ãšããŸããfine tuningçãªåŠç¿ã§æå®ãããšãããããããŸããã | |
| * `--network_train_text_encoder_only` | |
| * Text Encoderã«é¢é£ããLoRAã¢ãžã¥ãŒã«ã®ã¿æå¹ãšããŸããTextual Inversionçãªå¹æãæåŸ ã§ãããããããŸããã | |
| * `--unet_lr` | |
| * U-Netã«é¢é£ããLoRAã¢ãžã¥ãŒã«ã«ãéåžžã®åŠç¿çïŒ--learning_rateãªãã·ã§ã³ã§æå®ïŒãšã¯ç°ãªãåŠç¿çã䜿ãæã«æå®ããŸãã | |
| * `--text_encoder_lr` | |
| * Text Encoderã«é¢é£ããLoRAã¢ãžã¥ãŒã«ã«ãéåžžã®åŠç¿çïŒ--learning_rateãªãã·ã§ã³ã§æå®ïŒãšã¯ç°ãªãåŠç¿çã䜿ãæã«æå®ããŸããText Encoderã®ã»ããè¥å¹²äœãã®åŠç¿çïŒ5e-5ãªã©ïŒã«ããã»ããè¯ãããšãã話ãããããã§ãã | |
| * `--network_args` | |
| * è€æ°ã®åŒæ°ãæå®ã§ããŸããåŸè¿°ããŸãã | |
| * `--alpha_mask` | |
| * ç»åã®ã¢ã«ãã¡å€ããã¹ã¯ãšããŠäœ¿çšããŸããééç»åãåŠç¿ããéã«äœ¿çšããŸãã[PR #1223](https://github.com/kohya-ss/sd-scripts/pull/1223) | |
| `--network_train_unet_only` ãš `--network_train_text_encoder_only` ã®äž¡æ¹ãšãæªæå®æïŒããã©ã«ãïŒã¯Text EncoderãšU-Netã®äž¡æ¹ã®LoRAã¢ãžã¥ãŒã«ãæå¹ã«ããŸãã | |
| # ãã®ä»ã®åŠç¿æ¹æ³ | |
| ## LoRA-C3Lier ãåŠç¿ãã | |
| `--network_args` ã«ä»¥äžã®ããã«æå®ããŠãã ããã`conv_dim` ã§ Conv2d (3x3) ã® rank ãã`conv_alpha` ã§ alpha ãæå®ããŠãã ããã | |
| ``` | |
| --network_args "conv_dim=4" "conv_alpha=1" | |
| ``` | |
| 以äžã®ããã« alpha çç¥æã¯1ã«ãªããŸãã | |
| ``` | |
| --network_args "conv_dim=4" | |
| ``` | |
| ## DyLoRA | |
| DyLoRAã¯ãã¡ãã®è«æã§ææ¡ããããã®ã§ãã[DyLoRA: Parameter Efficient Tuning of Pre-trained Models using Dynamic Search-Free Low-Rank Adaptation](https://arxiv.org/abs/2210.07558)ãå ¬åŒå®è£ ã¯[ãã¡ã](https://github.com/huawei-noah/KD-NLP/tree/main/DyLoRA)ã§ãã | |
| è«æã«ãããšãLoRAã®rankã¯å¿ ãããé«ãã»ããè¯ãããã§ã¯ãªãã察象ã®ã¢ãã«ãããŒã¿ã»ãããã¿ã¹ã¯ãªã©ã«ããé©åãªrankãæ¢ãå¿ èŠãããããã§ããDyLoRAã䜿ããšãæå®ããdim(rank)以äžã®ããŸããŸãªrankã§åæã«LoRAãåŠç¿ããŸããããã«ããæé©ãªrankãããããåŠç¿ããŠæ¢ãæéãçãããšãã§ããŸãã | |
| åœãªããžããªã®å®è£ ã¯å ¬åŒå®è£ ãããŒã¹ã«ç¬èªã®æ¡åŒµãå ããŠããŸãïŒãã®ããäžå ·åãªã©ãããããããŸããïŒã | |
| ### åœãªããžããªã®DyLoRAã®ç¹åŸŽ | |
| åŠç¿åŸã®DyLoRAã®ã¢ãã«ãã¡ã€ã«ã¯LoRAãšäºææ§ããããŸãããŸããã¢ãã«ãã¡ã€ã«ããæå®ããdim(rank)以äžã®è€æ°ã®dimã®LoRAãæœåºã§ããŸãã | |
| DyLoRA-LierLaãDyLoRA-C3Lierã®ã©ã¡ããåŠç¿ã§ããŸãã | |
| ### DyLoRAã§åŠç¿ãã | |
| `--network_module=networks.dylora` ã®ããã«ãDyLoRAã«å¯Ÿå¿ãã`network.dylora`ãæå®ããŠãã ããã | |
| ãŸã `--network_args` ã«ãããšãã°`--network_args "unit=4"`ã®ããã«`unit`ãæå®ããŸãã`unit`ã¯rankãåå²ããåäœã§ããããšãã°`--network_dim=16 --network_args "unit=4"` ã®ããã«æå®ããŸãã`unit`ã¯`network_dim`ãå²ãåããå€ïŒ`network_dim`ã¯`unit`ã®åæ°ïŒãšããŠãã ããã | |
| `unit`ãæå®ããªãå Žåã¯ã`unit=1`ãšããŠæ±ãããŸãã | |
| èšè¿°äŸã¯ä»¥äžã§ãã | |
| ``` | |
| --network_module=networks.dylora --network_dim=16 --network_args "unit=4" | |
| --network_module=networks.dylora --network_dim=32 --network_alpha=16 --network_args "unit=4" | |
| ``` | |
| DyLoRA-C3Lierã®å Žåã¯ã`--network_args` ã«`"conv_dim=4"`ã®ããã«`conv_dim`ãæå®ããŸããéåžžã®LoRAãšç°ãªãã`conv_dim`ã¯`network_dim`ãšåãå€ã§ããå¿ èŠããããŸããèšè¿°äŸã¯ä»¥äžã§ãã | |
| ``` | |
| --network_module=networks.dylora --network_dim=16 --network_args "conv_dim=16" "unit=4" | |
| --network_module=networks.dylora --network_dim=32 --network_alpha=16 --network_args "conv_dim=32" "conv_alpha=16" "unit=8" | |
| ``` | |
| ããšãã°dim=16ãunit=4ïŒåŸè¿°ïŒã§åŠç¿ãããšã4ã8ã12ã16ã®4ã€ã®rankã®LoRAãåŠç¿ãæœåºã§ããŸããæœåºããåã¢ãã«ã§ç»åãçæããæ¯èŒããããšã§ãæé©ãªrankã®LoRAãéžæã§ããŸãã | |
| ãã®ä»ã®ãªãã·ã§ã³ã¯éåžžã®LoRAãšåãã§ãã | |
| â» `unit`ã¯åœãªããžããªã®ç¬èªæ¡åŒµã§ãDyLoRAã§ã¯ådim(rank)ã®éåžžLoRAã«æ¯ã¹ããšåŠç¿æéãé·ããªãããšãäºæ³ããããããåå²åäœã倧ãããããã®ã§ãã | |
| ### DyLoRAã®ã¢ãã«ããLoRAã¢ãã«ãæœåºãã | |
| `networks`ãã©ã«ãå ã® `extract_lora_from_dylora.py`ã䜿çšããŸããæå®ãã`unit`åäœã§ãDyLoRAã®ã¢ãã«ããLoRAã®ã¢ãã«ãæœåºããŸãã | |
| ã³ãã³ãã©ã€ã³ã¯ããšãã°ä»¥äžã®ããã«ãªããŸãã | |
| ```powershell | |
| python networks\extract_lora_from_dylora.py --model "foldername/dylora-model.safetensors" --save_to "foldername/dylora-model-split.safetensors" --unit 4 | |
| ``` | |
| `--model` ã«ã¯DyLoRAã®ã¢ãã«ãã¡ã€ã«ãæå®ããŸãã`--save_to` ã«ã¯æœåºããã¢ãã«ãä¿åãããã¡ã€ã«åãæå®ããŸãïŒrankã®æ°å€ããã¡ã€ã«åã«ä»å ãããŸãïŒã`--unit` ã«ã¯DyLoRAã®åŠç¿æã®`unit`ãæå®ããŸãã | |
| ## éå±€å¥åŠç¿ç | |
| 詳现ã¯[PR #355](https://github.com/kohya-ss/sd-scripts/pull/355) ãã芧ãã ããã | |
| ãã«ã¢ãã«ã®25åã®ãããã¯ã®éã¿ãæå®ã§ããŸããæåã®ãããã¯ã«è©²åœããLoRAã¯ååšããŸããããéå±€å¥LoRAé©çšçãšã®äºææ§ã®ããã«25åãšããŠããŸãããŸãconv2d3x3ã«æ¡åŒµããªãå Žåãäžéšã®ãããã¯ã«ã¯LoRAãååšããŸããããèšè¿°ãçµ±äžããããåžžã«25åã®å€ãæå®ããŠãã ããã | |
| SDXL ã§ã¯ down/up 9 åãmiddle 3 åã®å€ãæå®ããŠãã ããã | |
| `--network_args` ã§ä»¥äžã®åŒæ°ãæå®ããŠãã ããã | |
| - `down_lr_weight` : U-Netã®down blocksã®åŠç¿çã®éã¿ãæå®ããŸãã以äžãæå®å¯èœã§ãã | |
| - ãããã¯ããšã®éã¿ : `"down_lr_weight=0,0,0,0,0,0,1,1,1,1,1,1"` ã®ããã«12åïŒSDXL ã§ã¯ 9 åïŒã®æ°å€ãæå®ããŸãã | |
| - ããªã»ããããã®æå® : `"down_lr_weight=sine"` ã®ããã«æå®ããŸãïŒãµã€ã³ã«ãŒãã§éã¿ãæå®ããŸãïŒãsine, cosine, linear, reverse_linear, zeros ãæå®å¯èœã§ãããŸã `"down_lr_weight=cosine+.25"` ã®ããã« `+æ°å€` ã远å ãããšãæå®ããæ°å€ãå ç®ããŸãïŒ0.25~1.25ã«ãªããŸãïŒã | |
| - `mid_lr_weight` : U-Netã®mid blockã®åŠç¿çã®éã¿ãæå®ããŸãã`"down_lr_weight=0.5"` ã®ããã«æ°å€ãäžã€ã ãæå®ããŸãïŒSDXL ã®å Žå㯠3 åïŒã | |
| - `up_lr_weight` : U-Netã®up blocksã®åŠç¿çã®éã¿ãæå®ããŸããdown_lr_weightãšåæ§ã§ãã | |
| - æå®ãçç¥ããéšåã¯1.0ãšããŠæ±ãããŸãããŸãéã¿ã0ã«ãããšãã®ãããã¯ã®LoRAã¢ãžã¥ãŒã«ã¯äœæãããŸããã | |
| - `block_lr_zero_threshold` : éã¿ããã®å€ä»¥äžã®å ŽåãLoRAã¢ãžã¥ãŒã«ãäœæããŸãããããã©ã«ãã¯0ã§ãã | |
| ### éå±€å¥åŠç¿çã³ãã³ãã©ã€ã³æå®äŸ: | |
| ```powershell | |
| --network_args "down_lr_weight=0.5,0.5,0.5,0.5,1.0,1.0,1.0,1.0,1.5,1.5,1.5,1.5" "mid_lr_weight=2.0" "up_lr_weight=1.5,1.5,1.5,1.5,1.0,1.0,1.0,1.0,0.5,0.5,0.5,0.5" | |
| --network_args "block_lr_zero_threshold=0.1" "down_lr_weight=sine+.5" "mid_lr_weight=1.5" "up_lr_weight=cosine+.5" | |
| ``` | |
| ### éå±€å¥åŠç¿çtomlãã¡ã€ã«æå®äŸ: | |
| ```toml | |
| network_args = [ "down_lr_weight=0.5,0.5,0.5,0.5,1.0,1.0,1.0,1.0,1.5,1.5,1.5,1.5", "mid_lr_weight=2.0", "up_lr_weight=1.5,1.5,1.5,1.5,1.0,1.0,1.0,1.0,0.5,0.5,0.5,0.5",] | |
| network_args = [ "block_lr_zero_threshold=0.1", "down_lr_weight=sine+.5", "mid_lr_weight=1.5", "up_lr_weight=cosine+.5", ] | |
| ``` | |
| ## éå±€å¥dim (rank) | |
| ãã«ã¢ãã«ã®25åã®ãããã¯ã®dim (rank)ãæå®ã§ããŸããéå±€å¥åŠç¿çãšåæ§ã«äžéšã®ãããã¯ã«ã¯LoRAãååšããªãå ŽåããããŸãããåžžã«25åã®å€ãæå®ããŠãã ããã | |
| SDXL ã§ã¯ 23 åã®å€ãæå®ããŠãã ãããäžéšã®ãããã¯ã«ã¯LoRA ãååšããŸãããã`sdxl_train.py` ã®[éå±€å¥åŠç¿ç](./train_SDXL-en.md) ãšã®äºææ§ã®ããã§ãã | |
| 察å¿ã¯ã`0: time/label embed, 1-9: input blocks 0-8, 10-12: mid blocks 0-2, 13-21: output blocks 0-8, 22: out` ã§ãã | |
| `--network_args` ã§ä»¥äžã®åŒæ°ãæå®ããŠãã ããã | |
| - `block_dims` : åãããã¯ã®dim (rank)ãæå®ããŸãã`"block_dims=2,2,2,2,4,4,4,4,6,6,6,6,8,6,6,6,6,4,4,4,4,2,2,2,2"` ã®ããã«25åã®æ°å€ãæå®ããŸãã | |
| - `block_alphas` : åãããã¯ã®alphaãæå®ããŸããblock_dimsãšåæ§ã«25åã®æ°å€ãæå®ããŸããçç¥æã¯network_alphaã®å€ã䜿çšãããŸãã | |
| - `conv_block_dims` : LoRAãConv2d 3x3ã«æ¡åŒµããåãããã¯ã®dim (rank)ãæå®ããŸãã | |
| - `conv_block_alphas` : LoRAãConv2d 3x3ã«æ¡åŒµãããšãã®åãããã¯ã®alphaãæå®ããŸããçç¥æã¯conv_alphaã®å€ã䜿çšãããŸãã | |
| ### éå±€å¥dim (rank)ã³ãã³ãã©ã€ã³æå®äŸ: | |
| ```powershell | |
| --network_args "block_dims=2,4,4,4,8,8,8,8,12,12,12,12,16,12,12,12,12,8,8,8,8,4,4,4,2" | |
| --network_args "block_dims=2,4,4,4,8,8,8,8,12,12,12,12,16,12,12,12,12,8,8,8,8,4,4,4,2" "conv_block_dims=2,2,2,2,4,4,4,4,6,6,6,6,8,6,6,6,6,4,4,4,4,2,2,2,2" | |
| --network_args "block_dims=2,4,4,4,8,8,8,8,12,12,12,12,16,12,12,12,12,8,8,8,8,4,4,4,2" "block_alphas=2,2,2,2,4,4,4,4,6,6,6,6,8,6,6,6,6,4,4,4,4,2,2,2,2" | |
| ``` | |
| ### éå±€å¥dim (rank)tomlãã¡ã€ã«æå®äŸ: | |
| ```toml | |
| network_args = [ "block_dims=2,4,4,4,8,8,8,8,12,12,12,12,16,12,12,12,12,8,8,8,8,4,4,4,2",] | |
| network_args = [ "block_dims=2,4,4,4,8,8,8,8,12,12,12,12,16,12,12,12,12,8,8,8,8,4,4,4,2", "block_alphas=2,2,2,2,4,4,4,4,6,6,6,6,8,6,6,6,6,4,4,4,4,2,2,2,2",] | |
| ``` | |
| # ãã®ä»ã®ã¹ã¯ãªãã | |
| ããŒãžçLoRAã«é¢é£ããã¹ã¯ãªãã矀ã§ãã | |
| ## ããŒãžã¹ã¯ãªããã«ã€ã㊠| |
| merge_lora.pyã§Stable Diffusionã®ã¢ãã«ã«LoRAã®åŠç¿çµæãããŒãžããããè€æ°ã®LoRAã¢ãã«ãããŒãžãããã§ããŸãã | |
| SDXLåãã«ã¯sdxl_merge_lora.pyãçšæããŠããŸãããªãã·ã§ã³çã¯åäžã§ãã®ã§ã以äžã®merge_lora.pyãèªã¿æ¿ããŠãã ããã | |
| ### Stable Diffusionã®ã¢ãã«ã«LoRAã®ã¢ãã«ãããŒãžãã | |
| ããŒãžåŸã®ã¢ãã«ã¯éåžžã®Stable Diffusionã®ckptãšåæ§ã«æ±ããŸããããšãã°ä»¥äžã®ãããªã³ãã³ãã©ã€ã³ã«ãªããŸãã | |
| ``` | |
| python networks\merge_lora.py --sd_model ..\model\model.ckpt | |
| --save_to ..\lora_train1\model-char1-merged.safetensors | |
| --models ..\lora_train1\last.safetensors --ratios 0.8 | |
| ``` | |
| Stable Diffusion v2.xã®ã¢ãã«ã§åŠç¿ããããã«ããŒãžããå Žåã¯ã--v2ãªãã·ã§ã³ãæå®ããŠãã ããã | |
| --sd_modelãªãã·ã§ã³ã«ããŒãžã®å ãšãªãStable Diffusionã®ã¢ãã«ãã¡ã€ã«ãæå®ããŸãïŒ.ckptãŸãã¯.safetensorsã®ã¿å¯Ÿå¿ã§ãDiffusersã¯ä»ã®ãšãã察å¿ããŠããŸããïŒã | |
| --save_toãªãã·ã§ã³ã«ããŒãžåŸã®ã¢ãã«ã®ä¿åå ãæå®ããŸãïŒ.ckptãŸãã¯.safetensorsãæ¡åŒµåã§èªåå€å®ïŒã | |
| --modelsã«åŠç¿ããLoRAã®ã¢ãã«ãã¡ã€ã«ãæå®ããŸããè€æ°æå®ãå¯èœã§ããã®æã¯é ã«ããŒãžããŸãã | |
| --ratiosã«ããããã®ã¢ãã«ã®é©çšçïŒã©ã®ãããéã¿ãå ã¢ãã«ã«åæ ãããïŒã0~1.0ã®æ°å€ã§æå®ããŸããäŸãã°éåŠç¿ã«è¿ããããªå Žåã¯ãé©çšçãäžãããšãã·ã«ãªããããããŸãããã¢ãã«ã®æ°ãšåãã ãæå®ããŠãã ããã | |
| è€æ°æå®æã¯ä»¥äžã®ããã«ãªããŸãã | |
| ``` | |
| python networks\merge_lora.py --sd_model ..\model\model.ckpt | |
| --save_to ..\lora_train1\model-char1-merged.safetensors | |
| --models ..\lora_train1\last.safetensors ..\lora_train2\last.safetensors --ratios 0.8 0.5 | |
| ``` | |
| ### è€æ°ã®LoRAã®ã¢ãã«ãããŒãžãã | |
| --concatãªãã·ã§ã³ãæå®ãããšãè€æ°ã®LoRAãåçŽã«çµåããŠæ°ããLoRAã¢ãã«ãäœæã§ããŸãããã¡ã€ã«ãµã€ãºïŒããã³dim/rankïŒã¯æå®ããLoRAã®åèšãµã€ãºã«ãªããŸãïŒããŒãžæã«dim (rank)ã倿Žããå Žå㯠`svd_merge_lora.py` ã䜿çšããŠãã ããïŒã | |
| ããšãã°ä»¥äžã®ãããªã³ãã³ãã©ã€ã³ã«ãªããŸãã | |
| ``` | |
| python networks\merge_lora.py --save_precision bf16 | |
| --save_to ..\lora_train1\model-char1-style1-merged.safetensors | |
| --models ..\lora_train1\last.safetensors ..\lora_train2\last.safetensors | |
| --ratios 1.0 -1.0 --concat --shuffle | |
| ``` | |
| --concatãªãã·ã§ã³ãæå®ããŸãã | |
| ãŸã--shuffleãªãã·ã§ã³ã远å ããéã¿ãã·ã£ããã«ããŸããã·ã£ããã«ããªããšããŒãžåŸã®LoRAããå ã®LoRAãåãåºãããããã³ããŒæ©åŠç¿ãªã©ã®å Žåã«ã¯åŠç¿å ããŒã¿ãæããã«ãªããŸãããæ³šæãã ããã | |
| --save_toãªãã·ã§ã³ã«ããŒãžåŸã®LoRAã¢ãã«ã®ä¿åå ãæå®ããŸãïŒ.ckptãŸãã¯.safetensorsãæ¡åŒµåã§èªåå€å®ïŒã | |
| --modelsã«åŠç¿ããLoRAã®ã¢ãã«ãã¡ã€ã«ãæå®ããŸããäžã€ä»¥äžãæå®å¯èœã§ãã | |
| --ratiosã«ããããã®ã¢ãã«ã®æ¯çïŒã©ã®ãããéã¿ãå ã¢ãã«ã«åæ ãããïŒã0~1.0ã®æ°å€ã§æå®ããŸããäºã€ã®ã¢ãã«ãäžå¯Ÿäžã§ããŒãžããå Žåã¯ãã0.5 0.5ãã«ãªããŸããã1.0 1.0ãã§ã¯åèšã®éã¿ã倧ãããªããããŠãæããçµæã¯ããŸãæãŸãããªããã®ã«ãªããšæãããŸãã | |
| v1ã§åŠç¿ããLoRAãšv2ã§åŠç¿ããLoRAãrankïŒæ¬¡å æ°ïŒã®ç°ãªãLoRAã¯ããŒãžã§ããŸãããU-Netã ãã®LoRAãšU-Net+Text Encoderã®LoRAã¯ããŒãžã§ããã¯ãã§ãããçµæã¯æªç¥æ°ã§ãã | |
| ### ãã®ä»ã®ãªãã·ã§ã³ | |
| * precision | |
| * ããŒãžèšç®æã®ç²ŸåºŠãfloatãfp16ãbf16ããæå®ã§ããŸããçç¥æã¯ç²ŸåºŠã確ä¿ããããfloatã«ãªããŸããã¡ã¢ãªäœ¿çšéãæžããããå Žåã¯fp16/bf16ãæå®ããŠãã ããã | |
| * save_precision | |
| * ã¢ãã«ä¿åæã®ç²ŸåºŠãfloatãfp16ãbf16ããæå®ã§ããŸããçç¥æã¯precisionãšåã粟床ã«ãªããŸãã | |
| ä»ã«ãããã€ãã®ãªãã·ã§ã³ããããŸãã®ã§ã--helpã§ç¢ºèªããŠãã ããã | |
| ## è€æ°ã®rankãç°ãªãLoRAã®ã¢ãã«ãããŒãžãã | |
| è€æ°ã®LoRAãã²ãšã€ã®LoRAã§è¿äŒŒããŸãïŒå®å šãªåçŸã¯ã§ããŸããïŒã`svd_merge_lora.py`ãçšããŸããããšãã°ä»¥äžã®ãããªã³ãã³ãã©ã€ã³ã«ãªããŸãã | |
| ``` | |
| python networks\svd_merge_lora.py | |
| --save_to ..\lora_train1\model-char1-style1-merged.safetensors | |
| --models ..\lora_train1\last.safetensors ..\lora_train2\last.safetensors | |
| --ratios 0.6 0.4 --new_rank 32 --device cuda | |
| ``` | |
| `merge_lora.py` ãšäž»ãªãªãã·ã§ã³ã¯åäžã§ãã以äžã®ãªãã·ã§ã³ã远å ãããŠããŸãã | |
| - `--new_rank` | |
| - äœæããLoRAã®rankãæå®ããŸãã | |
| - `--new_conv_rank` | |
| - äœæãã Conv2d 3x3 LoRA ã® rank ãæå®ããŸããçç¥æã¯ `new_rank` ãšåãã«ãªããŸãã | |
| - `--device` | |
| - `--device cuda`ãšããŠcudaãæå®ãããšèšç®ãGPUäžã§è¡ããŸããåŠçãéããªããŸãã | |
| ## åœãªããžããªå ã®ç»åçæã¹ã¯ãªããã§çæãã | |
| gen_img.pyã«ã--network_moduleã--network_weightsã®åãªãã·ã§ã³ã远å ããŠãã ãããæå³ã¯åŠç¿æãšåæ§ã§ãã詳现ã¯[ç»åçæã¹ã¯ãªããã®ããã¥ã¡ã³ã](gen_img_README-ja.md)ãåç §ããŠãã ããã | |
| --network_mulãªãã·ã§ã³ã§0~1.0ã®æ°å€ãæå®ãããšãLoRAã®é©çšçãå€ããããŸãã | |
| ## Diffusersã®pipelineã§çæãã | |
| 以äžã®äŸãåèã«ããŠãã ãããå¿ èŠãªãã¡ã€ã«ã¯networks/lora.pyã®ã¿ã§ããDiffusersã®ããŒãžã§ã³ã¯0.10.2以å€ã§ã¯åäœããªãå¯èœæ§ããããŸãã | |
| ```python | |
| import torch | |
| from diffusers import StableDiffusionPipeline | |
| from networks.lora import LoRAModule, create_network_from_weights | |
| from safetensors.torch import load_file | |
| # if the ckpt is CompVis based, convert it to Diffusers beforehand with tools/convert_diffusers20_original_sd.py. See --help for more details. | |
| model_id_or_dir = r"model_id_on_hugging_face_or_dir" | |
| device = "cuda" | |
| # create pipe | |
| print(f"creating pipe from {model_id_or_dir}...") | |
| pipe = StableDiffusionPipeline.from_pretrained(model_id_or_dir, revision="fp16", torch_dtype=torch.float16) | |
| pipe = pipe.to(device) | |
| vae = pipe.vae | |
| text_encoder = pipe.text_encoder | |
| unet = pipe.unet | |
| # load lora networks | |
| print(f"loading lora networks...") | |
| lora_path1 = r"lora1.safetensors" | |
| sd = load_file(lora_path1) # If the file is .ckpt, use torch.load instead. | |
| network1, sd = create_network_from_weights(0.5, None, vae, text_encoder,unet, sd) | |
| network1.apply_to(text_encoder, unet) | |
| network1.load_state_dict(sd) | |
| network1.to(device, dtype=torch.float16) | |
| # # You can merge weights instead of apply_to+load_state_dict. network.set_multiplier does not work | |
| # network.merge_to(text_encoder, unet, sd) | |
| lora_path2 = r"lora2.safetensors" | |
| sd = load_file(lora_path2) | |
| network2, sd = create_network_from_weights(0.7, None, vae, text_encoder,unet, sd) | |
| network2.apply_to(text_encoder, unet) | |
| network2.load_state_dict(sd) | |
| network2.to(device, dtype=torch.float16) | |
| lora_path3 = r"lora3.safetensors" | |
| sd = load_file(lora_path3) | |
| network3, sd = create_network_from_weights(0.5, None, vae, text_encoder,unet, sd) | |
| network3.apply_to(text_encoder, unet) | |
| network3.load_state_dict(sd) | |
| network3.to(device, dtype=torch.float16) | |
| # prompts | |
| prompt = "masterpiece, best quality, 1girl, in white shirt, looking at viewer" | |
| negative_prompt = "bad quality, worst quality, bad anatomy, bad hands" | |
| # exec pipe | |
| print("generating image...") | |
| with torch.autocast("cuda"): | |
| image = pipe(prompt, guidance_scale=7.5, negative_prompt=negative_prompt).images[0] | |
| # if not merged, you can use set_multiplier | |
| # network1.set_multiplier(0.8) | |
| # and generate image again... | |
| # save image | |
| image.save(r"by_diffusers..png") | |
| ``` | |
| ## äºã€ã®ã¢ãã«ã®å·®åããLoRAã¢ãã«ãäœæãã | |
| [ãã¡ãã®ãã£ã¹ã«ãã·ã§ã³](https://github.com/cloneofsimo/lora/discussions/56)ãåèã«å®è£ ãããã®ã§ããæ°åŒã¯ãã®ãŸãŸäœ¿ãããŠããã ããŸããïŒããçè§£ããŠããŸãããè¿äŒŒã«ã¯ç¹ç°å€åè§£ãçšããããã§ãïŒã | |
| äºã€ã®ã¢ãã«ïŒããšãã°fine tuningã®å ã¢ãã«ãšfine tuningåŸã®ã¢ãã«ïŒã®å·®åããLoRAã§è¿äŒŒããŸãã | |
| ### ã¹ã¯ãªããã®å®è¡æ¹æ³ | |
| 以äžã®ããã«æå®ããŠãã ããã | |
| ``` | |
| python networks\extract_lora_from_models.py --model_org base-model.ckpt | |
| --model_tuned fine-tuned-model.ckpt | |
| --save_to lora-weights.safetensors --dim 4 | |
| ``` | |
| --model_orgãªãã·ã§ã³ã«å ã®Stable Diffusionã¢ãã«ãæå®ããŸããäœæããLoRAã¢ãã«ãé©çšããå Žåã¯ããã®ã¢ãã«ãæå®ããŠé©çšããããšã«ãªããŸãã.ckptãŸãã¯.safetensorsãæå®ã§ããŸãã | |
| --model_tunedãªãã·ã§ã³ã«å·®åãæœåºãã察象ã®Stable Diffusionã¢ãã«ãæå®ããŸããããšãã°fine tuningãDreamBoothåŸã®ã¢ãã«ãæå®ããŸãã.ckptãŸãã¯.safetensorsãæå®ã§ããŸãã | |
| --save_toã«LoRAã¢ãã«ã®ä¿åå ãæå®ããŸãã--dimã«LoRAã®æ¬¡å æ°ãæå®ããŸãã | |
| çæãããLoRAã¢ãã«ã¯ãåŠç¿ããLoRAã¢ãã«ãšåæ§ã«äœ¿çšã§ããŸãã | |
| Text Encoderãäºã€ã®ã¢ãã«ã§åãå Žåã«ã¯LoRAã¯U-Netã®ã¿ã®LoRAãšãªããŸãã | |
| ### ãã®ä»ã®ãªãã·ã§ã³ | |
| - `--v2` | |
| - v2.xã®Stable Diffusionã¢ãã«ã䜿ãå Žåã«æå®ããŠãã ããã | |
| - `--device` | |
| - ``--device cuda``ãšããŠcudaãæå®ãããšèšç®ãGPUäžã§è¡ããŸããåŠçãéããªããŸãïŒCPUã§ããããŸã§é ããªããããããããåïœæ°åçšåºŠã®ããã§ãïŒã | |
| - `--save_precision` | |
| - LoRAã®ä¿å圢åŒã"float", "fp16", "bf16"ããæå®ããŸããçç¥æã¯floatã«ãªããŸãã | |
| - `--conv_dim` | |
| - æå®ãããšLoRAã®é©çšç¯å²ã Conv2d 3x3 ãžæ¡å€§ããŸããConv2d 3x3 ã® rank ãæå®ããŸãã | |
| ## ç»åãªãµã€ãºã¹ã¯ãªãã | |
| ïŒã®ã¡ã»ã©ããã¥ã¡ã³ããæŽçããŸãããšããããããã«èª¬æãæžããŠãããŸããïŒ | |
| Aspect Ratio Bucketingã®æ©èœæ¡åŒµã§ãå°ããªç»åã«ã€ããŠã¯æ¡å€§ããªãã§ãã®ãŸãŸæåž«ããŒã¿ãšããããšãå¯èœã«ãªããŸãããå ã®æåž«ç»åãçž®å°ããç»åããæåž«ããŒã¿ã«å ãããšç²ŸåºŠãåäžãããšããå ±åãšãšãã«ååŠççšã®ã¹ã¯ãªãããããã ããŸããã®ã§æŽåããŠè¿œå ããŸãããbmaltaisæ°ã«æè¬ããŸãã | |
| ### ã¹ã¯ãªããã®å®è¡æ¹æ³ | |
| 以äžã®ããã«æå®ããŠãã ãããå ã®ç»åãã®ãŸãŸãããã³ãªãµã€ãºåŸã®ç»åã倿å ãã©ã«ãã«ä¿åãããŸãããªãµã€ãºåŸã®ç»åã«ã¯ããã¡ã€ã«åã« ``+512x512`` ã®ããã«ãªãµã€ãºå ã®è§£å床ãä»ãå ããããŸãïŒç»åãµã€ãºãšã¯ç°ãªããŸãïŒããªãµã€ãºå ã®è§£å床ããå°ããç»åã¯æ¡å€§ãããããšã¯ãããŸããã | |
| ``` | |
| python tools\resize_images_to_resolution.py --max_resolution 512x512,384x384,256x256 --save_as_png | |
| --copy_associated_files å ç»åãã©ã«ã 倿å ãã©ã«ã | |
| ``` | |
| å ç»åãã©ã«ãå ã®ç»åãã¡ã€ã«ããæå®ããè§£å床ïŒè€æ°æå®å¯ïŒãšåãé¢ç©ã«ãªãããã«ãªãµã€ãºããã倿å ãã©ã«ãã«ä¿åãããŸããç»å以å€ã®ãã¡ã€ã«ã¯ãã®ãŸãŸã³ããŒãããŸãã | |
| ``--max_resolution`` ãªãã·ã§ã³ã«ãªãµã€ãºå ã®ãµã€ãºãäŸã®ããã«æå®ããŠãã ãããé¢ç©ããã®ãµã€ãºã«ãªãããã«ãªãµã€ãºããŸããè€æ°æå®ãããšãããããã®è§£å床ã§ãªãµã€ãºãããŸãã``512x512,384x384,256x256``ãªãã倿å ãã©ã«ãã®ç»åã¯ãå ãµã€ãºãšãªãµã€ãºåŸãµã€ãºÃ3ã®èš4æã«ãªããŸãã | |
| ``--save_as_png`` ãªãã·ã§ã³ãæå®ãããšpng圢åŒã§ä¿åããŸããçç¥ãããšjpeg圢åŒïŒquality=100ïŒã§ä¿åãããŸãã | |
| ``--copy_associated_files`` ãªãã·ã§ã³ãæå®ãããšãæ¡åŒµåãé€ãç»åãšåããã¡ã€ã«åïŒããšãã°ãã£ãã·ã§ã³ãªã©ïŒã®ãã¡ã€ã«ãããªãµã€ãºåŸã®ç»åã®ãã¡ã€ã«åãšåãååã§ã³ããŒãããŸãã | |
| ### ãã®ä»ã®ãªãã·ã§ã³ | |
| - divisible_by | |
| - ãªãµã€ãºåŸã®ç»åã®ãµã€ãºïŒçžŠã暪ã®ããããïŒããã®å€ã§å²ãåããããã«ãç»åäžå¿ãåãåºããŸãã | |
| - interpolation | |
| - çž®å°æã®è£å®æ¹æ³ãæå®ããŸãã``area, cubic, lanczos4``ããéžæå¯èœã§ãããã©ã«ãã¯``area``ã§ãã | |
| # è¿œå æ å ± | |
| ## cloneofsimoæ°ã®ãªããžããªãšã®éã | |
| 2022/12/25æç¹ã§ã¯ãåœãªããžããªã¯LoRAã®é©çšåæãText Encoderã®MLPãU-Netã®FFNãTransformerã®in/out projectionã«æ¡å€§ãã衚çŸåãå¢ããŠããŸãããã ãã®ä»£ããã¡ã¢ãªäœ¿çšéã¯å¢ãã8GBããããã«ãªããŸããã | |
| ãŸãã¢ãžã¥ãŒã«å ¥ãæ¿ãæ©æ§ã¯å šãç°ãªããŸãã | |
| ## å°æ¥æ¡åŒµã«ã€ã㊠| |
| LoRAã ãã§ãªãä»ã®æ¡åŒµã«ã察å¿å¯èœã§ãã®ã§ããããã远å äºå®ã§ãã | |