Image-to-Image
PEFT
Safetensors
Chinese
English
moss-image
rgba
transparent-background
vae
lora
mage
Instructions to use leafmoone/Mage-Flow-RGBA-vae with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use leafmoone/Mage-Flow-RGBA-vae with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Rewrite README: weights + metrics only, drop samples
Browse files
README.md
CHANGED
|
@@ -17,54 +17,99 @@ library_name: peft
|
|
| 17 |
inference: false
|
| 18 |
---
|
| 19 |
|
| 20 |
-
#
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
## 文件
|
| 23 |
|
| 24 |
| 路径 | 说明 |
|
| 25 |
| --- | --- |
|
| 26 |
-
| `vae/
|
| 27 |
| `vae/mage_config.json` | 模型结构配置 |
|
| 28 |
| `vae/weights.json` | 合并前后数值一致性报告 |
|
| 29 |
| `checkpoint-00014000/lora/adapter_model.safetensors` | 未合并的 LoRA 适配器(43.7 MB,rank 32) |
|
| 30 |
| `checkpoint-00014000/lora/adapter_config.json` | PEFT/LoRA 配置 |
|
|
|
|
| 31 |
| `checkpoint-00014000/progress.json` | 训练进度(step 14000,next_batch 44000) |
|
|
|
|
| 32 |
|
| 33 |
## 训练指标
|
| 34 |
|
| 35 |
-
累计到 **14310** 步;**最后一个完整保存的 checkpoint 是 step 14000**。
|
|
|
|
|
|
|
| 36 |
|
| 37 |
-
| 指标
|
| 38 |
| --- | ---: | ---: | ---: |
|
| 39 |
-
| Total loss | 0.010546 | 0.008037 |
|
| 40 |
-
| Alpha MAE | 0.011672 | 0.008140 |
|
| 41 |
-
| AB-MSE | 0.001462 | 0.001092 |
|
| 42 |
-
| LPIPS | 0.009080 | 0.006940 |
|
| 43 |
| Latent RMS | 1.0184 | 1.0378 | +1.9% |
|
| 44 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
| PSNR ↑ | 29.99 dB | 31.12 dB | 31.12 dB @ 13500 |
|
| 50 |
-
| 黑底 PSNR ↑ | 31.98 dB | 32.70 dB | — |
|
| 51 |
-
| 白底 PSNR ↑ | 28.01 dB | 29.54 dB | — |
|
| 52 |
|
| 53 |
-
|
| 54 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 55 |
|
|
|
|
| 56 |
|
| 57 |
-
|
| 58 |
|
| 59 |
-
|
|
|
|
|
|
|
| 60 |
|
| 61 |
-
|
| 62 |
|
| 63 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
|
|
|
|
| 65 |
|
| 66 |
## 许可与致谢
|
| 67 |
|
| 68 |
-
基于[
|
|
|
|
| 69 |
[CKMoss-Image-RGBA](https://github.com/Chenkin-x/CKMoss-Image-RGBA)(GPLv3)。
|
| 70 |
本项目采用 GPLv3,见 `LICENSE` 与 `NOTICE`。
|
|
|
|
| 17 |
inference: false
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# CKMoss-Image-RGBA VAE — step 14000 continuation
|
| 21 |
+
|
| 22 |
+
本仓库发布 **Mage RGBA VAE 的续训权重(step 14000)**。结构参考
|
| 23 |
+
[ChenkinLab/CKMoss-Image-RGBA](https://huggingface.co/ChenkinLab/CKMoss-Image-RGBA)。
|
| 24 |
|
| 25 |
## 文件
|
| 26 |
|
| 27 |
| 路径 | 说明 |
|
| 28 |
| --- | --- |
|
| 29 |
+
| `vae/moss_rgba_vae.safetensors` | 已合并 LoRA 的 RGBA VAE 权重(404.6 MB),可直接加载 |
|
| 30 |
| `vae/mage_config.json` | 模型结构配置 |
|
| 31 |
| `vae/weights.json` | 合并前后数值一致性报告 |
|
| 32 |
| `checkpoint-00014000/lora/adapter_model.safetensors` | 未合并的 LoRA 适配器(43.7 MB,rank 32) |
|
| 33 |
| `checkpoint-00014000/lora/adapter_config.json` | PEFT/LoRA 配置 |
|
| 34 |
+
| `checkpoint-00014000/lora/mage_config.json` | 适配器对应的模型配置 |
|
| 35 |
| `checkpoint-00014000/progress.json` | 训练进度(step 14000,next_batch 44000) |
|
| 36 |
+
| `checkpoint-00014000/manifest.json` | 该 checkpoint 的文件清单与 SHA256 |
|
| 37 |
|
| 38 |
## 训练指标
|
| 39 |
|
| 40 |
+
从 step 3000 的官方 checkpoint 继续训练,累计到 **14310** 步;**最后一个完整保存的 checkpoint 是 step 14000**。
|
| 41 |
+
|
| 42 |
+
训练损失(1131 条日志,3010→14310,前 100 条 vs 后 100 条均值):
|
| 43 |
|
| 44 |
+
| 指标 | 前 100 条 | 后 100 条 | 变化 |
|
| 45 |
| --- | ---: | ---: | ---: |
|
| 46 |
+
| Total loss | 0.010546 | 0.008037 | -23.8% |
|
| 47 |
+
| Alpha MAE | 0.011672 | 0.008140 | -30.3% |
|
| 48 |
+
| AB-MSE | 0.001462 | 0.001092 | -25.3% |
|
| 49 |
+
| LPIPS | 0.009080 | 0.006940 | -23.6% |
|
| 50 |
| Latent RMS | 1.0184 | 1.0378 | +1.9% |
|
| 51 |
|
| 52 |
+
固定 8 张验证图(每 500 步一次,共 22 次):
|
| 53 |
+
|
| 54 |
+
| 指标 | step 3500 | step 10000 | step 11500 | step 14000 | 最佳 |
|
| 55 |
+
| --- | ---: | ---: | ---: | ---: | --- |
|
| 56 |
+
| Alpha MAE ↓ | 0.01498 | 0.01111 | 0.01107 | 0.01131 | 0.01107 @ 11500 |
|
| 57 |
+
| PSNR ↑ | 29.99 dB | 30.99 dB | 31.08 dB | 31.12 dB | 31.12 dB @ 13500 |
|
| 58 |
+
| 黑底 PSNR ↑ | 31.98 dB | 32.62 dB | 32.66 dB | 32.70 dB | 32.71 dB @ 13500 |
|
| 59 |
+
| 白底 PSNR ↑ | 28.01 dB | 29.36 dB | 29.50 dB | 29.54 dB | 29.54 dB @ 14000 |
|
| 60 |
+
|
| 61 |
+
验证集全过程(step: alpha_mae / psnr):
|
| 62 |
+
|
| 63 |
+
```text
|
| 64 |
+
3500: 0.014978 / 29.991 4000: 0.014611 / 30.022 4500: 0.015406 / 30.041
|
| 65 |
+
5000: 0.013589 / 30.379 5500: 0.014032 / 30.254 6000: 0.011927 / 30.572
|
| 66 |
+
6500: 0.012099 / 30.638 7000: 0.013357 / 30.563 7500: 0.013159 / 30.746
|
| 67 |
+
8000: 0.012951 / 30.753 8500: 0.011807 / 30.862 9000: 0.013030 / 30.866
|
| 68 |
+
9500: 0.012347 / 30.934 10000: 0.011113 / 30.991 10500: 0.011471 / 31.030
|
| 69 |
+
11000: 0.011390 / 31.054 11500: 0.011066 / 31.081 12000: 0.011170 / 31.091
|
| 70 |
+
12500: 0.011085 / 31.101 13000: 0.011291 / 31.107 13500: 0.011092 / 31.121
|
| 71 |
+
14000: 0.011312 / 31.118
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
训练配置:512px、bf16、micro-batch 8 × accumulation 4(global batch 32)、LoRA rank/alpha 32、
|
| 75 |
+
单卡 DCU,平均约 **4.79 秒/步**,峰值显存约 39.5 GiB。
|
| 76 |
+
优化器与随机状态文件只用于精确续训,推理不需要。
|
| 77 |
+
|
| 78 |
+
## 使用
|
| 79 |
|
| 80 |
+
```python
|
| 81 |
+
from huggingface_hub import hf_hub_download
|
| 82 |
+
from safetensors.torch import load_file
|
|
|
|
|
|
|
|
|
|
| 83 |
|
| 84 |
+
repo = "leafmoone/CKMoss-Image-RGBA-vae"
|
| 85 |
+
weights = hf_hub_download(repo, "vae/moss_rgba_vae.safetensors")
|
| 86 |
+
state = load_file(weights)
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
VAE 输入/输出均为 4 通道、归一化到 `[-1, 1]`,Alpha 在通道 3,`+1` 表示不透明。
|
| 90 |
|
| 91 |
+
## 已知限制
|
| 92 |
|
| 93 |
+
本仓库只包含 **VAE 阶段**的权重,**没有训练 DiT LoRA 阶段**。
|
| 94 |
|
| 95 |
+
- VAE 对真实 RGBA 输入的重建可以正确恢复 Alpha(见上面的验证指标,alpha MAE 约 0.011);
|
| 96 |
+
- 但 MOSS-Image 的 DiT 只生成 RGB 潜变量,不携带透明度信息,因此直接用文本生成时
|
| 97 |
+
**模型原生 alpha 接近 1(不透明)**,在提示词中写 transparent background 也无法改变这一点。
|
| 98 |
|
| 99 |
+
实测对照(同一 prompt、seed 47、30 steps、512×768、MOSS-Image-Base):
|
| 100 |
|
| 101 |
+
| 配置 | alpha 均值 | 透明像素比例 |
|
| 102 |
+
| --- | ---: | ---: |
|
| 103 |
+
| 本 VAE,无 DiT LoRA | 0.987 | 0% |
|
| 104 |
+
| 官方 CKMoss VAE,无 DiT LoRA | 0.827 | 0% |
|
| 105 |
+
| 本 VAE + 官方 DiT LoRA | 0.417 | 12.2% |
|
| 106 |
+
| 官方 CKMoss VAE + 官方 DiT LoRA | 0.242 | 72.6% |
|
| 107 |
|
| 108 |
+
结论:透明生成能力主要来自 DiT LoRA 阶段。若要原生透明输出,需要补训 DiT LoRA。
|
| 109 |
|
| 110 |
## 许可与致谢
|
| 111 |
|
| 112 |
+
基于 [MOSS-Image](https://github.com/OpenMOSS/MOSS-Image)(Apache-2.0)、
|
| 113 |
+
[Mage](https://github.com/microsoft/Mage)(MIT)与
|
| 114 |
[CKMoss-Image-RGBA](https://github.com/Chenkin-x/CKMoss-Image-RGBA)(GPLv3)。
|
| 115 |
本项目采用 GPLv3,见 `LICENSE` 与 `NOTICE`。
|