Image-to-Image
Diffusion Single File
Safetensors
English
moss-image
tile
controlnet
lllite
image-restoration
comfyui
adapter
Instructions to use Xabsurd/MOSS-Image-Tile-Control with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusion Single File
How to use Xabsurd/MOSS-Image-Tile-Control with Diffusion Single File:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Translate development report into English
Browse files- README.md +1 -1
- RESEARCH_REPORT.md +103 -42
README.md
CHANGED
|
@@ -192,7 +192,7 @@ different load/apply nodes.
|
|
| 192 |
## Development notes and credits
|
| 193 |
|
| 194 |
See [RESEARCH_REPORT.md](RESEARCH_REPORT.md) for a concise development report in
|
| 195 |
-
|
| 196 |
selection. [SHA256SUMS.txt](SHA256SUMS.txt) records hashes of the distributed weights
|
| 197 |
and node ZIP.
|
| 198 |
|
|
|
|
| 192 |
## Development notes and credits
|
| 193 |
|
| 194 |
See [RESEARCH_REPORT.md](RESEARCH_REPORT.md) for a concise development report in
|
| 195 |
+
English, including the architectural choices, color investigation, and checkpoint
|
| 196 |
selection. [SHA256SUMS.txt](SHA256SUMS.txt) records hashes of the distributed weights
|
| 197 |
and node ZIP.
|
| 198 |
|
RESEARCH_REPORT.md
CHANGED
|
@@ -1,49 +1,110 @@
|
|
| 1 |
-
# MOSS-Image Tile Control
|
| 2 |
|
| 3 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
|
| 5 |
-
## 1.
|
| 6 |
|
| 7 |
-
|
|
|
|
|
|
|
|
|
|
| 8 |
|
| 9 |
-
|
|
|
|
|
|
|
| 10 |
|
| 11 |
-
## 2.
|
| 12 |
|
| 13 |
-
|
|
| 14 |
| --- | --- | --- |
|
| 15 |
-
| beta2 |
|
| 16 |
-
| beta3 | VAE
|
| 17 |
-
| V1 | RGB
|
| 18 |
-
|
| 19 |
-
|
| 20 |
-
|
| 21 |
-
|
| 22 |
-
|
| 23 |
-
|
| 24 |
-
|
| 25 |
-
|
| 26 |
-
|
| 27 |
-
|
| 28 |
-
|
| 29 |
-
|
| 30 |
-
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
-
|
| 37 |
-
|
| 38 |
-
|
| 39 |
-
|
| 40 |
-
|
| 41 |
-
|
| 42 |
-
|
| 43 |
-
|
| 44 |
-
|
| 45 |
-
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
|
| 49 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# MOSS-Image Tile Control: Development and Experimental Report
|
| 2 |
|
| 3 |
+
This project explores continuously adjustable image control for MOSS-Image,
|
| 4 |
+
balancing refinement after upscaling, subject-feature preservation, color
|
| 5 |
+
retention, and inference speed. **V1 is the final design established through four
|
| 6 |
+
iterations.** beta2 and beta3 remain available as earlier approaches with different
|
| 7 |
+
trade-offs.
|
| 8 |
|
| 9 |
+
## 1. The initial problem
|
| 10 |
|
| 11 |
+
The initial approach combined concatenated image conditioning with a backbone
|
| 12 |
+
LoRA. It provided strong control, but setting image-conditioning strength to zero
|
| 13 |
+
did not undo the LoRA's effect on the backbone. Disabling the image condition and
|
| 14 |
+
removing the controller entirely were therefore different operations.
|
| 15 |
|
| 16 |
+
This led to a concrete requirement: strength should scale an independent control
|
| 17 |
+
path. At zero strength, the node should return the input model directly and skip
|
| 18 |
+
control-image encoding. Subsequent implementations explicitly tested this behavior.
|
| 19 |
|
| 20 |
+
## 2. Released approaches and practical trade-offs
|
| 21 |
|
| 22 |
+
| Approach | Architecture | Observations and intended use |
|
| 23 |
| --- | --- | --- |
|
| 24 |
+
| beta2 | Concatenated image conditioning and backbone LoRA | Best suited to high control strength and minimal changes. Preserves original features most strongly, but does not repair fine details; slowest in the tested workflows. Zero conditioning strength does not remove the LoRA's effect |
|
| 25 |
+
| beta3 | VAE-latent conditioning and an independent low-rank residual branch | Resolves the zero-strength backbone-modification issue. Strong structural control and the fastest option, but tends to smooth high-frequency detail; currently not the general restoration recommendation |
|
| 26 |
+
| V1 | RGB conditioning encoder and LLLite adapters at the image-token Q/K/V projection inputs | Best clarity and detail enhancement in the author's visual comparisons, while preserving subject features and retaining colors well. The author's personal choice for overall quality |
|
| 27 |
+
|
| 28 |
+
The project underwent four design iterations; this table summarizes the three
|
| 29 |
+
versions retained for release. These observations come from the author's actual
|
| 30 |
+
workflows. The approaches also use different Base/SFT backbones, so speed and
|
| 31 |
+
quality differences cannot be attributed entirely to the control architecture.
|
| 32 |
+
No standardized cross-model benchmark has been established.
|
| 33 |
+
|
| 34 |
+
The author chose V1 primarily for clarity and detail enhancement, rather than
|
| 35 |
+
maximum similarity to the input alone. beta2's strong preservation behaves more
|
| 36 |
+
like a conservative upscaler, leaving less room for generative refinement. Its
|
| 37 |
+
LoRA changes backbone diffusion-layer weights as an implementation fact. Whether
|
| 38 |
+
this causes its weaker detail enhancement remains a hypothesis: no ablation has
|
| 39 |
+
isolated that relationship, and the backbone, training, and control strength may
|
| 40 |
+
also contribute.
|
| 41 |
+
|
| 42 |
+
## 3. Investigating color shifts
|
| 43 |
+
|
| 44 |
+
Early experiments showed changes in color, saturation, and contrast. In the
|
| 45 |
+
tested workflow, a VAE encode/decode round trip produced no obvious color shift,
|
| 46 |
+
and disabling the acceleration LoRA did not noticeably reduce it. Resampling
|
| 47 |
+
still changed colors with the control branch disabled. This indicates that the
|
| 48 |
+
backbone's regeneration behavior is also a source of color changes; the control
|
| 49 |
+
branch cannot account for every shift.
|
| 50 |
+
|
| 51 |
+
Experiments included color-preservation fine-tuning from existing residual
|
| 52 |
+
checkpoints and higher-learning-rate trials. Improvements were inconsistent
|
| 53 |
+
across checkpoints. More training or stronger color constraints did not reliably
|
| 54 |
+
improve overall appearance. Selection therefore remained based on actual visual
|
| 55 |
+
comparisons. Color Match is retained as optional post-processing; color
|
| 56 |
+
preservation is not considered fully solved.
|
| 57 |
+
|
| 58 |
+
## 4. Adapting LLLite and selecting the final design
|
| 59 |
+
|
| 60 |
+
The LLLite approach reuses ComfyUI's native Anima LLLite conditioning encoder and
|
| 61 |
+
adapter modules, adapting them to MOSS token layout, joint attention, and layer
|
| 62 |
+
count. Control acts on image-token inputs to the Q/K/V projections. Alignment
|
| 63 |
+
refers to the module math and injection location, not identical backbones or
|
| 64 |
+
training outcomes.
|
| 65 |
+
|
| 66 |
+
In a module-level numerical check using identical weights, conditioning-encoder
|
| 67 |
+
outputs were bitwise identical. The Q/K/V control-path maximum absolute error
|
| 68 |
+
was approximately `1.2e-6`, within floating-point arithmetic differences. This
|
| 69 |
+
does not establish equivalent end-to-end generation quality.
|
| 70 |
+
|
| 71 |
+
Early LLLite results were clearer, but color, style, and detail quality initially
|
| 72 |
+
fell short of expectations. Further training improved visual results. V1 was
|
| 73 |
+
then chosen through repeated visual comparisons. This improvement supports the
|
| 74 |
+
interpretation that early training was insufficient, but does not conclusively
|
| 75 |
+
establish underfitting or imply that later checkpoints are always better.
|
| 76 |
+
|
| 77 |
+
## 5. Inference engineering and validation
|
| 78 |
+
|
| 79 |
+
The nodes attach control through ComfyUI's ModelPatcher, additional-model loading,
|
| 80 |
+
and block-replacement interfaces, retaining native memory management. Zero
|
| 81 |
+
strength bypasses control without merging controller weights into the backbone.
|
| 82 |
+
The display names are **Load MOSS-LLLite** and **Apply MOSS-LLLite**; internal IDs
|
| 83 |
+
are preserved for existing workflows.
|
| 84 |
+
|
| 85 |
+
An inference optimization replaced three complete fused-QKV projections with
|
| 86 |
+
the required output rows for each projection, reusing native casting and
|
| 87 |
+
quantization paths. In one warmed-up, 1024-resolution, two-step sampling test
|
| 88 |
+
with CFG 3 and an INT8/ConvRot backbone, time fell from approximately `1.94 s` to
|
| 89 |
+
`1.52 s`, with bitwise-identical final latents. This is evidence for that local
|
| 90 |
+
optimization under the tested settings, not a fixed speedup for every workflow.
|
| 91 |
+
|
| 92 |
+
A separate attempt to reduce RGB encoder batch computation was rejected because
|
| 93 |
+
it changed BF16 numerical results. Preserving generation results constrained
|
| 94 |
+
optimization decisions, rather than runtime alone.
|
| 95 |
+
|
| 96 |
+
## 6. Conclusions and limits
|
| 97 |
+
|
| 98 |
+
The released options offer distinct trade-offs: **beta2 prioritizes preservation,
|
| 99 |
+
beta3 prioritizes speed, and V1 prioritizes refinement and overall appearance.**
|
| 100 |
+
V1 is the final design established through four iterations and repeated visual
|
| 101 |
+
testing, and the author's personal choice across clarity, detail enhancement,
|
| 102 |
+
subject features, and color retention.
|
| 103 |
+
|
| 104 |
+
Available evidence includes module-level numerical comparisons, node-behavior
|
| 105 |
+
tests, a limited performance benchmark, and visual generation comparisons.
|
| 106 |
+
Standardized evaluations of perceptual quality, color error, feature consistency,
|
| 107 |
+
and variation across random seeds are still missing. Methodological novelty has
|
| 108 |
+
not been established through a comprehensive literature comparison. This report
|
| 109 |
+
documents an engineering exploration; subjective observations are not presented
|
| 110 |
+
as rigorous academic findings.
|