Image-to-Image
Diffusion Single File
Safetensors
English
moss-image
tile
controlnet
lllite
image-restoration
comfyui
adapter
Instructions to use Xabsurd/MOSS-Image-Tile-Control with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusion Single File
How to use Xabsurd/MOSS-Image-Tile-Control with Diffusion Single File:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
|
Download RESEARCH_REPORT.md from Xabsurd/MOSS-Image-Tile-Control: direct link, hf CLI and curl.
- Browser
- Download file 6.8 kB
-
https://huggingface.co/Xabsurd/MOSS-Image-Tile-Control/resolve/main/RESEARCH_REPORT.md
- Command line
-
hf download hf://Xabsurd/MOSS-Image-Tile-Control/RESEARCH_REPORT.md
-
curl -L -o RESEARCH_REPORT.md https://huggingface.co/Xabsurd/MOSS-Image-Tile-Control/resolve/main/RESEARCH_REPORT.md
6.8 kB
| # MOSS-Image Tile Control: Development and Experimental Report | |
| This project explores continuously adjustable image control for MOSS-Image, | |
| balancing refinement after upscaling, subject-feature preservation, color | |
| retention, and inference speed. **V1 is the final design established through four | |
| iterations.** beta2 and beta3 remain available as earlier approaches with different | |
| trade-offs. | |
| ## 1. The initial problem | |
| The initial approach combined concatenated image conditioning with a backbone | |
| LoRA. It provided strong control, but setting image-conditioning strength to zero | |
| did not undo the LoRA's effect on the backbone. Disabling the image condition and | |
| removing the controller entirely were therefore different operations. | |
| This led to a concrete requirement: strength should scale an independent control | |
| path. At zero strength, the node should return the input model directly and skip | |
| control-image encoding. Subsequent implementations explicitly tested this behavior. | |
| ## 2. Released approaches and practical trade-offs | |
| | Approach | Architecture | Observations and intended use | | |
| | --- | --- | --- | | |
| | beta2 | Concatenated image conditioning and backbone LoRA | Best suited to high control strength and minimal changes. Preserves original features most strongly, but does not repair fine details; slowest in the tested workflows. Zero conditioning strength does not remove the LoRA's effect | | |
| | beta3 | VAE-latent conditioning and an independent low-rank residual branch | Resolves the zero-strength backbone-modification issue. Strong structural control and the fastest option, but tends to smooth high-frequency detail; currently not the general restoration recommendation | | |
| | V1 | RGB conditioning encoder and LLLite adapters at the image-token Q/K/V projection inputs | Best clarity and detail enhancement in the author's visual comparisons, while preserving subject features and retaining colors well. The author's personal choice for overall quality | | |
| The project underwent four design iterations; this table summarizes the three | |
| versions retained for release. These observations come from the author's actual | |
| workflows. The approaches also use different Base/SFT backbones, so speed and | |
| quality differences cannot be attributed entirely to the control architecture. | |
| No standardized cross-model benchmark has been established. | |
| The author chose V1 primarily for clarity and detail enhancement, rather than | |
| maximum similarity to the input alone. beta2's strong preservation behaves more | |
| like a conservative upscaler, leaving less room for generative refinement. Its | |
| LoRA changes backbone diffusion-layer weights as an implementation fact. Whether | |
| this causes its weaker detail enhancement remains a hypothesis: no ablation has | |
| isolated that relationship, and the backbone, training, and control strength may | |
| also contribute. | |
| ## 3. Investigating color shifts | |
| Early experiments showed changes in color, saturation, and contrast. In the | |
| tested workflow, a VAE encode/decode round trip produced no obvious color shift, | |
| and disabling the acceleration LoRA did not noticeably reduce it. Resampling | |
| still changed colors with the control branch disabled. This indicates that the | |
| backbone's regeneration behavior is also a source of color changes; the control | |
| branch cannot account for every shift. | |
| Experiments included color-preservation fine-tuning from existing residual | |
| checkpoints and higher-learning-rate trials. Improvements were inconsistent | |
| across checkpoints. More training or stronger color constraints did not reliably | |
| improve overall appearance. Selection therefore remained based on actual visual | |
| comparisons. Color Match is retained as optional post-processing; color | |
| preservation is not considered fully solved. | |
| ## 4. Adapting LLLite and selecting the final design | |
| The LLLite approach reuses ComfyUI's native Anima LLLite conditioning encoder and | |
| adapter modules, adapting them to MOSS token layout, joint attention, and layer | |
| count. Control acts on image-token inputs to the Q/K/V projections. Alignment | |
| refers to the module math and injection location, not identical backbones or | |
| training outcomes. | |
| In a module-level numerical check using identical weights, conditioning-encoder | |
| outputs were bitwise identical. The Q/K/V control-path maximum absolute error | |
| was approximately `1.2e-6`, within floating-point arithmetic differences. This | |
| does not establish equivalent end-to-end generation quality. | |
| Early LLLite results were clearer, but color, style, and detail quality initially | |
| fell short of expectations. Further training improved visual results. V1 was | |
| then chosen through repeated visual comparisons. This improvement supports the | |
| interpretation that early training was insufficient, but does not conclusively | |
| establish underfitting or imply that later checkpoints are always better. | |
| ## 5. Inference engineering and validation | |
| The nodes attach control through ComfyUI's ModelPatcher, additional-model loading, | |
| and block-replacement interfaces, retaining native memory management. Zero | |
| strength bypasses control without merging controller weights into the backbone. | |
| The display names are **Load MOSS-LLLite** and **Apply MOSS-LLLite**; internal IDs | |
| are preserved for existing workflows. | |
| An inference optimization replaced three complete fused-QKV projections with | |
| the required output rows for each projection, reusing native casting and | |
| quantization paths. In one warmed-up, 1024-resolution, two-step sampling test | |
| with CFG 3 and an INT8/ConvRot backbone, time fell from approximately `1.94 s` to | |
| `1.52 s`, with bitwise-identical final latents. This is evidence for that local | |
| optimization under the tested settings, not a fixed speedup for every workflow. | |
| A separate attempt to reduce RGB encoder batch computation was rejected because | |
| it changed BF16 numerical results. Preserving generation results constrained | |
| optimization decisions, rather than runtime alone. | |
| ## 6. Conclusions and limits | |
| The released options offer distinct trade-offs: **beta2 prioritizes preservation, | |
| beta3 prioritizes speed, and V1 prioritizes refinement and overall appearance.** | |
| V1 is the final design established through four iterations and repeated visual | |
| testing, and the author's personal choice across clarity, detail enhancement, | |
| subject features, and color retention. | |
| Available evidence includes module-level numerical comparisons, node-behavior | |
| tests, a limited performance benchmark, and visual generation comparisons. | |
| Standardized evaluations of perceptual quality, color error, feature consistency, | |
| and variation across random seeds are still missing. Methodological novelty has | |
| not been established through a comprehensive literature comparison. This report | |
| documents an engineering exploration; subjective observations are not presented | |
| as rigorous academic findings. | |