File size: 6,803 Bytes
f3fe995
08d4d84
f3fe995
 
 
 
 
08d4d84
f3fe995
08d4d84
f3fe995
 
 
 
08d4d84
f3fe995
 
 
08d4d84
f3fe995
08d4d84
f3fe995
08d4d84
f3fe995
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
# MOSS-Image Tile Control: Development and Experimental Report

This project explores continuously adjustable image control for MOSS-Image,
balancing refinement after upscaling, subject-feature preservation, color
retention, and inference speed. **V1 is the final design established through four
iterations.** beta2 and beta3 remain available as earlier approaches with different
trade-offs.

## 1. The initial problem

The initial approach combined concatenated image conditioning with a backbone
LoRA. It provided strong control, but setting image-conditioning strength to zero
did not undo the LoRA's effect on the backbone. Disabling the image condition and
removing the controller entirely were therefore different operations.

This led to a concrete requirement: strength should scale an independent control
path. At zero strength, the node should return the input model directly and skip
control-image encoding. Subsequent implementations explicitly tested this behavior.

## 2. Released approaches and practical trade-offs

| Approach | Architecture | Observations and intended use |
| --- | --- | --- |
| beta2 | Concatenated image conditioning and backbone LoRA | Best suited to high control strength and minimal changes. Preserves original features most strongly, but does not repair fine details; slowest in the tested workflows. Zero conditioning strength does not remove the LoRA's effect |
| beta3 | VAE-latent conditioning and an independent low-rank residual branch | Resolves the zero-strength backbone-modification issue. Strong structural control and the fastest option, but tends to smooth high-frequency detail; currently not the general restoration recommendation |
| V1 | RGB conditioning encoder and LLLite adapters at the image-token Q/K/V projection inputs | Best clarity and detail enhancement in the author's visual comparisons, while preserving subject features and retaining colors well. The author's personal choice for overall quality |

The project underwent four design iterations; this table summarizes the three
versions retained for release. These observations come from the author's actual
workflows. The approaches also use different Base/SFT backbones, so speed and
quality differences cannot be attributed entirely to the control architecture.
No standardized cross-model benchmark has been established.

The author chose V1 primarily for clarity and detail enhancement, rather than
maximum similarity to the input alone. beta2's strong preservation behaves more
like a conservative upscaler, leaving less room for generative refinement. Its
LoRA changes backbone diffusion-layer weights as an implementation fact. Whether
this causes its weaker detail enhancement remains a hypothesis: no ablation has
isolated that relationship, and the backbone, training, and control strength may
also contribute.

## 3. Investigating color shifts

Early experiments showed changes in color, saturation, and contrast. In the
tested workflow, a VAE encode/decode round trip produced no obvious color shift,
and disabling the acceleration LoRA did not noticeably reduce it. Resampling
still changed colors with the control branch disabled. This indicates that the
backbone's regeneration behavior is also a source of color changes; the control
branch cannot account for every shift.

Experiments included color-preservation fine-tuning from existing residual
checkpoints and higher-learning-rate trials. Improvements were inconsistent
across checkpoints. More training or stronger color constraints did not reliably
improve overall appearance. Selection therefore remained based on actual visual
comparisons. Color Match is retained as optional post-processing; color
preservation is not considered fully solved.

## 4. Adapting LLLite and selecting the final design

The LLLite approach reuses ComfyUI's native Anima LLLite conditioning encoder and
adapter modules, adapting them to MOSS token layout, joint attention, and layer
count. Control acts on image-token inputs to the Q/K/V projections. Alignment
refers to the module math and injection location, not identical backbones or
training outcomes.

In a module-level numerical check using identical weights, conditioning-encoder
outputs were bitwise identical. The Q/K/V control-path maximum absolute error
was approximately `1.2e-6`, within floating-point arithmetic differences. This
does not establish equivalent end-to-end generation quality.

Early LLLite results were clearer, but color, style, and detail quality initially
fell short of expectations. Further training improved visual results. V1 was
then chosen through repeated visual comparisons. This improvement supports the
interpretation that early training was insufficient, but does not conclusively
establish underfitting or imply that later checkpoints are always better.

## 5. Inference engineering and validation

The nodes attach control through ComfyUI's ModelPatcher, additional-model loading,
and block-replacement interfaces, retaining native memory management. Zero
strength bypasses control without merging controller weights into the backbone.
The display names are **Load MOSS-LLLite** and **Apply MOSS-LLLite**; internal IDs
are preserved for existing workflows.

An inference optimization replaced three complete fused-QKV projections with
the required output rows for each projection, reusing native casting and
quantization paths. In one warmed-up, 1024-resolution, two-step sampling test
with CFG 3 and an INT8/ConvRot backbone, time fell from approximately `1.94 s` to
`1.52 s`, with bitwise-identical final latents. This is evidence for that local
optimization under the tested settings, not a fixed speedup for every workflow.

A separate attempt to reduce RGB encoder batch computation was rejected because
it changed BF16 numerical results. Preserving generation results constrained
optimization decisions, rather than runtime alone.

## 6. Conclusions and limits

The released options offer distinct trade-offs: **beta2 prioritizes preservation,
beta3 prioritizes speed, and V1 prioritizes refinement and overall appearance.**
V1 is the final design established through four iterations and repeated visual
testing, and the author's personal choice across clarity, detail enhancement,
subject features, and color retention.

Available evidence includes module-level numerical comparisons, node-behavior
tests, a limited performance benchmark, and visual generation comparisons.
Standardized evaluations of perceptual quality, color error, feature consistency,
and variation across random seeds are still missing. Methodological novelty has
not been established through a comprehensive literature comparison. This report
documents an engineering exploration; subjective observations are not presented
as rigorous academic findings.