Upload 3 files
Browse files- ComfyUI-ChebyFlow/README.md +123 -0
- ComfyUI-ChebyFlow/__init__.py +3 -0
- ComfyUI-ChebyFlow/chebyflow.py +813 -0
ComfyUI-ChebyFlow/README.md
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ChebyFlow for ComfyUI
|
| 2 |
+
|
| 3 |
+
**ChebyFlow** is an adaptive two-stage Chebyshev flow sampler for ComfyUI, developed and tuned around MiniMax H3 / `CONST` rectified-flow sampling.
|
| 4 |
+
|
| 5 |
+
The core idea is to use the second model evaluation not at a fixed midpoint, but at an **adaptive internal stage** whose position is driven by the locally observed flow behavior. The sampler also adapts correction strength and can do so per latent channel.
|
| 6 |
+
|
| 7 |
+
## Nodes
|
| 8 |
+
|
| 9 |
+
### ChebyFlow Auto Sampler
|
| 10 |
+
|
| 11 |
+
Recommended default. It self-calibrates online during the current generation using the two model evaluations it already performs per interval. No prior runs, logs, or extra NFEs are required.
|
| 12 |
+
|
| 13 |
+
Controls:
|
| 14 |
+
|
| 15 |
+
```text
|
| 16 |
+
coordinate
|
| 17 |
+
base_stage_position
|
| 18 |
+
min_stage_position
|
| 19 |
+
max_stage_position
|
| 20 |
+
adaptivity
|
| 21 |
+
quality_bias
|
| 22 |
+
turbo_mode
|
| 23 |
+
correction_granularity
|
| 24 |
+
```
|
| 25 |
+
|
| 26 |
+
Presets:
|
| 27 |
+
|
| 28 |
+
```text
|
| 29 |
+
balanced
|
| 30 |
+
detail
|
| 31 |
+
smooth
|
| 32 |
+
stable
|
| 33 |
+
dynamic
|
| 34 |
+
```
|
| 35 |
+
|
| 36 |
+
`turbo_mode` is an orthogonal low-step operating mode intended for distilled / Turbo-LoRA workflows around roughly 6–10 steps. It shortens warm-up, accelerates the online observer, biases the internal stage later, and narrows the final correction range for stability on large intervals.
|
| 37 |
+
|
| 38 |
+
Recommended normal default:
|
| 39 |
+
|
| 40 |
+
```text
|
| 41 |
+
coordinate = flow_lambda
|
| 42 |
+
base_stage_position = 0.72
|
| 43 |
+
min_stage_position = 0.58
|
| 44 |
+
max_stage_position = 0.96
|
| 45 |
+
adaptivity = 1.00
|
| 46 |
+
quality_bias = balanced
|
| 47 |
+
turbo_mode = false
|
| 48 |
+
correction_granularity = per_channel
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
Recommended low-step / Turbo default:
|
| 52 |
+
|
| 53 |
+
```text
|
| 54 |
+
coordinate = flow_lambda
|
| 55 |
+
base_stage_position = 0.72
|
| 56 |
+
min_stage_position = 0.58
|
| 57 |
+
max_stage_position = 0.96
|
| 58 |
+
adaptivity = 1.00
|
| 59 |
+
quality_bias = balanced
|
| 60 |
+
turbo_mode = true
|
| 61 |
+
correction_granularity = per_channel
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
### ChebyFlow Advanced Sampler
|
| 65 |
+
|
| 66 |
+
Manually tunable version. Defaults keep the strongest combined configuration found during development:
|
| 67 |
+
|
| 68 |
+
```text
|
| 69 |
+
coordinate = flow_lambda
|
| 70 |
+
base_stage_position = 0.70
|
| 71 |
+
min_stage_position = 0.55
|
| 72 |
+
max_stage_position = 0.95
|
| 73 |
+
adapt_strength = 1.00
|
| 74 |
+
curvature_smoothing = 0.50
|
| 75 |
+
stage_curvature_scale = 0.12
|
| 76 |
+
|
| 77 |
+
curvature_metric = combined
|
| 78 |
+
residual_weight = 1.00
|
| 79 |
+
angle_weight = 1.00
|
| 80 |
+
magnitude_weight = 0.50
|
| 81 |
+
|
| 82 |
+
correction_mode = curvature_down
|
| 83 |
+
correction_granularity = per_channel
|
| 84 |
+
correction_min = 0.88
|
| 85 |
+
correction_max = 1.08
|
| 86 |
+
correction_curvature_scale = 0.12
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
## How ChebyFlow works
|
| 90 |
+
|
| 91 |
+
For each non-terminal outer interval, ChebyFlow:
|
| 92 |
+
|
| 93 |
+
1. evaluates the model at the current state;
|
| 94 |
+
2. predicts an internal Chebyshev stage;
|
| 95 |
+
3. evaluates the model again at that stage;
|
| 96 |
+
4. measures local flow behavior using residual change, angular turning, and magnitude change;
|
| 97 |
+
5. adapts the next internal-stage position;
|
| 98 |
+
6. adapts the current correction strength, optionally per latent channel;
|
| 99 |
+
7. applies the two-stage corrected update.
|
| 100 |
+
|
| 101 |
+
The Auto sampler additionally maintains fast and slow online statistics so it responds to **relative difficulty within the current generation**, rather than requiring hand-tuned absolute curvature scales.
|
| 102 |
+
|
| 103 |
+
Approximate cost: **~2 model evaluations per non-terminal interval**.
|
| 104 |
+
|
| 105 |
+
## Installation
|
| 106 |
+
|
| 107 |
+
Copy the folder to:
|
| 108 |
+
|
| 109 |
+
```text
|
| 110 |
+
ComfyUI/custom_nodes/ComfyUI-ChebyFlow/
|
| 111 |
+
```
|
| 112 |
+
|
| 113 |
+
Then restart ComfyUI completely.
|
| 114 |
+
|
| 115 |
+
Both nodes output `SAMPLER` and connect directly to the `sampler` input of `SamplerCustomAdvanced`.
|
| 116 |
+
|
| 117 |
+
## Compatibility
|
| 118 |
+
|
| 119 |
+
ChebyFlow currently validates that the active ComfyUI model sampling object is `CONST` / rectified-flow. It was developed specifically around MiniMax H3 and should be considered experimental outside that setting.
|
| 120 |
+
|
| 121 |
+
## Backward compatibility
|
| 122 |
+
|
| 123 |
+
The internal ComfyUI node class IDs remain compatible with the previous `H3 Adaptive Chebyshev` development builds, while the user-visible node names are now **ChebyFlow Auto Sampler** and **ChebyFlow Advanced Sampler**.
|
ComfyUI-ChebyFlow/__init__.py
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from .chebyflow import NODE_CLASS_MAPPINGS, NODE_DISPLAY_NAME_MAPPINGS
|
| 2 |
+
|
| 3 |
+
__all__ = ["NODE_CLASS_MAPPINGS", "NODE_DISPLAY_NAME_MAPPINGS"]
|
ComfyUI-ChebyFlow/chebyflow.py
ADDED
|
@@ -0,0 +1,813 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import math
|
| 2 |
+
import torch
|
| 3 |
+
|
| 4 |
+
import comfy.model_sampling
|
| 5 |
+
import comfy.samplers
|
| 6 |
+
from comfy.utils import model_trange as trange
|
| 7 |
+
|
| 8 |
+
_EPS = 1.0e-6
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
def _get_model_sampling(model):
|
| 12 |
+
try:
|
| 13 |
+
return model.inner_model.model_patcher.get_model_object("model_sampling")
|
| 14 |
+
except Exception as exc:
|
| 15 |
+
raise RuntimeError(
|
| 16 |
+
"ChebyFlow could not access ComfyUI model_sampling. "
|
| 17 |
+
"Use the normal SamplerCustomAdvanced / KSAMPLER path."
|
| 18 |
+
) from exc
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
def _validate_flow_model(model):
|
| 23 |
+
ms = _get_model_sampling(model)
|
| 24 |
+
if not isinstance(ms, comfy.model_sampling.CONST):
|
| 25 |
+
raise ValueError(
|
| 26 |
+
"ChebyFlow requires a CONST/rectified-flow model. "
|
| 27 |
+
f"Got {type(ms).__name__}."
|
| 28 |
+
)
|
| 29 |
+
return ms
|
| 30 |
+
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
def _sigma_list(sigmas):
|
| 34 |
+
vals = [float(v) for v in sigmas.detach().float().cpu().tolist()]
|
| 35 |
+
for i, s in enumerate(vals):
|
| 36 |
+
if s < -1.0e-7 or s > 1.0 + 1.0e-5:
|
| 37 |
+
raise ValueError(f"Flow sigma at index {i} is {s:.8g}, outside [0, 1].")
|
| 38 |
+
for i in range(max(0, len(vals) - 1)):
|
| 39 |
+
if vals[i + 1] > vals[i] + 1.0e-7:
|
| 40 |
+
raise ValueError("Sigmas must be monotonically non-increasing.")
|
| 41 |
+
return vals
|
| 42 |
+
|
| 43 |
+
|
| 44 |
+
|
| 45 |
+
def _flow_lambda(sigma):
|
| 46 |
+
s = min(max(float(sigma), _EPS), 1.0 - _EPS)
|
| 47 |
+
return math.log1p(-s) - math.log(s)
|
| 48 |
+
|
| 49 |
+
|
| 50 |
+
|
| 51 |
+
def _flow_sigma_from_lambda(lam):
|
| 52 |
+
lam = float(lam)
|
| 53 |
+
if lam >= 0.0:
|
| 54 |
+
e = math.exp(-lam)
|
| 55 |
+
return e / (1.0 + e)
|
| 56 |
+
e = math.exp(lam)
|
| 57 |
+
return 1.0 / (1.0 + e)
|
| 58 |
+
|
| 59 |
+
|
| 60 |
+
|
| 61 |
+
def _coordinate_value(sigma, coordinate):
|
| 62 |
+
if coordinate == "sigma":
|
| 63 |
+
return float(sigma)
|
| 64 |
+
if coordinate == "flow_lambda":
|
| 65 |
+
return _flow_lambda(sigma)
|
| 66 |
+
raise ValueError(f"Unknown coordinate: {coordinate}")
|
| 67 |
+
|
| 68 |
+
|
| 69 |
+
|
| 70 |
+
def _sigma_from_coordinate(q, coordinate):
|
| 71 |
+
if coordinate == "sigma":
|
| 72 |
+
return min(max(float(q), 0.0), 1.0)
|
| 73 |
+
if coordinate == "flow_lambda":
|
| 74 |
+
return min(max(_flow_sigma_from_lambda(q), 0.0), 1.0)
|
| 75 |
+
raise ValueError(f"Unknown coordinate: {coordinate}")
|
| 76 |
+
|
| 77 |
+
|
| 78 |
+
|
| 79 |
+
def _flow_velocity(x, denoised, sigma):
|
| 80 |
+
if float(sigma) <= _EPS:
|
| 81 |
+
return torch.zeros_like(x)
|
| 82 |
+
return (x - denoised) / float(sigma)
|
| 83 |
+
|
| 84 |
+
|
| 85 |
+
|
| 86 |
+
def _flow_rhs(x, denoised, sigma, coordinate):
|
| 87 |
+
v = _flow_velocity(x, denoised, sigma)
|
| 88 |
+
if coordinate == "sigma":
|
| 89 |
+
return v
|
| 90 |
+
if coordinate == "flow_lambda":
|
| 91 |
+
s = float(sigma)
|
| 92 |
+
return (-s * (1.0 - s)) * v
|
| 93 |
+
raise ValueError(f"Unknown coordinate: {coordinate}")
|
| 94 |
+
|
| 95 |
+
|
| 96 |
+
|
| 97 |
+
def _rms_scalar(t):
|
| 98 |
+
return torch.sqrt(torch.mean(t.float() * t.float()) + 1.0e-20)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
|
| 102 |
+
def _channel_reduce_dims(t):
|
| 103 |
+
return tuple(i for i in range(t.ndim) if i != 1)
|
| 104 |
+
|
| 105 |
+
|
| 106 |
+
|
| 107 |
+
def _metric_components_global(a, b):
|
| 108 |
+
af = a.float()
|
| 109 |
+
bf = b.float()
|
| 110 |
+
diff = bf - af
|
| 111 |
+
|
| 112 |
+
rms_a = _rms_scalar(af)
|
| 113 |
+
rms_b = _rms_scalar(bf)
|
| 114 |
+
residual = _rms_scalar(diff) / torch.clamp(rms_a, min=1.0e-12)
|
| 115 |
+
|
| 116 |
+
dot = torch.mean(af * bf)
|
| 117 |
+
denom = torch.clamp(rms_a * rms_b, min=1.0e-12)
|
| 118 |
+
cosine = torch.clamp(dot / denom, -1.0, 1.0)
|
| 119 |
+
angle = 1.0 - cosine
|
| 120 |
+
|
| 121 |
+
magnitude = torch.abs(torch.log(torch.clamp(rms_b, min=1.0e-12) / torch.clamp(rms_a, min=1.0e-12)))
|
| 122 |
+
return residual, angle, magnitude
|
| 123 |
+
|
| 124 |
+
|
| 125 |
+
|
| 126 |
+
def _metric_components_channel(a, b):
|
| 127 |
+
af = a.float()
|
| 128 |
+
bf = b.float()
|
| 129 |
+
dims = _channel_reduce_dims(af)
|
| 130 |
+
diff = bf - af
|
| 131 |
+
|
| 132 |
+
rms_a = torch.sqrt(torch.mean(af * af, dim=dims) + 1.0e-20)
|
| 133 |
+
rms_b = torch.sqrt(torch.mean(bf * bf, dim=dims) + 1.0e-20)
|
| 134 |
+
rms_d = torch.sqrt(torch.mean(diff * diff, dim=dims) + 1.0e-20)
|
| 135 |
+
residual = rms_d / torch.clamp(rms_a, min=1.0e-12)
|
| 136 |
+
|
| 137 |
+
dot = torch.mean(af * bf, dim=dims)
|
| 138 |
+
denom = torch.clamp(rms_a * rms_b, min=1.0e-12)
|
| 139 |
+
cosine = torch.clamp(dot / denom, -1.0, 1.0)
|
| 140 |
+
angle = 1.0 - cosine
|
| 141 |
+
|
| 142 |
+
magnitude = torch.abs(torch.log(torch.clamp(rms_b, min=1.0e-12) / torch.clamp(rms_a, min=1.0e-12)))
|
| 143 |
+
return residual, angle, magnitude
|
| 144 |
+
|
| 145 |
+
|
| 146 |
+
|
| 147 |
+
def _combine_metric(residual, angle, magnitude, metric, residual_weight, angle_weight, magnitude_weight):
|
| 148 |
+
if metric == "rms_delta":
|
| 149 |
+
return residual
|
| 150 |
+
if metric == "angle":
|
| 151 |
+
return angle
|
| 152 |
+
if metric == "magnitude":
|
| 153 |
+
return magnitude
|
| 154 |
+
if metric == "combined":
|
| 155 |
+
wr = float(max(residual_weight, 0.0))
|
| 156 |
+
wa = float(max(angle_weight, 0.0))
|
| 157 |
+
wm = float(max(magnitude_weight, 0.0))
|
| 158 |
+
denom = max(wr + wa + wm, 1.0e-12)
|
| 159 |
+
return (wr * residual + wa * angle + wm * magnitude) / denom
|
| 160 |
+
raise ValueError(f"Unknown curvature metric: {metric}")
|
| 161 |
+
|
| 162 |
+
|
| 163 |
+
|
| 164 |
+
def _metric_global(a, b, metric, rw, aw, mw):
|
| 165 |
+
parts = _metric_components_global(a, b)
|
| 166 |
+
return _combine_metric(*parts, metric, rw, aw, mw)
|
| 167 |
+
|
| 168 |
+
|
| 169 |
+
|
| 170 |
+
def _metric_channel(a, b, metric, rw, aw, mw):
|
| 171 |
+
parts = _metric_components_channel(a, b)
|
| 172 |
+
return _combine_metric(*parts, metric, rw, aw, mw)
|
| 173 |
+
|
| 174 |
+
|
| 175 |
+
|
| 176 |
+
def _score_from_metric(metric_value, scale):
|
| 177 |
+
scale = float(max(scale, 1.0e-6))
|
| 178 |
+
return metric_value / (metric_value + scale)
|
| 179 |
+
|
| 180 |
+
|
| 181 |
+
|
| 182 |
+
def _broadcast_channel(values, like):
|
| 183 |
+
shape = [1] * like.ndim
|
| 184 |
+
shape[1] = int(values.numel())
|
| 185 |
+
return values.to(device=like.device, dtype=like.dtype).reshape(shape)
|
| 186 |
+
|
| 187 |
+
|
| 188 |
+
|
| 189 |
+
def _observer_init(value):
|
| 190 |
+
v = value.detach().float()
|
| 191 |
+
base_dev = torch.clamp(torch.abs(v) * 0.25 + 0.05, min=0.05)
|
| 192 |
+
return {
|
| 193 |
+
"slow": v.clone(),
|
| 194 |
+
"fast": v.clone(),
|
| 195 |
+
"dev": base_dev.clone(),
|
| 196 |
+
}
|
| 197 |
+
|
| 198 |
+
|
| 199 |
+
|
| 200 |
+
def _observer_update(obs, value, slow_decay, fast_decay, dev_decay):
|
| 201 |
+
v = value.detach().float()
|
| 202 |
+
if obs is None:
|
| 203 |
+
return _observer_init(v)
|
| 204 |
+
slow = slow_decay * obs["slow"] + (1.0 - slow_decay) * v
|
| 205 |
+
fast = fast_decay * obs["fast"] + (1.0 - fast_decay) * v
|
| 206 |
+
dev = dev_decay * obs["dev"] + (1.0 - dev_decay) * torch.abs(v - slow)
|
| 207 |
+
dev = torch.clamp(dev, min=0.01)
|
| 208 |
+
return {"slow": slow, "fast": fast, "dev": dev}
|
| 209 |
+
|
| 210 |
+
|
| 211 |
+
|
| 212 |
+
def _observer_score(obs, value, current_weight, trend_weight, temperature):
|
| 213 |
+
if obs is None:
|
| 214 |
+
return torch.full_like(value.detach().float(), 0.5)
|
| 215 |
+
v = value.detach().float()
|
| 216 |
+
slow = obs["slow"]
|
| 217 |
+
fast = obs["fast"]
|
| 218 |
+
dev = torch.clamp(obs["dev"], min=0.01)
|
| 219 |
+
z_current = (v - slow) / dev
|
| 220 |
+
z_trend = (fast - slow) / dev
|
| 221 |
+
z = current_weight * z_current + trend_weight * z_trend
|
| 222 |
+
return torch.sigmoid(z / float(max(temperature, 1.0e-4)))
|
| 223 |
+
|
| 224 |
+
|
| 225 |
+
|
| 226 |
+
def _weighted_score(scores, weights):
|
| 227 |
+
num = None
|
| 228 |
+
denom = 0.0
|
| 229 |
+
for score, weight in zip(scores, weights):
|
| 230 |
+
w = float(max(weight, 0.0))
|
| 231 |
+
if w == 0.0:
|
| 232 |
+
continue
|
| 233 |
+
term = score * w
|
| 234 |
+
num = term if num is None else num + term
|
| 235 |
+
denom += w
|
| 236 |
+
if num is None or denom <= 0.0:
|
| 237 |
+
ref = scores[0]
|
| 238 |
+
return torch.full_like(ref, 0.5)
|
| 239 |
+
return num / denom
|
| 240 |
+
|
| 241 |
+
|
| 242 |
+
|
| 243 |
+
def _centered_response(score):
|
| 244 |
+
return 2.0 * torch.clamp(score, 0.0, 1.0) - 1.0
|
| 245 |
+
|
| 246 |
+
|
| 247 |
+
|
| 248 |
+
def _stage_from_difficulty(base_c, min_c, max_c, difficulty, adaptivity, stage_gain=1.0):
|
| 249 |
+
shift = float(torch.clamp(_centered_response(difficulty) * float(stage_gain), -1.0, 1.0).item())
|
| 250 |
+
if shift >= 0.0:
|
| 251 |
+
target = base_c + shift * (max_c - base_c)
|
| 252 |
+
else:
|
| 253 |
+
target = base_c + shift * (base_c - min_c)
|
| 254 |
+
c = (1.0 - adaptivity) * base_c + adaptivity * target
|
| 255 |
+
return min(max(float(c), min_c), max_c)
|
| 256 |
+
|
| 257 |
+
|
| 258 |
+
|
| 259 |
+
def _correction_strength_from_difficulty(difficulty, low, high):
|
| 260 |
+
low = float(low)
|
| 261 |
+
high = float(high)
|
| 262 |
+
shift = torch.clamp(_centered_response(difficulty), -1.0, 1.0)
|
| 263 |
+
hard = torch.clamp(shift, min=0.0)
|
| 264 |
+
easy = torch.clamp(-shift, min=0.0)
|
| 265 |
+
out = torch.ones_like(shift)
|
| 266 |
+
out = out + hard * (low - 1.0)
|
| 267 |
+
out = out + easy * (high - 1.0)
|
| 268 |
+
return out
|
| 269 |
+
|
| 270 |
+
|
| 271 |
+
|
| 272 |
+
def _auto_profile(name):
|
| 273 |
+
name = str(name)
|
| 274 |
+
if name == "detail":
|
| 275 |
+
return {
|
| 276 |
+
"weights": (0.50, 0.35, 0.15),
|
| 277 |
+
"slow_decay": 0.88,
|
| 278 |
+
"fast_decay": 0.42,
|
| 279 |
+
"dev_decay": 0.84,
|
| 280 |
+
"current_weight": 0.70,
|
| 281 |
+
"trend_weight": 0.30,
|
| 282 |
+
"temperature": 1.05,
|
| 283 |
+
"stage_gain": 1.10,
|
| 284 |
+
"corr_low": 0.92,
|
| 285 |
+
"corr_high": 1.10,
|
| 286 |
+
"warmup_steps": 2,
|
| 287 |
+
"soft_start_steps": 0,
|
| 288 |
+
}
|
| 289 |
+
if name == "smooth":
|
| 290 |
+
return {
|
| 291 |
+
"weights": (0.40, 0.45, 0.15),
|
| 292 |
+
"slow_decay": 0.93,
|
| 293 |
+
"fast_decay": 0.34,
|
| 294 |
+
"dev_decay": 0.89,
|
| 295 |
+
"current_weight": 0.62,
|
| 296 |
+
"trend_weight": 0.38,
|
| 297 |
+
"temperature": 1.35,
|
| 298 |
+
"stage_gain": 0.88,
|
| 299 |
+
"corr_low": 0.94,
|
| 300 |
+
"corr_high": 1.04,
|
| 301 |
+
"warmup_steps": 2,
|
| 302 |
+
"soft_start_steps": 0,
|
| 303 |
+
}
|
| 304 |
+
if name == "stable":
|
| 305 |
+
return {
|
| 306 |
+
"weights": (0.42, 0.43, 0.15),
|
| 307 |
+
"slow_decay": 0.94,
|
| 308 |
+
"fast_decay": 0.32,
|
| 309 |
+
"dev_decay": 0.90,
|
| 310 |
+
"current_weight": 0.60,
|
| 311 |
+
"trend_weight": 0.40,
|
| 312 |
+
"temperature": 1.45,
|
| 313 |
+
"stage_gain": 0.78,
|
| 314 |
+
"corr_low": 0.95,
|
| 315 |
+
"corr_high": 1.03,
|
| 316 |
+
"warmup_steps": 2,
|
| 317 |
+
"soft_start_steps": 0,
|
| 318 |
+
}
|
| 319 |
+
if name == "dynamic":
|
| 320 |
+
return {
|
| 321 |
+
"weights": (0.35, 0.50, 0.15),
|
| 322 |
+
"slow_decay": 0.87,
|
| 323 |
+
"fast_decay": 0.38,
|
| 324 |
+
"dev_decay": 0.83,
|
| 325 |
+
"current_weight": 0.67,
|
| 326 |
+
"trend_weight": 0.33,
|
| 327 |
+
"temperature": 1.00,
|
| 328 |
+
"stage_gain": 1.18,
|
| 329 |
+
"corr_low": 0.91,
|
| 330 |
+
"corr_high": 1.09,
|
| 331 |
+
"warmup_steps": 2,
|
| 332 |
+
"soft_start_steps": 0,
|
| 333 |
+
}
|
| 334 |
+
# balanced
|
| 335 |
+
return {
|
| 336 |
+
"weights": (0.45, 0.40, 0.15),
|
| 337 |
+
"slow_decay": 0.90,
|
| 338 |
+
"fast_decay": 0.40,
|
| 339 |
+
"dev_decay": 0.86,
|
| 340 |
+
"current_weight": 0.68,
|
| 341 |
+
"trend_weight": 0.32,
|
| 342 |
+
"temperature": 1.18,
|
| 343 |
+
"stage_gain": 1.00,
|
| 344 |
+
"corr_low": 0.90,
|
| 345 |
+
"corr_high": 1.08,
|
| 346 |
+
"warmup_steps": 2,
|
| 347 |
+
"soft_start_steps": 0,
|
| 348 |
+
}
|
| 349 |
+
|
| 350 |
+
|
| 351 |
+
def _apply_turbo_profile(profile):
|
| 352 |
+
"""Return a low-step controller variant without changing the preset's character."""
|
| 353 |
+
p = dict(profile)
|
| 354 |
+
# Faster online calibration: fewer historical steps exist, so both baselines must
|
| 355 |
+
# move sooner. Keep slow > fast so trend information remains meaningful.
|
| 356 |
+
p["slow_decay"] = max(0.68, float(p["slow_decay"]) - 0.15)
|
| 357 |
+
p["fast_decay"] = max(0.16, float(p["fast_decay"]) - 0.15)
|
| 358 |
+
p["dev_decay"] = max(0.68, float(p["dev_decay"]) - 0.12)
|
| 359 |
+
|
| 360 |
+
# A little more weight on the current interval and a lower temperature make the
|
| 361 |
+
# normalized controller useful after only one baseline interval.
|
| 362 |
+
p["current_weight"] = min(0.78, float(p["current_weight"]) + 0.05)
|
| 363 |
+
p["trend_weight"] = max(0.0, 1.0 - p["current_weight"])
|
| 364 |
+
p["temperature"] = max(0.75, float(p["temperature"]) * 0.78)
|
| 365 |
+
p["stage_gain"] = float(p["stage_gain"]) * 1.08
|
| 366 |
+
|
| 367 |
+
# Low-step intervals are large: move the probe decisively, but keep algebraic
|
| 368 |
+
# correction closer to 1.0 so one interval cannot redirect the whole trajectory.
|
| 369 |
+
p["corr_low"] = 1.0 + (float(p["corr_low"]) - 1.0) * 0.50
|
| 370 |
+
p["corr_high"] = 1.0 + (float(p["corr_high"]) - 1.0) * 0.50
|
| 371 |
+
|
| 372 |
+
p["warmup_steps"] = 1
|
| 373 |
+
p["soft_start_steps"] = 1
|
| 374 |
+
p["stage_base_shift"] = 0.05
|
| 375 |
+
p["stage_min_shift"] = 0.07
|
| 376 |
+
p["stage_max_shift"] = 0.01
|
| 377 |
+
return p
|
| 378 |
+
|
| 379 |
+
|
| 380 |
+
def _controller_confidence(step_index, warmup_steps, soft_start_steps):
|
| 381 |
+
"""0 during warmup, then a short confidence ramp before full adaptation."""
|
| 382 |
+
i = int(step_index)
|
| 383 |
+
warm = int(max(warmup_steps, 0))
|
| 384 |
+
soft = int(max(soft_start_steps, 0))
|
| 385 |
+
if i < warm:
|
| 386 |
+
return 0.0
|
| 387 |
+
if soft <= 0:
|
| 388 |
+
return 1.0
|
| 389 |
+
j = i - warm
|
| 390 |
+
if j < soft:
|
| 391 |
+
return float(j + 1) / float(soft + 1)
|
| 392 |
+
return 1.0
|
| 393 |
+
|
| 394 |
+
|
| 395 |
+
@torch.no_grad()
|
| 396 |
+
def sample_h3_adaptive_chebyshev_advanced(
|
| 397 |
+
model,
|
| 398 |
+
x,
|
| 399 |
+
sigmas,
|
| 400 |
+
extra_args=None,
|
| 401 |
+
callback=None,
|
| 402 |
+
disable=None,
|
| 403 |
+
coordinate="flow_lambda",
|
| 404 |
+
base_stage_position=0.70,
|
| 405 |
+
min_stage_position=0.55,
|
| 406 |
+
max_stage_position=0.95,
|
| 407 |
+
adapt_strength=1.0,
|
| 408 |
+
curvature_smoothing=0.50,
|
| 409 |
+
stage_curvature_scale=0.12,
|
| 410 |
+
curvature_metric="combined",
|
| 411 |
+
residual_weight=1.0,
|
| 412 |
+
angle_weight=1.0,
|
| 413 |
+
magnitude_weight=0.5,
|
| 414 |
+
correction_mode="curvature_down",
|
| 415 |
+
correction_granularity="per_channel",
|
| 416 |
+
correction_min=0.88,
|
| 417 |
+
correction_max=1.08,
|
| 418 |
+
correction_curvature_scale=0.12,
|
| 419 |
+
):
|
| 420 |
+
"""
|
| 421 |
+
Advanced adaptive 2-stage Chebyshev/GS sampler for H3 CONST flow models.
|
| 422 |
+
|
| 423 |
+
This is the manually tunable version. Defaults are intentionally set to a strong
|
| 424 |
+
"all three improvements enabled" configuration:
|
| 425 |
+
- combined curvature metric,
|
| 426 |
+
- curvature-dependent correction damping,
|
| 427 |
+
- per-channel correction strengths.
|
| 428 |
+
"""
|
| 429 |
+
extra_args = {} if extra_args is None else extra_args
|
| 430 |
+
_validate_flow_model(model)
|
| 431 |
+
sigma_vals = _sigma_list(sigmas)
|
| 432 |
+
if len(sigma_vals) <= 1:
|
| 433 |
+
return x
|
| 434 |
+
|
| 435 |
+
base_c = min(max(float(base_stage_position), 0.05), 0.99)
|
| 436 |
+
min_c = min(max(float(min_stage_position), 0.05), 0.99)
|
| 437 |
+
max_c = min(max(float(max_stage_position), min_c + 1.0e-4), 0.99)
|
| 438 |
+
adapt = float(max(adapt_strength, 0.0))
|
| 439 |
+
smooth = float(min(max(curvature_smoothing, 0.0), 0.99))
|
| 440 |
+
stage_scale = float(max(stage_curvature_scale, 1.0e-6))
|
| 441 |
+
|
| 442 |
+
corr_min = float(min(correction_min, correction_max))
|
| 443 |
+
corr_max = float(max(correction_min, correction_max))
|
| 444 |
+
corr_scale = float(max(correction_curvature_scale, 1.0e-6))
|
| 445 |
+
|
| 446 |
+
c_state = base_c
|
| 447 |
+
global_metric_ema = None
|
| 448 |
+
channel_metric_ema = None
|
| 449 |
+
s_in = x.new_ones([x.shape[0]])
|
| 450 |
+
|
| 451 |
+
for i in trange(len(sigmas) - 1, disable=disable):
|
| 452 |
+
sigma_s = sigma_vals[i]
|
| 453 |
+
sigma_t = sigma_vals[i + 1]
|
| 454 |
+
denoised_s = model(x, sigmas[i] * s_in, **extra_args)
|
| 455 |
+
|
| 456 |
+
if callback is not None:
|
| 457 |
+
callback({"x": x, "i": i, "sigma": sigmas[i], "sigma_hat": sigmas[i], "denoised": denoised_s})
|
| 458 |
+
|
| 459 |
+
if sigma_t <= _EPS:
|
| 460 |
+
x = denoised_s
|
| 461 |
+
break
|
| 462 |
+
|
| 463 |
+
c = min(max(float(c_state), min_c), max_c)
|
| 464 |
+
b_stage = 1.0 / (2.0 * c)
|
| 465 |
+
b_start = 1.0 - b_stage
|
| 466 |
+
|
| 467 |
+
q_s = _coordinate_value(sigma_s, coordinate)
|
| 468 |
+
q_t = _coordinate_value(sigma_t, coordinate)
|
| 469 |
+
h = q_t - q_s
|
| 470 |
+
if abs(h) < 1.0e-12:
|
| 471 |
+
continue
|
| 472 |
+
|
| 473 |
+
rhs_s = _flow_rhs(x, denoised_s, sigma_s, coordinate)
|
| 474 |
+
q_c = q_s + c * h
|
| 475 |
+
sigma_c = _sigma_from_coordinate(q_c, coordinate)
|
| 476 |
+
x_c = x + (c * h) * rhs_s
|
| 477 |
+
denoised_c = model(x_c, x.new_full((x.shape[0],), sigma_c), **extra_args)
|
| 478 |
+
rhs_c = _flow_rhs(x_c, denoised_c, sigma_c, coordinate)
|
| 479 |
+
|
| 480 |
+
x_first = x + h * rhs_s
|
| 481 |
+
x_cheb = x + h * (b_start * rhs_s + b_stage * rhs_c)
|
| 482 |
+
correction = x_cheb - x_first
|
| 483 |
+
|
| 484 |
+
g_metric = _metric_global(rhs_s, rhs_c, curvature_metric, residual_weight, angle_weight, magnitude_weight)
|
| 485 |
+
global_metric_ema = g_metric if global_metric_ema is None else (
|
| 486 |
+
smooth * global_metric_ema + (1.0 - smooth) * g_metric
|
| 487 |
+
)
|
| 488 |
+
g_score = _score_from_metric(global_metric_ema, stage_scale)
|
| 489 |
+
target_c = min_c + (max_c - min_c) * float(g_score.item())
|
| 490 |
+
c_state = (1.0 - adapt) * base_c + adapt * target_c
|
| 491 |
+
c_state = min(max(c_state, min_c), max_c)
|
| 492 |
+
|
| 493 |
+
if correction_mode == "constant":
|
| 494 |
+
strength = corr_max
|
| 495 |
+
else:
|
| 496 |
+
if correction_granularity == "per_channel":
|
| 497 |
+
c_metric = _metric_channel(rhs_s, rhs_c, curvature_metric, residual_weight, angle_weight, magnitude_weight)
|
| 498 |
+
channel_metric_ema = c_metric if channel_metric_ema is None else (
|
| 499 |
+
smooth * channel_metric_ema + (1.0 - smooth) * c_metric
|
| 500 |
+
)
|
| 501 |
+
score = _score_from_metric(channel_metric_ema, corr_scale)
|
| 502 |
+
if correction_mode == "curvature_down":
|
| 503 |
+
strength = corr_max - (corr_max - corr_min) * score
|
| 504 |
+
elif correction_mode == "curvature_up":
|
| 505 |
+
strength = corr_min + (corr_max - corr_min) * score
|
| 506 |
+
else:
|
| 507 |
+
raise ValueError(f"Unknown correction mode: {correction_mode}")
|
| 508 |
+
strength = _broadcast_channel(strength, correction)
|
| 509 |
+
else:
|
| 510 |
+
score = _score_from_metric(global_metric_ema, corr_scale)
|
| 511 |
+
if correction_mode == "curvature_down":
|
| 512 |
+
strength = corr_max - (corr_max - corr_min) * score
|
| 513 |
+
elif correction_mode == "curvature_up":
|
| 514 |
+
strength = corr_min + (corr_max - corr_min) * score
|
| 515 |
+
else:
|
| 516 |
+
raise ValueError(f"Unknown correction mode: {correction_mode}")
|
| 517 |
+
strength = float(strength.item())
|
| 518 |
+
|
| 519 |
+
x = x_first + correction * strength
|
| 520 |
+
|
| 521 |
+
return x
|
| 522 |
+
|
| 523 |
+
|
| 524 |
+
@torch.no_grad()
|
| 525 |
+
def sample_h3_adaptive_chebyshev_auto(
|
| 526 |
+
model,
|
| 527 |
+
x,
|
| 528 |
+
sigmas,
|
| 529 |
+
extra_args=None,
|
| 530 |
+
callback=None,
|
| 531 |
+
disable=None,
|
| 532 |
+
coordinate="flow_lambda",
|
| 533 |
+
base_stage_position=0.72,
|
| 534 |
+
min_stage_position=0.58,
|
| 535 |
+
max_stage_position=0.96,
|
| 536 |
+
adaptivity=1.0,
|
| 537 |
+
quality_bias="balanced",
|
| 538 |
+
turbo_mode=False,
|
| 539 |
+
correction_granularity="per_channel",
|
| 540 |
+
):
|
| 541 |
+
"""
|
| 542 |
+
Self-calibrating adaptive Chebyshev sampler.
|
| 543 |
+
|
| 544 |
+
No extra model evaluations are added. The sampler self-calibrates online during the
|
| 545 |
+
current generation using only the two evaluations it already performs per interval.
|
| 546 |
+
|
| 547 |
+
turbo_mode is an orthogonal numerical regime, not a quality preset. It keeps the
|
| 548 |
+
selected quality_bias character but accelerates observer dynamics, shortens startup,
|
| 549 |
+
biases the internal stage later, and narrows correction excursions for large low-step
|
| 550 |
+
intervals such as ~6-10 step distilled/Turbo-LoRA workflows.
|
| 551 |
+
"""
|
| 552 |
+
extra_args = {} if extra_args is None else extra_args
|
| 553 |
+
_validate_flow_model(model)
|
| 554 |
+
sigma_vals = _sigma_list(sigmas)
|
| 555 |
+
if len(sigma_vals) <= 1:
|
| 556 |
+
return x
|
| 557 |
+
|
| 558 |
+
profile = _auto_profile(quality_bias)
|
| 559 |
+
if bool(turbo_mode):
|
| 560 |
+
profile = _apply_turbo_profile(profile)
|
| 561 |
+
|
| 562 |
+
# User values remain the baseline. Turbo mode shifts the operating envelope later
|
| 563 |
+
# without requiring separate turbo presets. Keep the internal stage strictly below 1.
|
| 564 |
+
base_c = float(base_stage_position)
|
| 565 |
+
min_c = float(min_stage_position)
|
| 566 |
+
max_c = float(max_stage_position)
|
| 567 |
+
if bool(turbo_mode):
|
| 568 |
+
base_c += float(profile.get("stage_base_shift", 0.0))
|
| 569 |
+
min_c += float(profile.get("stage_min_shift", 0.0))
|
| 570 |
+
max_c += float(profile.get("stage_max_shift", 0.0))
|
| 571 |
+
|
| 572 |
+
base_c = min(max(base_c, 0.05), 0.99)
|
| 573 |
+
min_c = min(max(min_c, 0.05), 0.99)
|
| 574 |
+
max_c = min(max(max_c, min_c + 1.0e-4), 0.99)
|
| 575 |
+
base_c = min(max(base_c, min_c), max_c)
|
| 576 |
+
|
| 577 |
+
adaptivity = float(max(adaptivity, 0.0))
|
| 578 |
+
weights = profile["weights"]
|
| 579 |
+
warmup_steps = int(profile["warmup_steps"])
|
| 580 |
+
soft_start_steps = int(profile.get("soft_start_steps", 0))
|
| 581 |
+
|
| 582 |
+
obs_g_res = obs_g_ang = obs_g_mag = None
|
| 583 |
+
obs_c_res = obs_c_ang = obs_c_mag = None
|
| 584 |
+
c_state = base_c
|
| 585 |
+
s_in = x.new_ones([x.shape[0]])
|
| 586 |
+
|
| 587 |
+
for i in trange(len(sigmas) - 1, disable=disable):
|
| 588 |
+
sigma_s = sigma_vals[i]
|
| 589 |
+
sigma_t = sigma_vals[i + 1]
|
| 590 |
+
denoised_s = model(x, sigmas[i] * s_in, **extra_args)
|
| 591 |
+
|
| 592 |
+
if callback is not None:
|
| 593 |
+
callback({"x": x, "i": i, "sigma": sigmas[i], "sigma_hat": sigmas[i], "denoised": denoised_s})
|
| 594 |
+
|
| 595 |
+
if sigma_t <= _EPS:
|
| 596 |
+
x = denoised_s
|
| 597 |
+
break
|
| 598 |
+
|
| 599 |
+
confidence = _controller_confidence(i, warmup_steps, soft_start_steps)
|
| 600 |
+
c = base_c if confidence <= 0.0 else c_state
|
| 601 |
+
c = min(max(float(c), min_c), max_c)
|
| 602 |
+
b_stage = 1.0 / (2.0 * c)
|
| 603 |
+
b_start = 1.0 - b_stage
|
| 604 |
+
|
| 605 |
+
q_s = _coordinate_value(sigma_s, coordinate)
|
| 606 |
+
q_t = _coordinate_value(sigma_t, coordinate)
|
| 607 |
+
h = q_t - q_s
|
| 608 |
+
if abs(h) < 1.0e-12:
|
| 609 |
+
continue
|
| 610 |
+
|
| 611 |
+
rhs_s = _flow_rhs(x, denoised_s, sigma_s, coordinate)
|
| 612 |
+
q_c = q_s + c * h
|
| 613 |
+
sigma_c = _sigma_from_coordinate(q_c, coordinate)
|
| 614 |
+
x_c = x + (c * h) * rhs_s
|
| 615 |
+
denoised_c = model(x_c, x.new_full((x.shape[0],), sigma_c), **extra_args)
|
| 616 |
+
rhs_c = _flow_rhs(x_c, denoised_c, sigma_c, coordinate)
|
| 617 |
+
|
| 618 |
+
x_first = x + h * rhs_s
|
| 619 |
+
x_cheb = x + h * (b_start * rhs_s + b_stage * rhs_c)
|
| 620 |
+
correction = x_cheb - x_first
|
| 621 |
+
|
| 622 |
+
# Current raw local difficulty components.
|
| 623 |
+
g_res, g_ang, g_mag = _metric_components_global(rhs_s, rhs_c)
|
| 624 |
+
|
| 625 |
+
if obs_g_res is None:
|
| 626 |
+
g_difficulty = torch.full_like(g_res, 0.5)
|
| 627 |
+
else:
|
| 628 |
+
g_res_s = _observer_score(obs_g_res, g_res, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 629 |
+
g_ang_s = _observer_score(obs_g_ang, g_ang, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 630 |
+
g_mag_s = _observer_score(obs_g_mag, g_mag, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 631 |
+
g_difficulty = _weighted_score((g_res_s, g_ang_s, g_mag_s), weights)
|
| 632 |
+
|
| 633 |
+
# Global stage controller for NEXT interval. During turbo soft-start, the
|
| 634 |
+
# confidence factor ramps this from fixed-base behavior to full adaptation.
|
| 635 |
+
c_state = _stage_from_difficulty(
|
| 636 |
+
base_c,
|
| 637 |
+
min_c,
|
| 638 |
+
max_c,
|
| 639 |
+
g_difficulty,
|
| 640 |
+
adaptivity * confidence,
|
| 641 |
+
profile["stage_gain"],
|
| 642 |
+
)
|
| 643 |
+
|
| 644 |
+
c_res = c_ang = c_mag = None
|
| 645 |
+
if correction_granularity == "per_channel":
|
| 646 |
+
c_res, c_ang, c_mag = _metric_components_channel(rhs_s, rhs_c)
|
| 647 |
+
|
| 648 |
+
# Correction controller for CURRENT interval. The automatic strength is blended
|
| 649 |
+
# toward 1.0 by controller confidence, giving Turbo mode a 50% soft-start step.
|
| 650 |
+
if confidence <= 0.0:
|
| 651 |
+
strength = 1.0
|
| 652 |
+
else:
|
| 653 |
+
if correction_granularity == "per_channel":
|
| 654 |
+
if obs_c_res is None:
|
| 655 |
+
c_difficulty = torch.full_like(c_res, 0.5)
|
| 656 |
+
else:
|
| 657 |
+
c_res_s = _observer_score(obs_c_res, c_res, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 658 |
+
c_ang_s = _observer_score(obs_c_ang, c_ang, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 659 |
+
c_mag_s = _observer_score(obs_c_mag, c_mag, profile["current_weight"], profile["trend_weight"], profile["temperature"])
|
| 660 |
+
c_difficulty = _weighted_score((c_res_s, c_ang_s, c_mag_s), weights)
|
| 661 |
+
auto_strength = _correction_strength_from_difficulty(
|
| 662 |
+
c_difficulty, profile["corr_low"], profile["corr_high"]
|
| 663 |
+
)
|
| 664 |
+
auto_strength = 1.0 + confidence * (auto_strength - 1.0)
|
| 665 |
+
strength = _broadcast_channel(auto_strength, correction)
|
| 666 |
+
else:
|
| 667 |
+
auto_strength = _correction_strength_from_difficulty(
|
| 668 |
+
g_difficulty, profile["corr_low"], profile["corr_high"]
|
| 669 |
+
)
|
| 670 |
+
auto_strength = 1.0 + confidence * (auto_strength - 1.0)
|
| 671 |
+
strength = float(auto_strength.item())
|
| 672 |
+
|
| 673 |
+
x = x_first + correction * strength
|
| 674 |
+
|
| 675 |
+
# Update observers AFTER using them: current difficulty is judged against the
|
| 676 |
+
# previously learned baseline, rather than being normalized away immediately.
|
| 677 |
+
obs_g_res = _observer_update(obs_g_res, g_res, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 678 |
+
obs_g_ang = _observer_update(obs_g_ang, g_ang, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 679 |
+
obs_g_mag = _observer_update(obs_g_mag, g_mag, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 680 |
+
|
| 681 |
+
if correction_granularity == "per_channel":
|
| 682 |
+
obs_c_res = _observer_update(obs_c_res, c_res, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 683 |
+
obs_c_ang = _observer_update(obs_c_ang, c_ang, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 684 |
+
obs_c_mag = _observer_update(obs_c_mag, c_mag, profile["slow_decay"], profile["fast_decay"], profile["dev_decay"])
|
| 685 |
+
|
| 686 |
+
return x
|
| 687 |
+
|
| 688 |
+
|
| 689 |
+
class H3AdaptiveChebyshevAdvancedSampler:
|
| 690 |
+
@classmethod
|
| 691 |
+
def INPUT_TYPES(cls):
|
| 692 |
+
return {
|
| 693 |
+
"required": {
|
| 694 |
+
"coordinate": (["flow_lambda", "sigma"], {"default": "flow_lambda"}),
|
| 695 |
+
"base_stage_position": ("FLOAT", {"default": 0.70, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 696 |
+
"min_stage_position": ("FLOAT", {"default": 0.55, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 697 |
+
"max_stage_position": ("FLOAT", {"default": 0.95, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 698 |
+
"adapt_strength": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.5, "step": 0.05}),
|
| 699 |
+
"curvature_smoothing": ("FLOAT", {"default": 0.50, "min": 0.0, "max": 0.95, "step": 0.05}),
|
| 700 |
+
"stage_curvature_scale": ("FLOAT", {"default": 0.12, "min": 0.01, "max": 2.0, "step": 0.01}),
|
| 701 |
+
"curvature_metric": (["rms_delta", "angle", "magnitude", "combined"], {"default": "combined"}),
|
| 702 |
+
"residual_weight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 4.0, "step": 0.05}),
|
| 703 |
+
"angle_weight": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 4.0, "step": 0.05}),
|
| 704 |
+
"magnitude_weight": ("FLOAT", {"default": 0.5, "min": 0.0, "max": 4.0, "step": 0.05}),
|
| 705 |
+
"correction_mode": (["constant", "curvature_down", "curvature_up"], {"default": "curvature_down"}),
|
| 706 |
+
"correction_granularity": (["global", "per_channel"], {"default": "per_channel"}),
|
| 707 |
+
"correction_min": ("FLOAT", {"default": 0.88, "min": 0.0, "max": 1.5, "step": 0.01}),
|
| 708 |
+
"correction_max": ("FLOAT", {"default": 1.08, "min": 0.0, "max": 1.5, "step": 0.01}),
|
| 709 |
+
"correction_curvature_scale": ("FLOAT", {"default": 0.12, "min": 0.01, "max": 2.0, "step": 0.01}),
|
| 710 |
+
}
|
| 711 |
+
}
|
| 712 |
+
|
| 713 |
+
RETURN_TYPES = ("SAMPLER",)
|
| 714 |
+
RETURN_NAMES = ("sampler",)
|
| 715 |
+
FUNCTION = "get_sampler"
|
| 716 |
+
CATEGORY = "sampling/custom_sampling/samplers"
|
| 717 |
+
|
| 718 |
+
def get_sampler(
|
| 719 |
+
self,
|
| 720 |
+
coordinate,
|
| 721 |
+
base_stage_position,
|
| 722 |
+
min_stage_position,
|
| 723 |
+
max_stage_position,
|
| 724 |
+
adapt_strength,
|
| 725 |
+
curvature_smoothing,
|
| 726 |
+
stage_curvature_scale,
|
| 727 |
+
curvature_metric,
|
| 728 |
+
residual_weight,
|
| 729 |
+
angle_weight,
|
| 730 |
+
magnitude_weight,
|
| 731 |
+
correction_mode,
|
| 732 |
+
correction_granularity,
|
| 733 |
+
correction_min,
|
| 734 |
+
correction_max,
|
| 735 |
+
correction_curvature_scale,
|
| 736 |
+
):
|
| 737 |
+
sampler = comfy.samplers.KSAMPLER(
|
| 738 |
+
sample_h3_adaptive_chebyshev_advanced,
|
| 739 |
+
extra_options={
|
| 740 |
+
"coordinate": coordinate,
|
| 741 |
+
"base_stage_position": base_stage_position,
|
| 742 |
+
"min_stage_position": min_stage_position,
|
| 743 |
+
"max_stage_position": max_stage_position,
|
| 744 |
+
"adapt_strength": adapt_strength,
|
| 745 |
+
"curvature_smoothing": curvature_smoothing,
|
| 746 |
+
"stage_curvature_scale": stage_curvature_scale,
|
| 747 |
+
"curvature_metric": curvature_metric,
|
| 748 |
+
"residual_weight": residual_weight,
|
| 749 |
+
"angle_weight": angle_weight,
|
| 750 |
+
"magnitude_weight": magnitude_weight,
|
| 751 |
+
"correction_mode": correction_mode,
|
| 752 |
+
"correction_granularity": correction_granularity,
|
| 753 |
+
"correction_min": correction_min,
|
| 754 |
+
"correction_max": correction_max,
|
| 755 |
+
"correction_curvature_scale": correction_curvature_scale,
|
| 756 |
+
},
|
| 757 |
+
)
|
| 758 |
+
return (sampler,)
|
| 759 |
+
|
| 760 |
+
|
| 761 |
+
class H3AdaptiveChebyshevAutoSampler:
|
| 762 |
+
@classmethod
|
| 763 |
+
def INPUT_TYPES(cls):
|
| 764 |
+
return {
|
| 765 |
+
"required": {
|
| 766 |
+
"coordinate": (["flow_lambda", "sigma"], {"default": "flow_lambda"}),
|
| 767 |
+
"base_stage_position": ("FLOAT", {"default": 0.72, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 768 |
+
"min_stage_position": ("FLOAT", {"default": 0.58, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 769 |
+
"max_stage_position": ("FLOAT", {"default": 0.96, "min": 0.05, "max": 0.99, "step": 0.01}),
|
| 770 |
+
"adaptivity": ("FLOAT", {"default": 1.0, "min": 0.0, "max": 1.5, "step": 0.05}),
|
| 771 |
+
"quality_bias": (["balanced", "detail", "smooth", "stable", "dynamic"], {"default": "balanced"}),
|
| 772 |
+
"turbo_mode": ("BOOLEAN", {"default": False}),
|
| 773 |
+
"correction_granularity": (["global", "per_channel"], {"default": "per_channel"}),
|
| 774 |
+
}
|
| 775 |
+
}
|
| 776 |
+
|
| 777 |
+
RETURN_TYPES = ("SAMPLER",)
|
| 778 |
+
RETURN_NAMES = ("sampler",)
|
| 779 |
+
FUNCTION = "get_sampler"
|
| 780 |
+
CATEGORY = "sampling/custom_sampling/samplers"
|
| 781 |
+
|
| 782 |
+
def get_sampler(self, coordinate, base_stage_position, min_stage_position, max_stage_position, adaptivity, quality_bias, turbo_mode, correction_granularity):
|
| 783 |
+
sampler = comfy.samplers.KSAMPLER(
|
| 784 |
+
sample_h3_adaptive_chebyshev_auto,
|
| 785 |
+
extra_options={
|
| 786 |
+
"coordinate": coordinate,
|
| 787 |
+
"base_stage_position": base_stage_position,
|
| 788 |
+
"min_stage_position": min_stage_position,
|
| 789 |
+
"max_stage_position": max_stage_position,
|
| 790 |
+
"adaptivity": adaptivity,
|
| 791 |
+
"quality_bias": quality_bias,
|
| 792 |
+
"turbo_mode": turbo_mode,
|
| 793 |
+
"correction_granularity": correction_granularity,
|
| 794 |
+
},
|
| 795 |
+
)
|
| 796 |
+
return (sampler,)
|
| 797 |
+
|
| 798 |
+
|
| 799 |
+
# Backward-compatible alias for earlier pack versions.
|
| 800 |
+
H3AdaptiveChebyshevProSampler = H3AdaptiveChebyshevAdvancedSampler
|
| 801 |
+
|
| 802 |
+
|
| 803 |
+
NODE_CLASS_MAPPINGS = {
|
| 804 |
+
"H3AdaptiveChebyshevAdvancedSampler": H3AdaptiveChebyshevAdvancedSampler,
|
| 805 |
+
"H3AdaptiveChebyshevAutoSampler": H3AdaptiveChebyshevAutoSampler,
|
| 806 |
+
"H3AdaptiveChebyshevProSampler": H3AdaptiveChebyshevAdvancedSampler,
|
| 807 |
+
}
|
| 808 |
+
|
| 809 |
+
NODE_DISPLAY_NAME_MAPPINGS = {
|
| 810 |
+
"H3AdaptiveChebyshevAdvancedSampler": "ChebyFlow Advanced Sampler",
|
| 811 |
+
"H3AdaptiveChebyshevAutoSampler": "ChebyFlow Auto Sampler",
|
| 812 |
+
"H3AdaptiveChebyshevProSampler": "ChebyFlow Advanced Sampler",
|
| 813 |
+
}
|