Card: measured one-model-at-a-time offload (72.2 s, 50.4 GiB peak) and memory modes

README.md

@@ -61,6 +61,21 @@ name. Editing: pass `image=reference` (or a list) and refer to the images as `<P
 prompt. The prompting guidance of the original card applies unchanged: long, complete English descriptions;
 recommended sizes from 4096×4096 (1:1) to 6272×2688 (21:9).
 
+### Memory modes
+
+`enable_auto_cpu_offload` moves a model off the GPU only when the next one does not fit. On a 96 GB card the pruned
+models fit together, so after the first image nothing moves and the run is as fast as with everything on the GPU
+(peak 92.7 GiB). For the lowest peak, keep one model on the GPU at a time; set the strategy after loading, because
+adding the components resets it to the default:
+
+```python
+pipe.load_components(dtype=torch.bfloat16)
+manager.set_offload_strategy(lambda hooks, **kwargs: hooks)  # move every other model off the GPU first
+```
+
+The peak is then 50-52 GiB, set by the text encoder (48.9 GiB of weights), and each image moves both models over
+PCIe: about 72 s at 2048×2048 instead of 5.6 s (see Profiles).
+
 The components also load alone with the stock classes: `MiniMaxH3ImageTransformer2DModel.from_pretrained(repo,
 subfolder="transformer")` and `Qwen3VLForConditionalGeneration.from_pretrained(repo, subfolder="text_encoder")`.
 
@@ -95,15 +110,20 @@ warm-up image. Time includes prompt encoding, six steps and VAE decoding.
 
 | Run | 2048×2048 | Edit, 2048×2048 | 4736×3552 | GPU memory peak, 4736×3552 |
 |---|---|---|---|---|
-| Original, model offload | 97.6 s | 101.6 s | 120.8 s | 68.2 GiB |
-| Pruned, model offload (code above) | 5.6 s | 9.4 s | 26.4 s | 92.7 GiB |
+| Original, model offload (code of the original card) | 97.6 s | 101.6 s | 120.8 s | 68.2 GiB |
+| Pruned, one model on the GPU at a time | 72.2 s | 76.0 s | 92.8 s | 50.4 GiB (edit: 51.7 GiB) |
 | Pruned, all on the GPU (`pipe.to("cuda")`) | 5.6 s | 9.3 s | 26.7 s | 92.7 GiB |
 
-The time difference comes from memory, not from compute: the original (124 GiB of transformer and text encoder) does
-not fit a 96 GB card, so the offload manager moves both models for every image. The pruned models (86.4 GiB)
-stay on the GPU. The sampling steps themselves cost the same. On a GPU where the original fits (141 GB or more),
-expect similar times for both; on an 80 GB GPU the pruned models also take turns on the GPU, with less data to move
-(neither case was measured).
+The original does not fit a 96 GB card (124 GiB of transformer and text encoder), so the offload manager moves the
+transformer and the text encoder for every image. With one model on the GPU at a time the pruned checkpoint does the
+same, with 30 % less data to move. With everything on the GPU nothing moves; the stock auto offload (the code above)
+measured the same on this card, 5.6 s / 9.4 s / 26.4 s with a 92.7 GiB peak. The sampling steps themselves cost the
+same in all rows. GPUs with 80 GB or with 141 GB or more were not measured.
+
+All rows ran on the same machine. With every model on the GPU (auto offload on this card included), the edit took
+14-22 s instead of 9.3 s in three of five runs: the text encoder's pass over the reference image took 5-9 s instead of
+under 1 s. The cause is not identified; the table shows the fast runs.
+With one model on the GPU at a time the images are bit-identical to the all-on-GPU ones.
 
 ## Verification