Document multi-reference in-context editing
Browse files
README.md
CHANGED
|
@@ -41,6 +41,8 @@ Qwen3 text encoder.
|
|
| 41 |
| `VAEEncoder.aimodel` | 1024×1024 RGB image → latent (image-to-image / editing) |
|
| 42 |
| `Transformer_edit.aimodel` | In-context edit DiT — 1024, sequence 8192 (output + 1 reference) |
|
| 43 |
| `Transformer_edit_512.aimodel` | In-context edit DiT — 512, sequence 2048 |
|
|
|
|
|
|
|
| 44 |
| `tokenizer/`, `pipeline.json`, `vae_bn_*.npy` | Sidecar assets (auto-loaded) |
|
| 45 |
|
| 46 |
Weights are 4-bit quantized (int4, per-block, block size 32); compute precision
|
|
@@ -95,6 +97,14 @@ The stock apple/coreai-models runtime does text- and image-to-image only.
|
|
| 95 |
|
| 96 |
int4, ~25 s for a 1024 edit on a Mac GPU (4 steps, guidance 1.0).
|
| 97 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 98 |
## How it was converted
|
| 99 |
|
| 100 |
```bash
|
|
|
|
| 41 |
| `VAEEncoder.aimodel` | 1024×1024 RGB image → latent (image-to-image / editing) |
|
| 42 |
| `Transformer_edit.aimodel` | In-context edit DiT — 1024, sequence 8192 (output + 1 reference) |
|
| 43 |
| `Transformer_edit_512.aimodel` | In-context edit DiT — 512, sequence 2048 |
|
| 44 |
+
| `Transformer_edit_2ref.aimodel` | Two-reference edit DiT — 1024, sequence 12288 (output + 2 references) |
|
| 45 |
+
| `Transformer_edit_2ref_512.aimodel` | Two-reference edit DiT — 512, sequence 3072 |
|
| 46 |
| `tokenizer/`, `pipeline.json`, `vae_bn_*.npy` | Sidecar assets (auto-loaded) |
|
| 47 |
|
| 48 |
Weights are 4-bit quantized (int4, per-block, block size 32); compute precision
|
|
|
|
| 97 |
|
| 98 |
int4, ~25 s for a 1024 edit on a Mac GPU (4 steps, guidance 1.0).
|
| 99 |
|
| 100 |
+
### Multi-reference
|
| 101 |
+
|
| 102 |
+
**`Transformer_edit_2ref.aimodel`** takes **two** reference images at once — each concatenated at
|
| 103 |
+
its own time index (`T=10`, `T=20`) — so the instruction can combine them: *"put the subject from
|
| 104 |
+
the first image into the scene from the second image."* Same mechanism, longer sequence (12288).
|
| 105 |
+
`editImages(referenceImages:)` selects the 1- or 2-reference transformer by the number of images.
|
| 106 |
+
int4, ~43 s for a 1024 two-reference edit on a Mac GPU.
|
| 107 |
+
|
| 108 |
## How it was converted
|
| 109 |
|
| 110 |
```bash
|