| --- |
| tags: |
| - comfyui |
| - image-editing |
| - joyai |
| license: apache-2.0 |
| library_name: ComfyUI |
| pipeline_tag: image-to-image |
| base_model: jdopensource/JoyAI-Image-Edit-Diffusers |
| --- |
| |
| # JoyAI-Image-Edit-ComfyUI |
|
|
| Repackaged model files for ComfyUI. |
|
|
| Original model repository: https://huggingface.co/jdopensource/JoyAI-Image-Edit-Diffusers |
|
|
| You can also use the same weight in: https://huggingface.co/Comfy-Org/JoyAI-Image-Edit |
|
|
| Place the files in the following folders: |
|
|
| ``` |
| ComfyUI/ |
| `-- models/ |
| |-- diffusion_models/ |
| | |-- joyai_image_edit_bf16.safetensors |
| | `-- joyai_image_edit_int8_convrot.safetensors |
| |-- text_encoders/ |
| | |-- qwen3vl_8b_joyimage_edit_bf16.safetensors |
| | `-- qwen3vl_8b_joyimage_edit_int8_convrot.safetensors |
| `-- vae/ |
| `-- wan_2.1_vae.safetensors |
| ``` |
|
|
| Download directly into `ComfyUI/models/`: |
|
|
| ```bash |
| hf download jdopensource/JoyAI-Image-Edit-ComfyUI \ |
| --local-dir /path/to/ComfyUI/models |
| ``` |
|
|
|  |
|
|
| Workflow: [workflow_joyimage_edit.json](workflow_joyimage_edit.json) |
|
|
| ## Spatial Editing Reference |
|
|
| JoyAI-Image supports three spatial editing prompt patterns: **Object Move**, **Object Rotation**, and **Camera Control**. For the most stable behavior, we recommend following the prompt templates below as closely as possible. |
| For more information (including data curation and evaluation strategies), please refer to [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit). |
|
|
| ### Object Move |
|
|
| Use this pattern when you want to move a target object into a specified region. |
|
|
| **Prompt template:** |
|
|
| ```text |
| Move the <object> into the red box and finally remove the red box. |
| ``` |
|
|
| **Rules:** |
|
|
| - Replace `<object>` with a clear description of the target object to be moved. |
| - The **red box** indicates the target destination in the image. |
| - The phrase **"finally remove the red box"** means the guidance box should not appear in the final edited result. |
|
|
| **Example:** |
|
|
| ```text |
| Move the apple into the red box and finally remove the red box. |
| ``` |
|
|
| ### Object Rotation |
|
|
| Use this pattern when you want to rotate an object to a specific canonical view. |
|
|
| **Prompt template:** |
|
|
| ```text |
| Rotate the <object> to show the <view> side view. |
| ``` |
|
|
| **Supported `<view>` values:** |
|
|
| ``` |
| front, right, left, rear, front right, front left, rear right, rear left |
| ``` |
|
|
| **Rules:** |
|
|
| - Replace `<object>` with a clear description of the object to rotate. |
| - Replace `<view>` with one of the supported directions above. |
| - This instruction is intended to change the **object orientation**, while keeping the object identity and surrounding scene as consistent as possible. |
|
|
| **Examples:** |
|
|
| ```text |
| Rotate the chair to show the front side view. |
| Rotate the car to show the rear left side view. |
| ``` |
|
|
| ### Camera Control |
|
|
| Use this pattern when you want to change only the camera viewpoint while keeping the 3D scene itself unchanged. |
|
|
| **Prompt template:** |
|
|
| ```text |
| Move the camera. |
| - Camera rotation: Yaw {y_rotation}°, Pitch {p_rotation}°. |
| - Camera zoom: in/out/unchanged. |
| - Keep the 3D scene static; only change the viewpoint. |
| ``` |
|
|
| **Rules:** |
|
|
| - `{y_rotation}` specifies the yaw rotation angle in degrees. |
| - `{p_rotation}` specifies the pitch rotation angle in degrees. |
| - `Camera zoom` must be one of: `in`, `out`,`unchanged` |
| - The last line is important: it explicitly tells the model to preserve the 3D scene content and geometry, and only adjust the camera viewpoint. |
|
|
| **Examples:** |
|
|
| ```text |
| Move the camera. |
| - Camera rotation: Yaw 45°, Pitch 0°. |
| - Camera zoom: in. |
| - Keep the 3D scene static; only change the viewpoint. |
| ``` |
|
|
| ```text |
| Move the camera. |
| - Camera rotation: Yaw -90°, Pitch 20°. |
| - Camera zoom: unchanged. |
| - Keep the 3D scene static; only change the viewpoint. |
| ``` |
|
|
| ### Application |
|
|
| **3D Reconstruction:** |
|
|
| The first and third examples show point clouds with only a single given viewpoint. The second and fourth examples are augmented by [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit), which synthesizes richer spatial observations from the sparse input view. |
|
|
| **Conditional-frames Based Video Generation:** |
|
|
| Given the first frame, [SpatialEdit](https://github.com/EasonXiao-888/SpatialEdit) first generates the final frame of the video, and a video generation model then creates a smooth rotational transition between them while maintaining background consistency. |
|
|
| ## Links |
|
|
| - Source code and documentation: [github.com/jd-opensource/JoyAI-Image](https://github.com/jd-opensource/JoyAI-Image) |
| - Original Diffusers-format weights: [jdopensource/JoyAI-Image-Edit-Diffusers](https://huggingface.co/jdopensource/JoyAI-Image-Edit-Diffusers) |
| - Multi-image edit model for ComfyUI: [jdopensource/JoyAI-Image-Edit-Plus-ComfyUI](https://huggingface.co/jdopensource/JoyAI-Image-Edit-Plus-ComfyUI) |
|
|
