Video-to-Video
Diffusers
English
video-editing
video-restoration
ltx-video
ltx-2-3
dit
ic-lora
watermark-removal
subtitle-removal
super-resolution
hd-enhancement
joyfox
Instructions to use joyfox/LTX2.3-ICEdit-Insight with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use joyfox/LTX2.3-ICEdit-Insight with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("joyfox/LTX2.3-ICEdit-Insight", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Codex commited on
Commit ยท
d53b4d8
1
Parent(s): 9e741c4
Update model card with paper links and citation
Browse files
README.md
CHANGED
|
@@ -5,7 +5,7 @@ language:
|
|
| 5 |
library_name: diffusers
|
| 6 |
base_model:
|
| 7 |
- Lightricks/LTX-2.3
|
| 8 |
-
pipeline_tag:
|
| 9 |
tags:
|
| 10 |
- video-editing
|
| 11 |
- video-restoration
|
|
@@ -44,7 +44,31 @@ This release focuses on four practical video editing directions:
|
|
| 44 |
|
| 45 |
Unlike conventional frame-level enhancement pipelines, this model family operates as a **generative video restoration system** in latent video space. It is designed to preserve global structure, camera motion, object identity, and temporal consistency while reconstructing missing or degraded visual content.
|
| 46 |
|
| 47 |
-
Project links: [GitHub project](https://github.com/Valiant-Cat/LTX2-ICEdit-Insight) | [JoyFox on Hugging Face](https://huggingface.co/joyfox)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
|
| 49 |
## ๐ฆ Model Files
|
| 50 |
|
|
@@ -263,6 +287,22 @@ This is especially helpful for recovering:
|
|
| 263 |
- Frame count should follow the `8k + 1` rule.
|
| 264 |
- Output height and width should be multiples of `32` in single-stage inference.
|
| 265 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 266 |
## ๐๏ธ Training
|
| 267 |
|
| 268 |
This model family was trained and optimized by **JoyFox Lab** (**Chengdu Xuanhu Technology Co., Ltd.**).
|
|
|
|
| 5 |
library_name: diffusers
|
| 6 |
base_model:
|
| 7 |
- Lightricks/LTX-2.3
|
| 8 |
+
pipeline_tag: image-to-video
|
| 9 |
tags:
|
| 10 |
- video-editing
|
| 11 |
- video-restoration
|
|
|
|
| 44 |
|
| 45 |
Unlike conventional frame-level enhancement pipelines, this model family operates as a **generative video restoration system** in latent video space. It is designed to preserve global structure, camera motion, object identity, and temporal consistency while reconstructing missing or degraded visual content.
|
| 46 |
|
| 47 |
+
Project links: [GitHub project](https://github.com/Valiant-Cat/LTX2-ICEdit-Insight) | [JoyFox on Hugging Face](https://huggingface.co/joyfox) | [Paper (Research Square)](https://www.researchsquare.com/article/rs-9775063/v1) | [DOI](https://doi.org/10.21203/rs.3.rs-9775063/v1)
|
| 48 |
+
|
| 49 |
+
## ๐ Paper
|
| 50 |
+
|
| 51 |
+
- **Title**: *LTX-Insight: Unified Video Restoration and Semantic Editing via Task-Aware Adaptation and Temporal Consistency*
|
| 52 |
+
- **Authors**: Fan Tang, Siyuan Li
|
| 53 |
+
- **Venue**: Research Square
|
| 54 |
+
- **Posted date**: May 25, 2026
|
| 55 |
+
- **DOI**: [`10.21203/rs.3.rs-9775063/v1`](https://doi.org/10.21203/rs.3.rs-9775063/v1)
|
| 56 |
+
- **Paper URL**: https://www.researchsquare.com/article/rs-9775063/v1
|
| 57 |
+
|
| 58 |
+
This model release corresponds to the paper's unified video post-processing framework built around three high-level settings: video super-resolution/enhancement, occlusion removal and repair, and instruction-driven semantic editing.
|
| 59 |
+
|
| 60 |
+
In the currently released inference package, those ideas are exposed through four practical routes:
|
| 61 |
+
|
| 62 |
+
- **Video Restoration**
|
| 63 |
+
- **Video HD Enhancement**
|
| 64 |
+
- **Watermark Removal**
|
| 65 |
+
- **Subtitle Removal**
|
| 66 |
+
|
| 67 |
+
The paper introduces three core components:
|
| 68 |
+
|
| 69 |
+
- **TADA (Task-Aware Decoupled Adapter)** for separating structure restoration from texture/semantic editing under a shared backbone.
|
| 70 |
+
- **FGLC (Flow-Guided Latent Consistency)** for explicit temporal consistency modeling in latent space using optical-flow alignment and cross-frame attention bias.
|
| 71 |
+
- **PMTR (Progressive Multi-Task Routing)** for more stable joint multi-task training through curriculum scheduling and collaborative routing.
|
| 72 |
|
| 73 |
## ๐ฆ Model Files
|
| 74 |
|
|
|
|
| 287 |
- Frame count should follow the `8k + 1` rule.
|
| 288 |
- Output height and width should be multiples of `32` in single-stage inference.
|
| 289 |
|
| 290 |
+
## ๐ Citation
|
| 291 |
+
|
| 292 |
+
If this model family is useful in your research or application workflow, please cite:
|
| 293 |
+
|
| 294 |
+
```bibtex
|
| 295 |
+
@article{tang2026ltxinsight,
|
| 296 |
+
title = {LTX-Insight: Unified Video Restoration and Semantic Editing via Task-Aware Adaptation and Temporal Consistency},
|
| 297 |
+
author = {Tang, Fan and Li, Siyuan},
|
| 298 |
+
journal = {Research Square},
|
| 299 |
+
year = {2026},
|
| 300 |
+
month = {05},
|
| 301 |
+
doi = {10.21203/rs.3.rs-9775063/v1},
|
| 302 |
+
url = {https://www.researchsquare.com/article/rs-9775063/v1}
|
| 303 |
+
}
|
| 304 |
+
```
|
| 305 |
+
|
| 306 |
## ๐๏ธ Training
|
| 307 |
|
| 308 |
This model family was trained and optimized by **JoyFox Lab** (**Chengdu Xuanhu Technology Co., Ltd.**).
|