Instructions to use VAST-AI/EdenLive-14B-Diffusers with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use VAST-AI/EdenLive-14B-Diffusers with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("VAST-AI/EdenLive-14B-Diffusers", dtype=torch.bfloat16, device_map="cuda") pipe.to("cuda") prompt = "A man with short gray hair plays a red electric guitar." image = load_image( "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png" ) output = pipe(image=image, prompt=prompt).frames[0] export_to_video(output, "output.mp4") - Notebooks
- Google Colab
- Kaggle
Simplify model card and refresh README audit entry
Browse files- README.md +19 -80
- artifact_manifest.json +23 -5
README.md
CHANGED
|
@@ -13,101 +13,40 @@ tags:
|
|
| 13 |
|
| 14 |
# EdenLive 14B — Diffusers
|
| 15 |
|
| 16 |
-
|
| 17 |
|
| 18 |
-
|
| 19 |
|
| 20 |
-
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
|
| 25 |
|
| 26 |
```python
|
| 27 |
import torch
|
| 28 |
from diffusers import EdenLivePipeline
|
| 29 |
from diffusers.utils import export_to_video, load_image, load_video
|
| 30 |
|
| 31 |
-
|
| 32 |
-
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
|
| 36 |
frames = pipe(
|
| 37 |
-
image=
|
| 38 |
-
video=
|
| 39 |
-
prompt="A character
|
| 40 |
-
num_frames=
|
| 41 |
-
generator=torch.Generator(device="cuda").manual_seed(42),
|
| 42 |
).frames[0]
|
| 43 |
export_to_video(frames, "edenlive.mp4", fps=16)
|
| 44 |
```
|
| 45 |
|
| 46 |
-
|
| 47 |
-
|
| 48 |
-
## Causal and dynamic prompts
|
| 49 |
-
|
| 50 |
-
Override the transformer and scheduler using their standard component loaders:
|
| 51 |
-
|
| 52 |
-
```python
|
| 53 |
-
from diffusers import EdenLiveCausalPipeline, EdenLiveFlowMatchEulerDiscreteScheduler, EdenLiveTransformer3DModel
|
| 54 |
-
|
| 55 |
-
causal_pipe = EdenLiveCausalPipeline.from_pretrained(
|
| 56 |
-
model_id,
|
| 57 |
-
transformer=EdenLiveTransformer3DModel.from_pretrained(
|
| 58 |
-
model_id, subfolder="transformer_causal", torch_dtype=torch.bfloat16
|
| 59 |
-
),
|
| 60 |
-
scheduler=EdenLiveFlowMatchEulerDiscreteScheduler.from_pretrained(model_id, subfolder="scheduler_causal"),
|
| 61 |
-
torch_dtype=dtype,
|
| 62 |
-
).to("cuda")
|
| 63 |
-
a = "A character walks through snow."
|
| 64 |
-
b = "The same character walks through a warm forest."
|
| 65 |
-
frames = causal_pipe(
|
| 66 |
-
image=image,
|
| 67 |
-
video=video,
|
| 68 |
-
chunk_prompts=[a, a, a, b, b, b, a, a],
|
| 69 |
-
num_frames=85,
|
| 70 |
-
guidance_scale=3.0,
|
| 71 |
-
negative_prompt="blurry, distorted",
|
| 72 |
-
generator=torch.Generator(device="cuda").manual_seed(42),
|
| 73 |
-
).frames[0]
|
| 74 |
-
export_to_video(frames, "edenlive-causal.mp4", fps=16)
|
| 75 |
-
```
|
| 76 |
-
|
| 77 |
-
At the default chunk size of three latent frames, 85 video frames require eight chunk prompts. Each prompt applies to one latent chunk, including the final short chunk. Repeated text is encoded once. `chunk_prompts` is for one video, overrides a single `prompt`, and cannot be combined with positive `prompt_embeds`; the negative prompt stays fixed. For a static causal prompt, replace `chunk_prompts` with `prompt=a`. History belongs to the current call and is cleared when it ends.
|
| 78 |
-
|
| 79 |
-
## Optional LightTAE decoder
|
| 80 |
-
|
| 81 |
-
The full Wan VAE is the default decoder. To use LightTAE, load it explicitly and pass it as the `decoder` component when constructing either pipeline:
|
| 82 |
-
|
| 83 |
-
```python
|
| 84 |
-
from diffusers import EdenLiveLightTAEDecoder
|
| 85 |
-
|
| 86 |
-
decoder = EdenLiveLightTAEDecoder.from_pretrained(
|
| 87 |
-
model_id, subfolder="decoder", torch_dtype=torch.bfloat16
|
| 88 |
-
)
|
| 89 |
-
pipe = EdenLivePipeline.from_pretrained(model_id, decoder=decoder, torch_dtype=dtype).to("cuda")
|
| 90 |
-
```
|
| 91 |
-
|
| 92 |
-
For the causal pipeline, add `decoder=decoder` to its loading call above. `save_pretrained()` saves the selected components; reload that saved directory using the corresponding pipeline class.
|
| 93 |
-
|
| 94 |
-
## Conversion and verification
|
| 95 |
-
|
| 96 |
-
```bash
|
| 97 |
-
python scripts/convert_edenlive_to_diffusers.py \
|
| 98 |
-
--source /path/to/original/EdenLive --size 14b --output ./EdenLive-14B-Diffusers
|
| 99 |
-
```
|
| 100 |
-
|
| 101 |
-
`conversion_manifest.json` records the names, shapes, dtypes, and SHA256 hashes of the converted tensor values. The converter strictly checks the state dictionaries and verifies saved/reloaded values. `source_manifest.json` records the original source-file hashes and revision. `artifact_manifest.json` records the files in this package. The directory contains all its components and has no internal symlinks.
|
| 102 |
|
| 103 |
## License
|
| 104 |
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
| Components | License and notices |
|
| 108 |
-
| --- | --- |
|
| 109 |
-
| Transformers, Wan VAE, text components, schedulers, LightTAE | [Apache 2.0](LICENSE) |
|
| 110 |
-
| Trained DINOv3 encoder and projection | [DINOv3 License](proxy_encoder/LICENSE.md), [source notice](proxy_encoder/NOTICE.md) |
|
| 111 |
-
| CLIP image encoder and processor | [MIT](image_encoder/LICENSE.md), [source notice](image_encoder/NOTICE.md) |
|
| 112 |
-
|
| 113 |
-
DINOv3 is by Meta Platforms, Inc.; the trained encoder and projection are derivatives from Vast AI. The original notice and license are retained beside both components. See the [EdenLive project](https://github.com/VAST-AI-Research/EdenLive) for model details and source acknowledgments.
|
|
|
|
| 13 |
|
| 14 |
# EdenLive 14B — Diffusers
|
| 15 |
|
| 16 |
+
[EdenLive](https://github.com/VAST-AI-Research/EdenLive) generates video from a proxy video, a reference image, and a text prompt. This repository includes the 14B bidirectional and causal models for Diffusers.
|
| 17 |
|
| 18 |
+
## Usage
|
| 19 |
|
| 20 |
+
Install the EdenLive integration and dependencies:
|
| 21 |
|
| 22 |
+
```bash
|
| 23 |
+
pip install "git+https://github.com/skygoo2000/diffusers.git@edenlive" \
|
| 24 |
+
torch torchvision "transformers>=4.56.0" accelerate sentencepiece ftfy imageio imageio-ffmpeg
|
| 25 |
+
```
|
| 26 |
|
| 27 |
+
Prepare a reference image and a proxy video with at least 49 frames sampled at 16 fps.
|
| 28 |
|
| 29 |
```python
|
| 30 |
import torch
|
| 31 |
from diffusers import EdenLivePipeline
|
| 32 |
from diffusers.utils import export_to_video, load_image, load_video
|
| 33 |
|
| 34 |
+
pipe = EdenLivePipeline.from_pretrained(
|
| 35 |
+
"VAST-AI/EdenLive-14B-Diffusers",
|
| 36 |
+
torch_dtype={"default": torch.bfloat16, "proxy_encoder": torch.float32, "proxy_projection": torch.float32},
|
| 37 |
+
).to("cuda")
|
| 38 |
+
|
| 39 |
frames = pipe(
|
| 40 |
+
image=load_image("reference.png"),
|
| 41 |
+
video=load_video("proxy-16fps.mp4"),
|
| 42 |
+
prompt="A character explores a snowy landscape.",
|
| 43 |
+
num_frames=49,
|
|
|
|
| 44 |
).frames[0]
|
| 45 |
export_to_video(frames, "edenlive.mp4", fps=16)
|
| 46 |
```
|
| 47 |
|
| 48 |
+
See the [usage guide](https://github.com/skygoo2000/diffusers/blob/edenlive/docs/source/en/api/pipelines/edenlive.md) for causal generation, dynamic prompts, and LightTAE, and the [inference examples](https://github.com/skygoo2000/diffusers/tree/edenlive/examples/edenlive#multiple-gpus) for multi-GPU usage.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 49 |
|
| 50 |
## License
|
| 51 |
|
| 52 |
+
EdenLive components use [Apache 2.0](LICENSE). DINOv3 components are covered by the [DINOv3 License](proxy_encoder/LICENSE.md). CLIP components use the [MIT license](image_encoder/LICENSE.md).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
artifact_manifest.json
CHANGED
|
@@ -14,7 +14,7 @@
|
|
| 14 |
],
|
| 15 |
"package": "EdenLive-14B-Diffusers",
|
| 16 |
"file_count": 64,
|
| 17 |
-
"total_bytes":
|
| 18 |
"provenance": {
|
| 19 |
"source_repository": "VAST-AI/EdenLive",
|
| 20 |
"source_revision": "de40b28f3f6eb6a0f12bb3d3967ca35578273713",
|
|
@@ -879,13 +879,13 @@
|
|
| 879 |
},
|
| 880 |
{
|
| 881 |
"relative_path": "README.md",
|
| 882 |
-
"size_bytes":
|
| 883 |
-
"sha256": "
|
| 884 |
-
"mtime_ns":
|
| 885 |
"stat_unchanged": true,
|
| 886 |
"resolved_within_package": true,
|
| 887 |
"symlink": false,
|
| 888 |
-
"elapsed_seconds": 0.
|
| 889 |
},
|
| 890 |
{
|
| 891 |
"relative_path": "conversion_manifest.json",
|
|
@@ -1537,6 +1537,24 @@
|
|
| 1537 |
},
|
| 1538 |
"previous_manifest_sha256": "5eeae66758d6dba576fa102cab4a304775a378dec51d558219e70ec1641b1607",
|
| 1539 |
"verification": "README SHA256/size/mtime and package total refreshed; original weight hashes and original audit facts retained."
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1540 |
}
|
| 1541 |
],
|
| 1542 |
"readme_publication_update": {
|
|
|
|
| 14 |
],
|
| 15 |
"package": "EdenLive-14B-Diffusers",
|
| 16 |
"file_count": 64,
|
| 17 |
+
"total_bytes": 145736862630,
|
| 18 |
"provenance": {
|
| 19 |
"source_repository": "VAST-AI/EdenLive",
|
| 20 |
"source_revision": "de40b28f3f6eb6a0f12bb3d3967ca35578273713",
|
|
|
|
| 879 |
},
|
| 880 |
{
|
| 881 |
"relative_path": "README.md",
|
| 882 |
+
"size_bytes": 1923,
|
| 883 |
+
"sha256": "f0ec61b9f70c652d62d0d02e1024556ec76e789d7d2b4eb17064f3c0e5210b56",
|
| 884 |
+
"mtime_ns": 1792482661327287082,
|
| 885 |
"stat_unchanged": true,
|
| 886 |
"resolved_within_package": true,
|
| 887 |
"symlink": false,
|
| 888 |
+
"elapsed_seconds": 0.005263928323984146
|
| 889 |
},
|
| 890 |
{
|
| 891 |
"relative_path": "conversion_manifest.json",
|
|
|
|
| 1537 |
},
|
| 1538 |
"previous_manifest_sha256": "5eeae66758d6dba576fa102cab4a304775a378dec51d558219e70ec1641b1607",
|
| 1539 |
"verification": "README SHA256/size/mtime and package total refreshed; original weight hashes and original audit facts retained."
|
| 1540 |
+
},
|
| 1541 |
+
{
|
| 1542 |
+
"at": "2026-10-20T07:51:01.330746+00:00",
|
| 1543 |
+
"reason": "Simplify the public model card to overview, installation, basic inference, documentation links, and licenses.",
|
| 1544 |
+
"files": [
|
| 1545 |
+
"README.md"
|
| 1546 |
+
],
|
| 1547 |
+
"before": {
|
| 1548 |
+
"size_bytes": 5939,
|
| 1549 |
+
"sha256": "25747aedd2b8f6ea9d23a9a4a1f9e7ef5f12c78245a5ad66c99c8708dd1a810d",
|
| 1550 |
+
"mtime_ns": 1792421016687864599
|
| 1551 |
+
},
|
| 1552 |
+
"after": {
|
| 1553 |
+
"size_bytes": 1923,
|
| 1554 |
+
"sha256": "f0ec61b9f70c652d62d0d02e1024556ec76e789d7d2b4eb17064f3c0e5210b56",
|
| 1555 |
+
"mtime_ns": 1792482661327287082
|
| 1556 |
+
},
|
| 1557 |
+
"verification": "Only README bytes and its audit entry changed; original weight entries and audit provenance retained."
|
| 1558 |
}
|
| 1559 |
],
|
| 1560 |
"readme_publication_update": {
|