ZYVIDEO 1.2
Text-to-video, 256Γ256, 16 or 32 frames @ 8 fps, plus chained long videos. Same architecture as ZYVIDEO 1.1 (built on ZYI-256 Flash). What changed, in a short fine-tune:
- Anatomy and quality: high-quality FLUX-Reason-6M images (people and animals first) trained together with the videos, and only the most aesthetic OpenVid clips, half of them with people or animals.
- Long videos: a second stage at 32 frames, and
pipe.long(...)to chain several clips into one video.
Chained long video: longo_encadeado.mp4
| Parameters | 490M (343M spatial from ZYI-256 Flash + 147M temporal) |
| Latent | DC-AE f32c32 per frame β 8 Γ 8 Γ 32 per frame (64 tokens) |
| Text | flan-T5-base (frozen), English prompts |
| Objective | rectified flow, logit-normal t with shift 2.0, + 0.5Γ loss on frame-to-frame differences |
| Data | OpenVid-1M (streamed, filter "quality"), 16,890 unique clips + 146,944 image samples from FLUX-Reason-6M |
| Training | +1,148 steps on top of 1.1 (31,584 videos), 1Γ RTX PRO 6000, 90 min session |
| Validation loss, videos | - (ZYVIDEO 1.1: -) |
| Validation loss, images of people/animals | 1.1663 (ZYVIDEO 1.1: 3.1844) |
| Average motion of the samples (0-255) | 9.33 (ZYVIDEO 1.1: 11.65) |
Usage
import sys
from huggingface_hub import snapshot_download
d = snapshot_download("caikybaldo999/zyvideo-1.2")
sys.path.insert(0, d)
from zyvideo import ZyVideoPipeline, save_mp4
pipe = ZyVideoPipeline(d)
save_mp4(pipe("a woman walking on a sidewalk, full body", seed=0)[0], "out.mp4", fps=8) # 16 frames
save_mp4(pipe("a woman walking on a sidewalk, full body", frames=32, seed=0)[0], "out32.mp4", fps=8)
save_mp4(pipe.long("a man walking along a beach at sunset", segments=3, seed=0), "long.mp4", fps=8) # ~10 s
Requires torch, diffusers, transformers, safetensors, huggingface_hub, imageio, imageio-ffmpeg.
The stage-1 weights (before the long-video stage) are in etapa1/.
Limitations
256Γ256. Each latent token covers 32Γ32 pixels, so hands, small faces and text are still hard. Long chained videos can drift after a few segments. English prompts only.
- Downloads last month
- 92
Model tree for caikybaldo999/zyvideo-1.2
Unable to build the model tree, the base model loops to the model itself. Learn more.
