ZYVIDEO 1.2

Text-to-video, 256Γ—256, 16 or 32 frames @ 8 fps, plus chained long videos. Same architecture as ZYVIDEO 1.1 (built on ZYI-256 Flash). What changed, in a short fine-tune:

  • Anatomy and quality: high-quality FLUX-Reason-6M images (people and animals first) trained together with the videos, and only the most aesthetic OpenVid clips, half of them with people or animals.
  • Long videos: a second stage at 32 frames, and pipe.long(...) to chain several clips into one video.

samples

Chained long video: longo_encadeado.mp4

Parameters 490M (343M spatial from ZYI-256 Flash + 147M temporal)
Latent DC-AE f32c32 per frame β€” 8 Γ— 8 Γ— 32 per frame (64 tokens)
Text flan-T5-base (frozen), English prompts
Objective rectified flow, logit-normal t with shift 2.0, + 0.5Γ— loss on frame-to-frame differences
Data OpenVid-1M (streamed, filter "quality"), 16,890 unique clips + 146,944 image samples from FLUX-Reason-6M
Training +1,148 steps on top of 1.1 (31,584 videos), 1Γ— RTX PRO 6000, 90 min session
Validation loss, videos - (ZYVIDEO 1.1: -)
Validation loss, images of people/animals 1.1663 (ZYVIDEO 1.1: 3.1844)
Average motion of the samples (0-255) 9.33 (ZYVIDEO 1.1: 11.65)

Usage

import sys
from huggingface_hub import snapshot_download
d = snapshot_download("caikybaldo999/zyvideo-1.2")
sys.path.insert(0, d)
from zyvideo import ZyVideoPipeline, save_mp4
pipe = ZyVideoPipeline(d)
save_mp4(pipe("a woman walking on a sidewalk, full body", seed=0)[0], "out.mp4", fps=8)                  # 16 frames
save_mp4(pipe("a woman walking on a sidewalk, full body", frames=32, seed=0)[0], "out32.mp4", fps=8)
save_mp4(pipe.long("a man walking along a beach at sunset", segments=3, seed=0), "long.mp4", fps=8)        # ~10 s

Requires torch, diffusers, transformers, safetensors, huggingface_hub, imageio, imageio-ffmpeg. The stage-1 weights (before the long-video stage) are in etapa1/.

Limitations

256Γ—256. Each latent token covers 32Γ—32 pixels, so hands, small faces and text are still hard. Long chained videos can drift after a few segments. English prompts only.

Downloads last month
92
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for caikybaldo999/zyvideo-1.2

Unable to build the model tree, the base model loops to the model itself. Learn more.