StillDeadcode's picture
Upload README.md with huggingface_hub
58b0732 verified
|
Raw History Blame Contribute Delete
7.4 kB
metadata
license: other
license_name: ltx-2.x-community-license-agreement
license_link: https://huggingface.co/StillDeadcode/radiance-ltx-2.5-fp8/blob/main/LICENSE
pipeline_tag: text-to-video
library_name: radiance
extra_gated_prompt: >-
  This is a modified copy of LTX-2.5 (Lightricks/LTX-2.5-Diffusers), distributed
  under the LTX-2.x Community License Agreement (LICENSE in this repository).
  Its use is subject to that agreement, including Section 4 and Attachment A
  (use restrictions), in their entirety. Entities with annual revenues of at
  least US$10,000,000 must obtain a paid license from Lightricks
  (licensing@ltx.io) before any use other than a Non-Commercial Purpose.
extra_gated_button_content: Agree to the LTX-2.x Community License
tags:
  - radiance
  - rocm
  - amd
  - rdna4
  - fp8
  - text-to-video
  - image-to-video
  - text-to-audio-video
base_model:
  - Lightricks/LTX-2.5-Diffusers

LTX-2.5, FP8 — radiance container

LTX-2.5 — video with a synchronized stereo soundtrack from text, or from text and a first frame — as a single .rad container for the radiance inference engine (AMD RDNA4, ROCm).

This is not an official Lightricks release and is not endorsed by Lightricks.

File ltx-2.5-fp8.rad — 36.73 GiB
Weights the Gemma 4 text encoder's, the text connectors' and the diffusion transformer's linears FP8 E4M3 per 128×128 block (bf16 scales); the vocoders f32; everything else (the video and audio VAEs, the latent upsampler, the duration head, the diffusion decoder, the small linears and norms) as released, bf16
Steps 30 by default (num_inference_steps, up to 64), each up to four evaluations of the DiT (classifier-free, spatio-temporal and modality guidance; the defaults are the release's: video 3, 1, 3 with rescale 0.7, audio 7, 1, 3 with rescale 0.7)
Needs radiance built from main after 1.8.1 — LTX-2.5 support is not in 1.8.1 and comes with the next release

Serve

radiance --model ltx-2.5-fp8.rad --vision-tower-placement ram

LTX-2.5 runs on one card (--tp above 1 is refused). The text encoder (13 GiB in FP8) does not fit a 32 GB card beside the DiT: --vision-tower-placement ram keeps it in pinned host memory, read once a prompt.

The server speaks OpenAI's video API: POST /v1/videos queues a job (JSON, or multipart as OpenAI's SDKs send it), GET /v1/videos/{id} reports its progress and GET /v1/videos/{id}/content downloads the MP4 (H.264 and 48 kHz stereo AAC).

curl -s http://localhost:8000/v1/videos -H 'Content-Type: application/json' -d '{
  "prompt": "A red fox trots through fresh snow at golden hour, snow crunching under its paws",
  "size": "960x544", "seed": 7}'
  • seconds: rounded up to steps of 8 frames at 24 a second, 17 to 161 frames. Left out (or "auto"), the release's duration head chooses the length the prompt implies, as diffusers does when num_frames is omitted.
  • size: sides multiples of 32 from 256, at most 1920x1088 of area (121 frames at that size), at most 3:1; 960x544 by default. A frame larger than 960x544 is decoded in diffusers' VAE tiles.
  • input_reference: a first frame (image to video), resized to size and put through H.264 at CRF 18 first, as the release was trained.
  • decoder: "vae" (the default) or "diffusion", the release's diffusion decoder: neighbourhood-attention stages that upsample the latent, then one diffusion step over pixel patches, a tile at a time with diffusers' tiling. Frames up to 960x544 of area (a larger size is refused with it). Its weights stay in host memory with --vision-tower-placement ram.
  • Guidance: negative_prompt (empty by default), guidance_scale, stg_scale, modality_scale, guidance_rescale, and the same prefixed audio_ for the soundtrack. A scale of 1 (0 for stg_scale) drops that evaluation.

On one Radeon AI PRO R9700 (gfx1201) at the default 30 guided steps: 3 seconds (73 frames) at 768x512 in about 112 s, 5 seconds (121 frames) at 960x544 in about 262 s. "decoder": "diffusion" adds about 22 s at 960x544 (121 frames) over the VAE's 2.

The model was trained on long, single-paragraph captions describing the shot, the motion, the light and the sound; short prompts give generic results.

How this file was made

The full DiT is transformer_full/ of the diffusers download, read as the pipeline's transformer, with the scheduler's shifting turned back on as the release's README does:

mkdir ltx-2.5-full && ln -s Lightricks/LTX-2.5-Diffusers/* ltx-2.5-full/
rm ltx-2.5-full/transformer ltx-2.5-full/transformer_full
ln -s Lightricks/LTX-2.5-Diffusers/transformer_full ltx-2.5-full/transformer
rad-convert ltx-2.5-full --recipe ltx-2.5-fp8.recipe -o ltx-2.5-fp8.rad \
    --set scheduler.use_dynamic_shifting=1 --set scheduler.shift_terminal=0.1

The recipe (ltx-2.5-fp8.recipe in this repository) names what is quantised or cast; everything it does not name is the checkpoint's own dtype:

te.blk.*.attn_q.weight               rtn  codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_k.weight               rtn  codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_v.weight               rtn  codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_output.weight          rtn  codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.ffn_gate_up.weight          rtn  codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.ffn_down.weight             rtn  codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.proj.weight                   rtn  codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.qkv.weight              rtn  codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.out.weight              rtn  codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.ff_up.weight            rtn  codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.ff_down.weight          rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.qkv.weight                 rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.q.weight                   rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.kv.weight                  rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.out.weight                 rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.ff_up.weight               rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.ff_down.weight             rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.audio_ff_up.weight         rtn  codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.audio_ff_down.weight       rtn  codes=fp8_e4m3 block=128x128 scale=bf16
voc.conv_*                           cast dtype=f32
voc.upsamplers.*                     cast dtype=f32
voc.resnets.*                        cast dtype=f32
voc.act_out.*                        cast dtype=f32
bwe.*                                cast dtype=f32

License

LTX-2.5 is licensed under the LTX-2.x Community License Agreement (LICENSE), and its use is subject to that agreement in its entirety, including the use restrictions of Section 4 and Attachment A. Entities with annual revenues of at least US$10,000,000 must obtain a paid license from Lightricks before any use other than a Non-Commercial Purpose. This container is a modified copy (NOTICE says how); no rights beyond the agreement's are granted with it.