StillDeadcode's picture
Upload README.md with huggingface_hub
58b0732 verified
|
Raw History Blame Contribute Delete
7.4 kB
---
license: other
license_name: ltx-2.x-community-license-agreement
license_link: https://huggingface.co/StillDeadcode/radiance-ltx-2.5-fp8/blob/main/LICENSE
pipeline_tag: text-to-video
library_name: radiance
extra_gated_prompt: >-
This is a modified copy of LTX-2.5 (Lightricks/LTX-2.5-Diffusers), distributed under the LTX-2.x
Community License Agreement (LICENSE in this repository). Its use is subject to that agreement,
including Section 4 and Attachment A (use restrictions), in their entirety. Entities with annual
revenues of at least US$10,000,000 must obtain a paid license from Lightricks (licensing@ltx.io)
before any use other than a Non-Commercial Purpose.
extra_gated_button_content: Agree to the LTX-2.x Community License
tags:
- radiance
- rocm
- amd
- rdna4
- fp8
- text-to-video
- image-to-video
- text-to-audio-video
base_model:
- Lightricks/LTX-2.5-Diffusers
---
# LTX-2.5, FP8 — radiance container
[LTX-2.5](https://huggingface.co/Lightricks/LTX-2.5-Diffusers) — video with a synchronized stereo
soundtrack from text, or from text and a first frame — as a single `.rad` container for the
[radiance](https://codeberg.org/StillDeadcode/radiance) inference engine (AMD RDNA4, ROCm).
This is not an official Lightricks release and is not endorsed by Lightricks.
| | |
|---|---|
| File | `ltx-2.5-fp8.rad` — 36.73 GiB |
| Weights | the Gemma 4 text encoder's, the text connectors' and the diffusion transformer's linears FP8 E4M3 per 128×128 block (bf16 scales); the vocoders f32; everything else (the video and audio VAEs, the latent upsampler, the duration head, the diffusion decoder, the small linears and norms) as released, bf16 |
| Steps | 30 by default (`num_inference_steps`, up to 64), each up to four evaluations of the DiT (classifier-free, spatio-temporal and modality guidance; the defaults are the release's: video 3, 1, 3 with rescale 0.7, audio 7, 1, 3 with rescale 0.7) |
| Needs | radiance built from main after 1.8.1 — LTX-2.5 support is not in 1.8.1 and comes with the next release |
## Serve
```sh
radiance --model ltx-2.5-fp8.rad --vision-tower-placement ram
```
LTX-2.5 runs on one card (`--tp` above 1 is refused). The text encoder (13 GiB in FP8) does not fit a
32 GB card beside the DiT: `--vision-tower-placement ram` keeps it in pinned host memory, read once a
prompt.
The server speaks OpenAI's video API: `POST /v1/videos` queues a job (JSON, or multipart as OpenAI's SDKs
send it), `GET /v1/videos/{id}` reports its progress and `GET /v1/videos/{id}/content` downloads the MP4
(H.264 and 48 kHz stereo AAC).
```sh
curl -s http://localhost:8000/v1/videos -H 'Content-Type: application/json' -d '{
"prompt": "A red fox trots through fresh snow at golden hour, snow crunching under its paws",
"size": "960x544", "seed": 7}'
```
- **`seconds`**: rounded up to steps of 8 frames at 24 a second, 17 to 161 frames. Left out (or `"auto"`),
the release's duration head chooses the length the prompt implies, as diffusers does when `num_frames`
is omitted.
- **`size`**: sides multiples of 32 from 256, at most 1920x1088 of area (121 frames at that size), at most
3:1; 960x544 by default. A frame larger than 960x544 is decoded in diffusers' VAE tiles.
- **`input_reference`**: a first frame (image to video), resized to `size` and put through H.264 at CRF 18
first, as the release was trained.
- **`decoder`**: `"vae"` (the default) or `"diffusion"`, the release's diffusion decoder: neighbourhood-attention
stages that upsample the latent, then one diffusion step over pixel patches, a tile at a time with
diffusers' tiling. Frames up to 960x544 of area (a larger `size` is refused with it). Its weights stay in host memory with `--vision-tower-placement ram`.
- **Guidance**: `negative_prompt` (empty by default), `guidance_scale`, `stg_scale`, `modality_scale`, `guidance_rescale`, and the same prefixed `audio_` for the soundtrack. A scale of 1 (0 for `stg_scale`) drops that evaluation.
On one Radeon AI PRO R9700 (gfx1201) at the default 30 guided steps: 3 seconds (73 frames) at 768x512 in about 112 s, 5 seconds (121 frames) at 960x544 in about 262 s. `"decoder": "diffusion"` adds about 22 s at 960x544 (121 frames) over the VAE's 2.
The model was trained on long, single-paragraph captions describing the shot, the motion, the light and
the sound; short prompts give generic results.
## How this file was made
The full DiT is `transformer_full/` of the diffusers download, read as the pipeline's transformer, with the scheduler's shifting turned back on as the release's README does:
```sh
mkdir ltx-2.5-full && ln -s Lightricks/LTX-2.5-Diffusers/* ltx-2.5-full/
rm ltx-2.5-full/transformer ltx-2.5-full/transformer_full
ln -s Lightricks/LTX-2.5-Diffusers/transformer_full ltx-2.5-full/transformer
rad-convert ltx-2.5-full --recipe ltx-2.5-fp8.recipe -o ltx-2.5-fp8.rad \
--set scheduler.use_dynamic_shifting=1 --set scheduler.shift_terminal=0.1
```
The recipe (`ltx-2.5-fp8.recipe` in this repository) names what is quantised or cast; everything it does
not name is the checkpoint's own dtype:
```
te.blk.*.attn_q.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_k.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_v.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.attn_output.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.ffn_gate_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
te.blk.*.ffn_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.proj.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.qkv.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.out.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.ff_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
conn.*.blk.*.ff_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.qkv.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.q.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.kv.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.out.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.ff_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.ff_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.audio_ff_up.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
dit.blk.*.audio_ff_down.weight rtn codes=fp8_e4m3 block=128x128 scale=bf16
voc.conv_* cast dtype=f32
voc.upsamplers.* cast dtype=f32
voc.resnets.* cast dtype=f32
voc.act_out.* cast dtype=f32
bwe.* cast dtype=f32
```
## License
LTX-2.5 is licensed under the LTX-2.x Community License Agreement (`LICENSE`), and its use is subject to
that agreement in its entirety, including the use restrictions of Section 4 and Attachment A. Entities
with annual revenues of at least US$10,000,000 must obtain a paid license from Lightricks before any use
other than a Non-Commercial Purpose. This container is a modified copy (`NOTICE` says how); no rights
beyond the agreement's are granted with it.