| --- |
| base_model: meituan-longcat/LongCat-Video-Avatar-1.5 |
| license: mit |
| pipeline_tag: image-to-video |
| tags: |
| - longcat |
| - image-to-video |
| - video-generation |
| - talking-head |
| - audio-driven |
| - quantized |
| - fp8 |
| - e4m3 |
| base_model_relation: quantized |
| --- |
| # LongCat-Video-Avatar-1.5-e4m3 |
|
|
| FP8 (e4m3) quantized |
| [LongCat-Video-Avatar-1.5](https://huggingface.co/meituan-longcat/LongCat-Video-Avatar-1.5) — |
| Meituan's audio-driven avatar video model, which animates a portrait from a |
| speech track. |
|
|
| ## What was changed |
|
|
| Only the `LongCatVideoAvatarTransformer3DModel` is included here, quantized as |
| described in `quantization_config.json`: |
|
|
| - weights: `float8_e4m3fn`, symmetric, **per output channel** |
| (one fp32 scale per row) |
| - activations: `float8_e4m3fn`, dynamic **per row** |
| - matmul: `torch._scaled_mm` in RowWise mode (Hopper and newer) |
| - compute dtype: bf16 |
| - `final_layer.linear` is skipped and stays bf16 |
|
|
| Per-channel weight scales plus per-row activation scales cost slightly more |
| bookkeeping than a single per-tensor scale, but they keep the output stable on |
| a 48-layer, 4096-wide transformer where a per-tensor scale gets dragged around |
| by outliers. |
|
|
| ## Usage |
|
|
| This repository contains the quantized transformer only. The text encoder, VAE |
| and audio encoder come from the upstream repository; download both and point |
| the upstream inference code at this transformer directory. |
|
|
| Running the fp8 path needs a GPU with `torch._scaled_mm` RowWise support |
| (H100/H200 or newer). On older hardware, use the upstream bf16 weights. |
|
|
| ## License |
|
|
| MIT, inherited from LongCat-Video-Avatar-1.5. |
|
|
