wlsaidhi commited on
Commit
32b98cf
·
verified ·
1 Parent(s): d91ad12

Use FastH3 Omni name and add local AI recipes

Browse files
Files changed (1) hide show
  1. README.md +21 -6
README.md CHANGED
@@ -9,12 +9,12 @@ tags:
9
  - text-to-audio-video
10
  - fastvideo
11
  - minimax-h3
12
- - omniref
13
  ---
14
 
15
- # FastH3 OmniRef — BF16 Release
16
 
17
- **This is the final BF16 release checkpoint of FastH3 OmniRef.** It generates video with synchronized audio from a text prompt and ordered image, video and audio references, using eight transformer evaluations.
18
 
19
  [All release variants](https://huggingface.co/collections/FastVideo/fasth3-omniref) · [UniServe](https://github.com/hao-ai-lab/uniserve) · [FastVideo](https://github.com/hao-ai-lab/FastVideo)
20
 
@@ -25,11 +25,26 @@ tags:
25
  | NVIDIA GB200 / B200 | [UniServe deployment guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#deployments) |
26
  | NVIDIA RTX PRO 6000 Blackwell | [UniServe deployment guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#deployments) |
27
  | NVIDIA Hopper, including H100 / H200 | [UniServe requirements and setup](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#requirements) |
28
- | NVIDIA DGX Spark (GB10) / RTX 5090 | [FastVideo local OmniRef guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) with the [NVFP4 QAD variant](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD) |
29
 
30
  For server deployment, use this BF16 checkpoint with UniServe's reference-conditioned (`ref2va`) configurations. The guide covers installation, model downloads, GPU placement, request capacities and the video/audio API. For the BF16 deployment, choose UniServe's `quality` precision preset. NVFP4 presets require Blackwell hardware.
31
 
32
- For local AI on Spark or RTX 5090, use FastVideo's device profiles and the NVFP4 QAD checkpoint. Follow the guide's memory, offload, AdaLN-table and decoder setup for your device. For visual workflows, use the [ComfyUI variants](https://huggingface.co/FastVideo/FastH3-OmniRef-Comfy).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
33
 
34
  ## Inputs and outputs
35
 
@@ -42,7 +57,7 @@ For local AI on Spark or RTX 5090, use FastVideo's device profiles and the NVFP4
42
 
43
  ```bash
44
  hf download FastVideo/FastVideo-FastH3-Omni-8-Step-V1 \
45
- --local-dir ./FastH3-OmniRef-BF16
46
  ```
47
 
48
  The repository includes the BF16 reference transformer, schedulers and inference configuration. The runtime loads the shared text encoder, tokenizer, processor and video/audio VAEs from MiniMax-H3. Follow the runtime guide to prepare those components.
 
9
  - text-to-audio-video
10
  - fastvideo
11
  - minimax-h3
12
+ - fasth3-omni
13
  ---
14
 
15
+ # FastH3 Omni — BF16 Release
16
 
17
+ **This is the final BF16 release checkpoint of FastH3 Omni.** It generates video with synchronized audio from a text prompt and ordered image, video and audio references, using eight transformer evaluations.
18
 
19
  [All release variants](https://huggingface.co/collections/FastVideo/fasth3-omniref) · [UniServe](https://github.com/hao-ai-lab/uniserve) · [FastVideo](https://github.com/hao-ai-lab/FastVideo)
20
 
 
25
  | NVIDIA GB200 / B200 | [UniServe deployment guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#deployments) |
26
  | NVIDIA RTX PRO 6000 Blackwell | [UniServe deployment guide](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#deployments) |
27
  | NVIDIA Hopper, including H100 / H200 | [UniServe requirements and setup](https://github.com/hao-ai-lab/uniserve/blob/main/docs/minimax_h3/minimax_h3.md#requirements) |
28
+ | NVIDIA DGX Spark (GB10) / RTX 5090 | [FastVideo local FastH3 Omni guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) with the [NVFP4 QAD variant](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD) |
29
 
30
  For server deployment, use this BF16 checkpoint with UniServe's reference-conditioned (`ref2va`) configurations. The guide covers installation, model downloads, GPU placement, request capacities and the video/audio API. For the BF16 deployment, choose UniServe's `quality` precision preset. NVFP4 presets require Blackwell hardware.
31
 
32
+ ## Local AI on DGX Spark and RTX 5090
33
+
34
+ Use **FastVideo** with the [NVFP4 QAD variant](https://huggingface.co/FastVideo/FastH3-OmniRef-NVFP4-QAD) for local inference. QAD means quantization-aware distillation; these weights are trained for NVFP4 execution on Blackwell. Start with the [FastVideo cookbook](https://haoailab.com/FastVideo/cookbook/) and [FastH3 Omni recipe builder](https://haoailab.com/FastVideo/cookbook/minimax-h3/fasth3-omniref/).
35
+
36
+ | Device | Setup and inference recipe |
37
+ | --- | --- |
38
+ | DGX Spark (GB10, 128 GB unified memory) | [Install FastVideo](https://haoailab.com/FastVideo/getting_started/installation/spark/) · [`spark` recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_spark.yaml) |
39
+ | RTX 5090 (32 GB) | [Install FastVideo](https://haoailab.com/FastVideo/getting_started/installation/gpu/) · [`rtx-5090` recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx5090.yaml) |
40
+
41
+ Both profiles default to 480p. Spark keeps the model components resident; the RTX 5090 offloads the text encoder and requires a precomputed AdaLN table. Follow the [local inference guide](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md) to compose the model directory, prepare the table and choose a decoder. Keep the device recipe's attention and precision settings.
42
+
43
+ For FP8, see the [FastH3 Omni FP8 weights](https://huggingface.co/FastVideo/FastH3-OmniRef-FP8) and [FP8 device setup](https://github.com/hao-ai-lab/FastVideo/blob/main/docs/inference/fasth3_omniref.md#device-profiles), including the [RTX 4090 recipe](https://github.com/hao-ai-lab/FastVideo/blob/main/examples/inference/basic/basic_fasth3_omniref_rtx4090.yaml) for hardware without FP4 support. Its bundled NVFP4 text encoder requires Blackwell; non-FP4 hardware needs compatible BF16 encoder components. FP8 and NVFP4 QAD outputs can differ from BF16.
44
+
45
+ The optional [NVFP4 VAE decoders](https://huggingface.co/FastVideo/FastH3-NVFP4-VAE-Decoder) offer 36-layer quality and 8-layer fast tiers. The guide documents decoder selection with `--decoder`. These local variants have their own memory and precision settings.
46
+
47
+ For visual workflows, use the [ComfyUI variants](https://huggingface.co/FastVideo/FastH3-OmniRef-Comfy).
48
 
49
  ## Inputs and outputs
50
 
 
57
 
58
  ```bash
59
  hf download FastVideo/FastVideo-FastH3-Omni-8-Step-V1 \
60
+ --local-dir ./FastH3-Omni-BF16
61
  ```
62
 
63
  The repository includes the BF16 reference transformer, schedulers and inference configuration. The runtime loads the shared text encoder, tokenizer, processor and video/audio VAEs from MiniMax-H3. Follow the runtime guide to prepare those components.