Instructions to use nvidia/Cosmos3-Edge-FP8 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Cosmos
How to use nvidia/Cosmos3-Edge-FP8 with Cosmos:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Use fastest measured image-to-video backend per GPU
Browse filesUse CUDNN_ATTN for the H20 image-to-video result, yielding 3.30 frames/s from 121 frames / 36.677 s. Document the best measured image-to-video backend for each GPU: FLASH_ATTN for H100/H200 and CUDNN_ATTN for H20/RTX PRO 6000. Calculations retain the full precision of the measured latency. Other benchmark values are unchanged.
README.md
CHANGED
|
@@ -302,7 +302,7 @@ The table reports single-GPU inference performance for Cosmos3-Edge-FP8 using FL
|
|
| 302 |
| --- | ---: | ---: | ---: | ---: |
|
| 303 |
| H100 | 13.04 | 108.1 | 312.64 | - |
|
| 304 |
| H200 | 12.96 | 103.0 | 341.18 | - |
|
| 305 |
-
| H20 | 3.
|
| 306 |
| DGX Station | - | - | - | - |
|
| 307 |
| DGX Spark | - | - | - | - |
|
| 308 |
| RTX PRO 6000 | 7.52 | - | - | - |
|
|
@@ -314,10 +314,10 @@ The table reports single-GPU inference performance for Cosmos3-Edge-FP8 using FL
|
|
| 314 |
|
| 315 |
A dash (-) indicates unavailable benchmark results for the specified configuration.
|
| 316 |
|
| 317 |
-
- **Generator:** FP8 image-to-video (`i2v`) with vLLM-Omni, 832 × 480 resolution, 121 generated frames, 35 denoising steps, concurrency 1, and one warmup request. Throughput is 121 divided by end-to-end server inference time in seconds and measures inference throughput. The reported times are 9.277 s (H100), 9.339 s (H200), 36.
|
| 318 |
- **Reasoner:** FP8 vLLM with concurrency 1, 50 input text tokens plus a 30-second, 374 × 374 video sampled at FPS 1 (30 frames), and 100 output tokens. Both metrics use this workload and FLASH_ATTN; output throughput copies the reported output-token throughput.
|
| 319 |
- **Action Policy:** FP8 PyTorch, Policy Robot (DROID), 480p, one GPU, and 32-action chunks. Throughput is 32 divided by average generation time in seconds, reported in actions per second.
|
| 320 |
-
- **Hardware:** Generator measurements use one H100 SXM 80 GB, H200 SXM 141 GB, or H20 SXM 96 GB GPU. The
|
| 321 |
- **Embedded hardware:** Reasoner measurements use one integrated GPU: Jetson AGX Thor T5000 (128 GB), a T5000 devkit configured to emulate T3000 (12 SMs, 8 CPUs, 32 GB), and Jetson AGX Orin (64 GB), all in MAXN mode with clocks locked. Orin runs the FP8 checkpoint with weight-only W8A16 compute; Thor uses FP8 compute. T3000 results are emulated measurements. Hardware variants for other rows require matching benchmark results.
|
| 322 |
- Throughputs are calculated before rounding to two decimal places; time to first token is displayed to one decimal place. No estimates are included.
|
| 323 |
|
|
|
|
| 302 |
| --- | ---: | ---: | ---: | ---: |
|
| 303 |
| H100 | 13.04 | 108.1 | 312.64 | - |
|
| 304 |
| H200 | 12.96 | 103.0 | 341.18 | - |
|
| 305 |
+
| H20 | 3.30 | 163.6 | 262.42 | - |
|
| 306 |
| DGX Station | - | - | - | - |
|
| 307 |
| DGX Spark | - | - | - | - |
|
| 308 |
| RTX PRO 6000 | 7.52 | - | - | - |
|
|
|
|
| 314 |
|
| 315 |
A dash (-) indicates unavailable benchmark results for the specified configuration.
|
| 316 |
|
| 317 |
+
- **Generator:** FP8 image-to-video (`i2v`) with vLLM-Omni, 832 × 480 resolution, 121 generated frames, 35 denoising steps, concurrency 1, and one warmup request. Each GPU uses its fastest measured backend for this image-to-video workload: FLASH_ATTN for H100 and H200; CUDNN_ATTN for H20 and RTX PRO 6000. Throughput is 121 divided by end-to-end server inference time in seconds and measures inference throughput. The reported times are 9.277 s (H100), 9.339 s (H200), 36.677 s (H20), and 16.085 s (RTX PRO 6000).
|
| 318 |
- **Reasoner:** FP8 vLLM with concurrency 1, 50 input text tokens plus a 30-second, 374 × 374 video sampled at FPS 1 (30 frames), and 100 output tokens. Both metrics use this workload and FLASH_ATTN; output throughput copies the reported output-token throughput.
|
| 319 |
- **Action Policy:** FP8 PyTorch, Policy Robot (DROID), 480p, one GPU, and 32-action chunks. Throughput is 32 divided by average generation time in seconds, reported in actions per second.
|
| 320 |
+
- **Hardware:** Generator measurements use one H100 SXM 80 GB, H200 SXM 141 GB, or H20 SXM 96 GB GPU. The RTX PRO 6000 generator measurement uses one RTX PRO 6000 Blackwell Server Edition GPU.
|
| 321 |
- **Embedded hardware:** Reasoner measurements use one integrated GPU: Jetson AGX Thor T5000 (128 GB), a T5000 devkit configured to emulate T3000 (12 SMs, 8 CPUs, 32 GB), and Jetson AGX Orin (64 GB), all in MAXN mode with clocks locked. Orin runs the FP8 checkpoint with weight-only W8A16 compute; Thor uses FP8 compute. T3000 results are emulated measurements. Hardware variants for other rows require matching benchmark results.
|
| 322 |
- Throughputs are calculated before rounding to two decimal places; time to first token is displayed to one decimal place. No estimates are included.
|
| 323 |
|