Update FP8 performance benchmarks and remove unmeasured T2000 row (#6)

- Update FP8 performance benchmarks and remove unmeasured T2000 row (00213e1b98514e65c963df7fc4806eb72c278ad8)
- Remove B200 from model card test hardware list (c9370cc4e13a4017f18da94603b137f4b7b6811e)


Co-authored-by: Sergiy Fefilatyev <fsergiy@users.noreply.huggingface.co>

README.md

@@ -298,26 +298,27 @@ The Edge model is a strong initialization for downstream action tasks. For examp
 
 The table reports single-GPU inference performance for Cosmos3-Edge-FP8 using FLASH_ATTN unless otherwise noted.
 
-| Hardware | Generator — 480p (frames/s) | Reasoner — time to first token (ms) | Reasoner — output token throughput (tokens/s) | Action Policy — 480p (actions/s, 16-action chunks) |
+| Hardware | Generator — 480p (frames/s) | Reasoner — time to first token (ms) | Reasoner — output token throughput (tokens/s) | Action Policy — 480p (actions/s, 32-action chunks) |
 | --- | ---: | ---: | ---: | ---: |
-| H100 | 13.04 | 108.1 | 312.64 | 16.33 |
-| H200 | 12.96 | 103.0 | 341.18 | 16.39 |
-| H20 | 3.30 | 163.6 | 262.42 | 6.05 |
-| DGX Station | - | - | - | - |
-| DGX Spark | - | - | - | - |
-| RTX PRO 6000 | 7.52 | - | - | 6.91 |
-| RTX PRO 5000 | - | - | - | - |
-| Jetson Thor T5000 | - | 205.7 | 76.69 | - |
-| Jetson T3000 | - | 384.3 | 62.16 | - |
-| Jetson T2000 | - | - | - | - |
+| H100 | 13.04 | 108.1 | 312.64 | 20.50 |
+| H200 | 12.96 | 103.0 | 341.18 | 21.11 |
+| H20 | 3.30 | 163.6 | 262.42 | 6.81 |
+| DGX Station | 30.63 | 65.4 | 425.65 | 30.48 |
+| DGX Spark | 1.65 | 213.2 | 69.46 | 4.03 |
+| RTX PRO 6000 | 7.52 | 121.0 | 245.38 | 10.77 |
+| RTX PRO 5000 | 4.51 | 99.7 | 243.16 | 9.58 |
+| Jetson Thor T5000 | 1.92 | 205.7 | 76.69 | 3.79 |
+| Jetson T3000 | 0.88 | 384.3 | 62.16 | 1.81 |
 | Jetson AGX Orin | - | 817.8 | 41.46 | - |
 
 A dash (-) indicates unavailable benchmark results for the specified configuration.
 
-- **Generator:** FP8 image-to-video (`i2v`) with vLLM-Omni, 832 × 480 resolution, 121 generated frames, 35 denoising steps, concurrency 1, and one warmup request. Each GPU uses its fastest measured backend for this image-to-video workload: FLASH_ATTN for H100 and H200; CUDNN_ATTN for H20 and RTX PRO 6000. Throughput is 121 divided by end-to-end server inference time in seconds and measures inference throughput. The reported times are 9.277 s (H100), 9.339 s (H200), 36.677 s (H20), and 16.085 s (RTX PRO 6000).
-- **Reasoner:** FP8 vLLM with concurrency 1, 50 input text tokens plus a 30-second, 374 × 374 video sampled at FPS 1 (30 frames), and 100 output tokens. Both metrics use this workload and FLASH_ATTN; output throughput copies the reported output-token throughput.
-- **Action Policy:** FP8 vLLM-Omni, Policy Robot (DROID), 480p, one GPU, 16-action chunks, 30 denoising steps, concurrency 1, and one warmup request. Throughput is 16 divided by end-to-end server inference time in seconds, reported in actions per second. Each GPU uses its fastest measured backend for this workload: CUDNN_ATTN for H100, H200, H20, and RTX PRO 6000. The reported times are 0.980 s (H100), 0.976 s (H200), 2.645 s (H20), and 2.315 s (RTX PRO 6000).
-- **Hardware:** Generator measurements use one H100 SXM 80 GB, H200 SXM 141 GB, or H20 SXM 96 GB GPU. The RTX PRO 6000 generator measurement uses one RTX PRO 6000 Blackwell Server Edition GPU.
+- **Generator:** FP8 image-to-video (`i2v`) with vLLM-Omni, 832 × 480 resolution, 121 generated frames, 35 denoising steps, concurrency 1, and one warmup request. Each GPU uses its fastest measured backend for this image-to-video workload: FLASH_ATTN for H100 and H200; CUDNN_ATTN for H20, RTX PRO 6000, DGX Station, DGX Spark, RTX PRO 5000, Thor T5000, and emulated T3000. Throughput is 121 divided by end-to-end server inference time in seconds and measures inference throughput.
+- **Reasoner:** FP8 vLLM with concurrency 1, 50 input text tokens plus a 30-second, 374 × 374 video sampled at FPS 1 (30 frames), and 100 output tokens. H100, H200, H20, RTX PRO 6000, and embedded hardware use this workload and FLASH_ATTN; output throughput copies the reported output-token throughput.
+- **DGX Station, DGX Spark, and RTX PRO 5000 reasoner:** OSL (output sequence length) 100, FPS 1, and concurrency 1. The attention backend, input text length, video duration, and resolution are unspecified for these results.
+- **Action Policy:** FP8 vLLM-Omni, Policy Robot (DROID), 480p, one GPU, 32-action chunks, 30 denoising steps, concurrency 1, and one warmup request. Throughput is 32 divided by end-to-end server inference time in seconds, reported in actions per second. Each GPU uses its fastest measured backend for this workload: FLASH_ATTN for H100 and H200; CUDNN_ATTN for H20, RTX PRO 6000, DGX Station, DGX Spark, RTX PRO 5000, Thor T5000, and emulated T3000.
+- **Hardware:** Generator measurements use one H100 SXM 80 GB, H200 SXM 141 GB, or H20 SXM 96 GB GPU. The RTX PRO 6000 generator measurement uses one RTX PRO 6000 Blackwell Server Edition GPU. The RTX PRO 5000 generator measurement uses the 72 GB variant.
+- **Embedded generator and action policy:** Thor T5000 and emulated T3000 use FP8 with CUDNN_ATTN and one warmup request. Policy requests use 736 × 544 output in the 480p model tier, with 32 actions plus one conditioning frame. Orin generator and policy measurements use a separate BF16 checkpoint and are omitted from this FP8 table.
 - **Embedded hardware:** Reasoner measurements use one integrated GPU: Jetson AGX Thor T5000 (128 GB), a T5000 devkit configured to emulate T3000 (12 SMs, 8 CPUs, 32 GB), and Jetson AGX Orin (64 GB), all in MAXN mode with clocks locked. Orin runs the FP8 checkpoint with weight-only W8A16 compute; Thor uses FP8 compute. T3000 results are emulated measurements. Hardware variants for other rows require matching benchmark results.
 - Throughputs are calculated before rounding to two decimal places; time to first token is displayed to one decimal place. No estimates are included.
 
@@ -968,7 +969,7 @@ Cosmos3 outputs should not be treated as physically accurate simulation, reliabl
 
 **Acceleration Engine:** [vLLM-Omni](https://github.com/vllm-project/vllm-omni), [vLLM](https://github.com/vllm-project/vllm), [PyTorch](https://pytorch.org/)
 
-**Test Hardware:** B200, H100, H20, RTX PRO 6000, DGX Station, DGX Spark, Jetson Thor, Jetson AGX Orin
+**Test Hardware:** H100, H20, RTX PRO 6000, DGX Station, DGX Spark, Jetson Thor, Jetson AGX Orin
 
 ## Ethical Considerations