granularity results tiera-rerun-fixed-moonep @ 2026-08-30T08:12:54Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0 (part 2)
Browse files
tiera-rerun-fixed-moonep/echo/tierbtrace/provenance.txt
ADDED
|
@@ -0,0 +1,48 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
==============================================================
|
| 2 |
+
PROVENANCE: echo__tierbtrace__gbs16
|
| 3 |
+
==============================================================
|
| 4 |
+
captured_at : 2026-08-30T08:12:54+00:00
|
| 5 |
+
hostname : sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
|
| 6 |
+
kernel : 6.8.0-1043-nvidia
|
| 7 |
+
cwd : /workspace/blackwell_run_scripts
|
| 8 |
+
command : run_ladder.sh --phase tierbtrace --gbs 16 --out /workspace/tiera-rerun-fixed-moonep/echo/tierbtrace --nsys
|
| 9 |
+
|
| 10 |
+
---- GPU / driver --------------------------------------------
|
| 11 |
+
0, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 12 |
+
1, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 13 |
+
2, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 14 |
+
3, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 15 |
+
4, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 16 |
+
5, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 17 |
+
6, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 18 |
+
7, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 19 |
+
persistence/clocks:
|
| 20 |
+
0, 120 MHz, 1965 MHz, 1000.00 W, 28
|
| 21 |
+
1, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 22 |
+
2, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 23 |
+
3, 120 MHz, 1965 MHz, 1000.00 W, 28
|
| 24 |
+
4, 120 MHz, 1965 MHz, 1000.00 W, 29
|
| 25 |
+
5, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 26 |
+
6, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 27 |
+
7, 120 MHz, 1965 MHz, 1000.00 W, 28
|
| 28 |
+
|
| 29 |
+
NVLink topology:
|
| 30 |
+
[4mGPU0 GPU1 GPU2 GPU3 GPU4 GPU5 GPU6 GPU7 NIC0 NIC1 NIC2 NIC3 NIC4 NIC5 NIC6 NIC7 NIC8 CPU Affinity NUMA Affinity GPU NUMA ID[0m
|
| 31 |
+
GPU0 X NV18 NV18 NV18 NV18 NV18 NV18 NV18 PIX NODE SYS SYS SYS SYS SYS SYS NODE 0-27 0 N/A
|
| 32 |
+
GPU1 NV18 X NV18 NV18 NV18 NV18 NV18 NV18 NODE PIX SYS SYS SYS SYS SYS SYS NODE 0-27 0 N/A
|
| 33 |
+
GPU2 NV18 NV18 X NV18 NV18 NV18 NV18 NV18 SYS SYS PIX NODE SYS SYS SYS SYS SYS 28-55 1 N/A
|
| 34 |
+
GPU3 NV18 NV18 NV18 X NV18 NV18 NV18 NV18 SYS SYS NODE PIX SYS SYS SYS SYS SYS 28-55 1 N/A
|
| 35 |
+
GPU4 NV18 NV18 NV18 NV18 X NV18 NV18 NV18 SYS SYS SYS SYS PIX NODE SYS SYS SYS 56-83 2 N/A
|
| 36 |
+
GPU5 NV18 NV18 NV18 NV18 NV18 X NV18 NV18 SYS SYS SYS SYS NODE PIX SYS SYS SYS 56-83 2 N/A
|
| 37 |
+
GPU6 NV18 NV18 NV18 NV18 NV18 NV18 X NV18 SYS SYS SYS SYS SYS SYS PIX NODE SYS 84-111 3 N/A
|
| 38 |
+
GPU7 NV18 NV18 NV18 NV18 NV18 NV18 NV18 X SYS SYS SYS SYS SYS SYS NODE PIX SYS 84-111 3 N/A
|
| 39 |
+
NIC0 PIX NODE SYS SYS SYS SYS SYS SYS X NODE SYS SYS SYS SYS SYS SYS NODE
|
| 40 |
+
NIC1 NODE PIX SYS SYS SYS SYS SYS SYS NODE X SYS SYS SYS SYS SYS SYS NODE
|
| 41 |
+
NIC2 SYS SYS PIX NODE SYS SYS SYS SYS SYS SYS X NODE SYS SYS SYS SYS SYS
|
| 42 |
+
NIC3 SYS SYS NODE PIX SYS SYS SYS SYS SYS SYS NODE X SYS SYS SYS SYS SYS
|
| 43 |
+
NIC4 SYS SYS SYS SYS PIX NODE SYS SYS SYS SYS SYS SYS X NODE SYS SYS SYS
|
| 44 |
+
unavailable
|
| 45 |
+
|
| 46 |
+
---- toolchain -----------------------------------------------
|
| 47 |
+
nvcc : release 12.9, V12.9.86
|
| 48 |
+
gcc : gcc (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
|
tiera-rerun-fixed-moonep/hf_watcher.log
CHANGED
|
@@ -75,3 +75,6 @@ Watching /workspace/tiera-rerun-fixed-moonep -> HF every 600s (kill to stop)
|
|
| 75 |
[2026-08-30T08:12:42+00:00] backup pass
|
| 76 |
uploaded: /workspace/tiera-rerun-fixed-moonep -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tiera-rerun-fixed-moonep
|
| 77 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
|
|
|
|
|
|
|
|
|
|
|
| 75 |
[2026-08-30T08:12:42+00:00] backup pass
|
| 76 |
uploaded: /workspace/tiera-rerun-fixed-moonep -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tiera-rerun-fixed-moonep
|
| 77 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 78 |
+
Already logged in to Hugging Face as: AbhishekGhosh1998
|
| 79 |
+
Watching /workspace/tiera-rerun-fixed-moonep -> HF every 600s (kill to stop)
|
| 80 |
+
[2026-08-30T08:12:54+00:00] backup pass
|
tiera-rerun-fixed-moonep/hf_watcher.pid
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
|
|
|
|
| 1 |
+
864258
|