granularity results tierb-fixed-moonep-full @ 2026-08-30T00:58:13Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
Browse files
tierb-fixed-moonep-full/hf_watcher.log
CHANGED
|
@@ -37,3 +37,5 @@ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
|
| 37 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 38 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 39 |
[2026-08-30T00:50:37+00:00] backup pass
|
|
|
|
|
|
|
|
|
| 37 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 38 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 39 |
[2026-08-30T00:50:37+00:00] backup pass
|
| 40 |
+
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 41 |
+
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
tierb-fixed-moonep-full/study.log
CHANGED
|
@@ -589,3 +589,14 @@ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
|
| 589 |
[00:45:24] TRACE-ONLY QWEN3_30B__ultraep__matched__seq4096__gbs8__cap1p11328125 (ranks: 0 1 2 3 4 5 6 7)
|
| 590 |
[guard] iter 6/10 (70s)
|
| 591 |
[guard] iter 8/10 (80s)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 589 |
[00:45:24] TRACE-ONLY QWEN3_30B__ultraep__matched__seq4096__gbs8__cap1p11328125 (ranks: 0 1 2 3 4 5 6 7)
|
| 590 |
[guard] iter 6/10 (70s)
|
| 591 |
[guard] iter 8/10 (80s)
|
| 592 |
+
[guard] no progress for 420s at iter 8/10; treating as done
|
| 593 |
+
[guard] nsys stop (flushing report)
|
| 594 |
+
[guard] tearing down process group (outcome=STALLED)
|
| 595 |
+
[00:54:34] trace verified -> QWEN3_30B__ultraep__matched__seq4096__gbs8__cap1p11328125.nsys-rep (25M, all 8 GPUs + ultraep markers)
|
| 596 |
+
[00:54:34] TRACE-ONLY DSV2_LITE_L24__ultraep__matched__seq8192__gbs8__cap1p0755208333333333 (ranks: 0 1 2 3 4 5 6 7)
|
| 597 |
+
[guard] iter 6/10 (70s)
|
| 598 |
+
[guard] iter 8/10 (80s)
|
| 599 |
+
[guard] iter 10/10 (120s)
|
| 600 |
+
[guard] nsys stop (flushing report)
|
| 601 |
+
[00:57:06] trace verified -> DSV2_LITE_L24__ultraep__matched__seq8192__gbs8__cap1p0755208333333333.nsys-rep (14M, all 8 GPUs + ultraep markers)
|
| 602 |
+
[00:57:06] TRACE-ONLY QWEN2_57B__ultraep__matched__seq4096__gbs8__cap1p08984375 (ranks: 0 1 2 3 4 5 6 7)
|