granularity results tierb-fixed-moonep-full @ 2026-08-30T00:37:48Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
Browse files
tierb-fixed-moonep-full/hf_watcher.log
CHANGED
|
@@ -31,3 +31,5 @@ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
|
| 31 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 32 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 33 |
[2026-08-30T00:30:27+00:00] backup pass
|
|
|
|
|
|
|
|
|
| 31 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 32 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 33 |
[2026-08-30T00:30:27+00:00] backup pass
|
| 34 |
+
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 35 |
+
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
tierb-fixed-moonep-full/study.log
CHANGED
|
@@ -545,3 +545,9 @@ driver_repo: /workspace/blackwell_run_scripts @ eec9aee
|
|
| 545 |
[00:26:39] TRACE-ONLY DSV2_LITE_L24__echo__matched__seq8192__gbs8__cap1p0755208333333333 (ranks: 0 1 2 3 4 5 6 7)
|
| 546 |
[guard] iter 5/10 (80s)
|
| 547 |
[guard] iter 8/10 (90s)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 545 |
[00:26:39] TRACE-ONLY DSV2_LITE_L24__echo__matched__seq8192__gbs8__cap1p0755208333333333 (ranks: 0 1 2 3 4 5 6 7)
|
| 546 |
[guard] iter 5/10 (80s)
|
| 547 |
[guard] iter 8/10 (90s)
|
| 548 |
+
[guard] no progress for 420s at iter 8/10; treating as done
|
| 549 |
+
[guard] nsys stop (flushing report)
|
| 550 |
+
[guard] tearing down process group (outcome=STALLED)
|
| 551 |
+
[00:35:58] trace verified -> DSV2_LITE_L24__echo__matched__seq8192__gbs8__cap1p0755208333333333.nsys-rep (18M, all 8 GPUs + echo markers)
|
| 552 |
+
[00:35:58] TRACE-ONLY QWEN2_57B__echo__matched__seq4096__gbs8__cap1p08984375 (ranks: 0 1 2 3 4 5 6 7)
|
| 553 |
+
[guard] iter 8/10 (80s)
|