granularity results tierb-fixed-moonep-full @ 2026-08-30T00:27:37Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
Browse files
tierb-fixed-moonep-full/hf_watcher.log
CHANGED
|
@@ -28,3 +28,5 @@ Watching /workspace/tierb-fixed-moonep-full -> HF every 600s (kill to stop)
|
|
| 28 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 29 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 30 |
[2026-08-30T00:20:20+00:00] backup pass
|
|
|
|
|
|
|
|
|
| 28 |
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 29 |
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
| 30 |
[2026-08-30T00:20:20+00:00] backup pass
|
| 31 |
+
uploaded: /workspace/tierb-fixed-moonep-full -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full
|
| 32 |
+
https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
|
tierb-fixed-moonep-full/study.log
CHANGED
|
@@ -538,3 +538,8 @@ driver_repo: /workspace/blackwell_run_scripts @ eec9aee
|
|
| 538 |
[00:17:19] TRACE-ONLY QWEN3_30B__echo__matched__seq4096__gbs8__cap1p11328125 (ranks: 0 1 2 3 4 5 6 7)
|
| 539 |
[guard] iter 3/10 (80s)
|
| 540 |
[guard] iter 8/10 (90s)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 538 |
[00:17:19] TRACE-ONLY QWEN3_30B__echo__matched__seq4096__gbs8__cap1p11328125 (ranks: 0 1 2 3 4 5 6 7)
|
| 539 |
[guard] iter 3/10 (80s)
|
| 540 |
[guard] iter 8/10 (90s)
|
| 541 |
+
[guard] no progress for 420s at iter 8/10; treating as done
|
| 542 |
+
[guard] nsys stop (flushing report)
|
| 543 |
+
[guard] tearing down process group (outcome=STALLED)
|
| 544 |
+
[00:26:39] trace verified -> QWEN3_30B__echo__matched__seq4096__gbs8__cap1p11328125.nsys-rep (33M, all 8 GPUs + echo markers)
|
| 545 |
+
[00:26:39] TRACE-ONLY DSV2_LITE_L24__echo__matched__seq8192__gbs8__cap1p0755208333333333 (ranks: 0 1 2 3 4 5 6 7)
|