granularity results tierb-fixed-moonep-full @ 2026-08-29T23:33:50Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
Browse files
tierb-fixed-moonep-full/PATCH_RERUN_MANIFEST.txt
CHANGED
|
@@ -1,5 +1,5 @@
|
|
| 1 |
campaign=tierb-patch-rerun
|
| 2 |
-
created_at=2026-08-29T23:
|
| 3 |
study_root=/workspace/tierb-fixed-moonep-full
|
| 4 |
parts=probe timing memory trace
|
| 5 |
echo_ultra_models=QWEN2_57B
|
|
@@ -12,3 +12,4 @@ moonep_megatron_branch=moonep_backend_async_grad_reduce
|
|
| 12 |
qwen_shared_expert_overlap=1
|
| 13 |
moonep_async_grad_reduce=1
|
| 14 |
skip_moonep=0
|
|
|
|
|
|
| 1 |
campaign=tierb-patch-rerun
|
| 2 |
+
created_at=2026-08-29T23:33:50+00:00
|
| 3 |
study_root=/workspace/tierb-fixed-moonep-full
|
| 4 |
parts=probe timing memory trace
|
| 5 |
echo_ultra_models=QWEN2_57B
|
|
|
|
| 12 |
qwen_shared_expert_overlap=1
|
| 13 |
moonep_async_grad_reduce=1
|
| 14 |
skip_moonep=0
|
| 15 |
+
skip_echo_ultra=1
|
tierb-fixed-moonep-full/STATUS.txt
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
study_root: /workspace/tierb-fixed-moonep-full
|
| 2 |
gbs: 8
|
| 3 |
-
updated: 2026-08-29T23:
|
| 4 |
-
current:
|
| 5 |
|
| 6 |
completed steps:
|
| 7 |
|
|
|
|
| 1 |
study_root: /workspace/tierb-fixed-moonep-full
|
| 2 |
gbs: 8
|
| 3 |
+
updated: 2026-08-29T23:33:50+00:00
|
| 4 |
+
current: moonep__tierbprobe__gbs8
|
| 5 |
|
| 6 |
completed steps:
|
| 7 |
|
tierb-fixed-moonep-full/hf_watcher.log
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
Already logged in to Hugging Face as: AbhishekGhosh1998
|
| 2 |
Watching /workspace/tierb-fixed-moonep-full -> HF every 600s (kill to stop)
|
| 3 |
-
[2026-08-29T23:
|
|
|
|
| 1 |
Already logged in to Hugging Face as: AbhishekGhosh1998
|
| 2 |
Watching /workspace/tierb-fixed-moonep-full -> HF every 600s (kill to stop)
|
| 3 |
+
[2026-08-29T23:33:50+00:00] backup pass
|
tierb-fixed-moonep-full/hf_watcher.pid
CHANGED
|
@@ -1 +1 @@
|
|
| 1 |
-
|
|
|
|
| 1 |
+
212923
|
tierb-fixed-moonep-full/moonep/tierbprobe/provenance.txt
ADDED
|
@@ -0,0 +1,29 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
==============================================================
|
| 2 |
+
PROVENANCE: moonep__tierbprobe__gbs8
|
| 3 |
+
==============================================================
|
| 4 |
+
captured_at : 2026-08-29T23:33:50+00:00
|
| 5 |
+
hostname : sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
|
| 6 |
+
kernel : 6.8.0-1043-nvidia
|
| 7 |
+
cwd : /workspace/blackwell_run_scripts
|
| 8 |
+
command : run_ladder.sh --phase tierbprobe --gbs 8 --out /workspace/tierb-fixed-moonep-full/moonep/tierbprobe --nsys
|
| 9 |
+
|
| 10 |
+
---- GPU / driver --------------------------------------------
|
| 11 |
+
0, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 12 |
+
1, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 13 |
+
2, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 14 |
+
3, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 15 |
+
4, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 16 |
+
5, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 17 |
+
6, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 18 |
+
7, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
|
| 19 |
+
persistence/clocks:
|
| 20 |
+
0, 120 MHz, 1965 MHz, 1000.00 W, 27
|
| 21 |
+
1, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 22 |
+
2, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 23 |
+
3, 120 MHz, 1965 MHz, 1000.00 W, 27
|
| 24 |
+
4, 120 MHz, 1965 MHz, 1000.00 W, 29
|
| 25 |
+
5, 120 MHz, 1965 MHz, 1000.00 W, 31
|
| 26 |
+
6, 120 MHz, 1965 MHz, 1000.00 W, 32
|
| 27 |
+
7, 120 MHz, 1965 MHz, 1000.00 W, 28
|
| 28 |
+
|
| 29 |
+
NVLink topology:
|
tierb-fixed-moonep-full/study.log
CHANGED
|
@@ -4,9 +4,9 @@ stages: 7
|
|
| 4 |
nsys: 1
|
| 5 |
hf_repo: MoE-CUDA-Graph-Analysis/tierb-b200-gbs8
|
| 6 |
hf_remote_root: tierb-fixed-moonep-full
|
| 7 |
-
started: 2026-08-29T23:
|
| 8 |
-
driver_repo: /workspace/blackwell_run_scripts @
|
| 9 |
|
| 10 |
=== STAGE 7 -- Tier B probe and zero-drop capacity selection
|
| 11 |
|
| 12 |
-
=== STEP
|
|
|
|
| 4 |
nsys: 1
|
| 5 |
hf_repo: MoE-CUDA-Graph-Analysis/tierb-b200-gbs8
|
| 6 |
hf_remote_root: tierb-fixed-moonep-full
|
| 7 |
+
started: 2026-08-29T23:33:50+00:00
|
| 8 |
+
driver_repo: /workspace/blackwell_run_scripts @ 5054c89
|
| 9 |
|
| 10 |
=== STAGE 7 -- Tier B probe and zero-drop capacity selection
|
| 11 |
|
| 12 |
+
=== STEP moonep__tierbprobe__gbs8
|