AbhishekGhosh1998 commited on
Commit
afca2c8
·
verified ·
1 Parent(s): 846693e

granularity results tiera-rerun-fixed-moonep @ 2026-08-30T08:03:48Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0

Browse files
tiera-rerun-fixed-moonep/CAPACITY_DECISION.txt ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ capacity_mode=perrung
2
+ derived_global=2.40625
3
+ per_rung_G1=2.40625
4
+ per_rung_G2=1.08203125
5
+ per_rung_G3=1.072265625
6
+ per_rung_G4=1.095703125
7
+ reason=the single global capacity OOMed at least one cell, so each rung
8
+ reason=uses its own capacity. Backends remain comparable WITHIN a rung;
9
+ reason=the cross-rung trend must be read with the capacity column.
tiera-rerun-fixed-moonep/STATUS.txt CHANGED
@@ -1,7 +1,7 @@
1
  study_root: /workspace/tiera-rerun-fixed-moonep
2
  gbs: 16
3
- updated: 2026-08-30T08:03:36+00:00
4
- current: idle
5
 
6
  completed steps:
7
  echo__a1__gbs16
 
1
  study_root: /workspace/tiera-rerun-fixed-moonep
2
  gbs: 16
3
+ updated: 2026-08-30T08:03:47+00:00
4
+ current: echo__tierbmem__gbs16
5
 
6
  completed steps:
7
  echo__a1__gbs16
tiera-rerun-fixed-moonep/echo/tierbmem/provenance.txt ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ==============================================================
2
+ PROVENANCE: echo__tierbmem__gbs16
3
+ ==============================================================
4
+ captured_at : 2026-08-30T08:03:47+00:00
5
+ hostname : sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
6
+ kernel : 6.8.0-1043-nvidia
7
+ cwd : /workspace/blackwell_run_scripts
8
+ command : run_ladder.sh --phase tierbmem --gbs 16 --out /workspace/tiera-rerun-fixed-moonep/echo/tierbmem
9
+
10
+ ---- GPU / driver --------------------------------------------
11
+ 0, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
12
+ 1, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
13
+ 2, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
14
+ 3, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
15
+ 4, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
16
+ 5, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
17
+ 6, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
18
+ 7, NVIDIA B200, 10.0, 183359 MiB, 580.95.05
19
+ persistence/clocks:
20
+ 0, 120 MHz, 1965 MHz, 1000.00 W, 29
21
+ 1, 120 MHz, 1965 MHz, 1000.00 W, 34
22
+ 2, 120 MHz, 1965 MHz, 1000.00 W, 34
23
+ 3, 120 MHz, 1965 MHz, 1000.00 W, 29
24
+ 4, 120 MHz, 1965 MHz, 1000.00 W, 30
25
+ 5, 120 MHz, 1965 MHz, 1000.00 W, 34
26
+ 6, 120 MHz, 1965 MHz, 1000.00 W, 34
27
+ 7, 120 MHz, 1965 MHz, 1000.00 W, 30
28
+
29
+ NVLink topology:
tiera-rerun-fixed-moonep/hf_watcher.log CHANGED
@@ -55,3 +55,6 @@ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
55
  [2026-08-30T08:00:57+00:00] backup pass
56
  uploaded: /workspace/tiera-rerun-fixed-moonep -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tiera-rerun-fixed-moonep
57
  https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
 
 
 
 
55
  [2026-08-30T08:00:57+00:00] backup pass
56
  uploaded: /workspace/tiera-rerun-fixed-moonep -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tiera-rerun-fixed-moonep
57
  https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
58
+ Already logged in to Hugging Face as: AbhishekGhosh1998
59
+ Watching /workspace/tiera-rerun-fixed-moonep -> HF every 600s (kill to stop)
60
+ [2026-08-30T08:03:48+00:00] backup pass
tiera-rerun-fixed-moonep/hf_watcher.pid CHANGED
@@ -1 +1 @@
1
- 588250
 
1
+ 841364
tiera-rerun-fixed-moonep/study.log CHANGED
@@ -1101,3 +1101,36 @@ ECHO note: echo_logged_pre_rank_metric_valid=0 and echo_logged_load_reduction_va
1101
  The branch's built-in fields mix source-rank and destination-rank axes; this tool reconstructs both globally.
1102
  no memory-dump runs found under /workspace/tiera-rerun-fixed-moonep
1103
  Already logged in to Hugging Face as: AbhishekGhosh1998
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1101
  The branch's built-in fields mix source-rank and destination-rank axes; this tool reconstructs both globally.
1102
  no memory-dump runs found under /workspace/tiera-rerun-fixed-moonep
1103
  Already logged in to Hugging Face as: AbhishekGhosh1998
1104
+ uploaded: /workspace/tiera-rerun-fixed-moonep -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tiera-rerun-fixed-moonep
1105
+ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
1106
+
1107
+ === STUDY COMPLETE
1108
+ results : /workspace/tiera-rerun-fixed-moonep
1109
+ status : /workspace/tiera-rerun-fixed-moonep/STATUS.txt
1110
+ steps : /workspace/tiera-rerun-fixed-moonep/steps.tsv
1111
+ hub : https://huggingface.co/datasets/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8
1112
+
1113
+ status campaign phase gbs started ended result_dir
1114
+ FAIL echo tierbprobe 16 2026-08-30T04:52:38+00:00 2026-08-30T05:06:38+00:00 /workspace/tiera-rerun-fixed-moonep/echo/tierbprobe
1115
+ FAIL ultraep tierbprobe 16 2026-08-30T05:06:44+00:00 2026-08-30T05:20:40+00:00 /workspace/tiera-rerun-fixed-moonep/ultraep/tierbprobe
1116
+ OK echo a1 16 2026-08-30T06:20:31+00:00 2026-08-30T06:36:28+00:00 /workspace/tiera-rerun-fixed-moonep/echo/a1
1117
+ OK echo ponly 16 2026-08-30T06:36:31+00:00 2026-08-30T06:44:11+00:00 /workspace/tiera-rerun-fixed-moonep/echo/ponly
1118
+ OK echo a2 16 2026-08-30T06:44:13+00:00 2026-08-30T07:00:25+00:00 /workspace/tiera-rerun-fixed-moonep/echo/a2
1119
+ OK ultraep a1 16 2026-08-30T07:00:27+00:00 2026-08-30T07:15:17+00:00 /workspace/tiera-rerun-fixed-moonep/ultraep/a1
1120
+ OK ultraep ponly 16 2026-08-30T07:15:19+00:00 2026-08-30T07:22:01+00:00 /workspace/tiera-rerun-fixed-moonep/ultraep/ponly
1121
+ OK ultraep a2 16 2026-08-30T07:22:03+00:00 2026-08-30T07:37:07+00:00 /workspace/tiera-rerun-fixed-moonep/ultraep/a2
1122
+ OK moonep a1 16 2026-08-30T07:37:09+00:00 2026-08-30T07:41:05+00:00 /workspace/tiera-rerun-fixed-moonep/moonep/a1
1123
+ OK moonep ponly 16 2026-08-30T07:41:07+00:00 2026-08-30T07:48:20+00:00 /workspace/tiera-rerun-fixed-moonep/moonep/ponly
1124
+ OK moonep a2 16 2026-08-30T07:48:22+00:00 2026-08-30T08:03:34+00:00 /workspace/tiera-rerun-fixed-moonep/moonep/a2
1125
+ study_root: /workspace/tiera-rerun-fixed-moonep
1126
+ gbs: 16
1127
+ stages: 8
1128
+ nsys: 0
1129
+ hf_repo: MoE-CUDA-Graph-Analysis/tierb-b200-gbs8
1130
+ hf_remote_root: tiera-rerun-fixed-moonep
1131
+ started: 2026-08-30T08:03:47+00:00
1132
+ driver_repo: /workspace/blackwell_run_scripts @ f69d9ab
1133
+
1134
+ === STAGE 8 -- Tier B allocator dumps (static CG capacities + dynamic no-CG)
1135
+
1136
+ === STEP echo__tierbmem__gbs16