AbhishekGhosh1998 commited on
Commit
5b28cac
·
verified ·
1 Parent(s): 08ac786

granularity results tierb-fixed-moonep-full @ 2026-08-30T02:50:07Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0 (part 2)

Browse files
tierb-fixed-moonep-full/steps.tsv CHANGED
@@ -6,3 +6,4 @@ OK echo tierbtrace 8 2026-08-30T00:10:16+00:00 2026-08-30T00:38:30+00:00 /worksp
6
  OK ultraep tierbtrace 8 2026-08-30T00:38:33+00:00 2026-08-30T00:59:28+00:00 /workspace/tierb-fixed-moonep-full/ultraep/tierbtrace
7
  SUPERSEDED moonep tierbtrace 8 2026-08-30T00:59:40+00:00 2026-08-30T01:06:12+00:00 /workspace/tierb-fixed-moonep-full/moonep/tierbtrace
8
  OK moonep tierb 8 2026-08-30T01:30:54+00:00 2026-08-30T01:46:01+00:00 /workspace/tierb-fixed-moonep-full/moonep/tierb
 
 
6
  OK ultraep tierbtrace 8 2026-08-30T00:38:33+00:00 2026-08-30T00:59:28+00:00 /workspace/tierb-fixed-moonep-full/ultraep/tierbtrace
7
  SUPERSEDED moonep tierbtrace 8 2026-08-30T00:59:40+00:00 2026-08-30T01:06:12+00:00 /workspace/tierb-fixed-moonep-full/moonep/tierbtrace
8
  OK moonep tierb 8 2026-08-30T01:30:54+00:00 2026-08-30T01:46:01+00:00 /workspace/tierb-fixed-moonep-full/moonep/tierb
9
+ OK moonep tierbtrace 8 2026-08-30T02:38:13+00:00 2026-08-30T02:49:59+00:00 /workspace/tierb-fixed-moonep-full/moonep/tierbtrace
tierb-fixed-moonep-full/study.log CHANGED
@@ -1114,3 +1114,86 @@ driver_repo: /workspace/blackwell_run_scripts @ e8bd2bd
1114
  [guard] nsys stop (flushing report)
1115
  [02:47:36] trace verified -> DSV2_LITE_L24__moonep__matched__seq8192__gbs8__capnone__async.nsys-rep (13M, all 8 GPUs + moonep markers)
1116
  [02:47:37] TRACE-ONLY QWEN2_57B__moonep__matched__seq4096__gbs8__capnone__async (ranks: 0 1 2 3 4 5 6 7)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1114
  [guard] nsys stop (flushing report)
1115
  [02:47:36] trace verified -> DSV2_LITE_L24__moonep__matched__seq8192__gbs8__capnone__async.nsys-rep (13M, all 8 GPUs + moonep markers)
1116
  [02:47:37] TRACE-ONLY QWEN2_57B__moonep__matched__seq4096__gbs8__capnone__async (ranks: 0 1 2 3 4 5 6 7)
1117
+ [guard] iter 8/10 (60s)
1118
+ [guard] iter 10/10 (110s)
1119
+ [guard] nsys stop (flushing report)
1120
+ [02:49:59] trace verified -> QWEN2_57B__moonep__matched__seq4096__gbs8__capnone__async.nsys-rep (5.6M, all 8 GPUs + moonep markers)
1121
+ --------------------------------------------------------------------------
1122
+
1123
+ ==============================================================
1124
+ SWEEP COMPLETE ok=4 failed=0
1125
+ ==============================================================
1126
+ config backend policy slots seq gbs cap recompute rep status iter_ms_mean iter_ms_min peak_mib run_dir
1127
+
1128
+ Results: /workspace/tierb-fixed-moonep-full/moonep/tierbtrace/tierbtrace_gbs8/results.tsv
1129
+ Roll up: python3 /workspace/blackwell_run_scripts/granularity/tools/collect.py /workspace/tierb-fixed-moonep-full/moonep/tierbtrace/tierbtrace_gbs8
1130
+ ==============================================================
1131
+
1132
+ Campaign results: /workspace/tierb-fixed-moonep-full/moonep/tierbtrace
1133
+ Roll up: python3 /workspace/blackwell_run_scripts/granularity/tools/collect.py /workspace/tierb-fixed-moonep-full/moonep/tierbtrace --compare
1134
+ Already logged in to Hugging Face as: AbhishekGhosh1998
1135
+ uploaded: /workspace/tierb-fixed-moonep-full/moonep/tierbtrace -> MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tierb-fixed-moonep-full/moonep/tierbtrace
1136
+ https://huggingface.co/MoE-CUDA-Graph-Analysis/tierb-b200-gbs8/tree/main
1137
+
1138
+ === ROLL-UP
1139
+ 40 runs ok=36 oom=0 -> /workspace/tierb-fixed-moonep-full/collected_all.tsv
1140
+
1141
+ (excluded 12 nsys-traced run(s) from the timing comparison)
1142
+
1143
+ phase config seq gbs policy rc cg asy backend iter_ms vs_best
1144
+ -------------------------------------------------------------------------------------
1145
+ tierb DSV2_LITE_L24 8192 8 matched none 1 1 moonep 593.43 1.00x
1146
+
1147
+ tierb QWEN2_57B 4096 8 matched none 1 1 moonep 180.12 1.00x
1148
+
1149
+ tierb QWEN3_235B 8192 8 matched none 1 1 moonep 317.80 1.00x
1150
+
1151
+ tierb QWEN3_30B 4096 8 matched none 1 1 moonep 355.16 1.00x
1152
+
1153
+ tierbmem DSV2_LITE_L24 8192 8 matched none 1 1 moonep 622.80 1.00x
1154
+
1155
+ tierbmem QWEN2_57B 4096 8 matched none 1 1 moonep 196.60 1.00x
1156
+
1157
+ tierbmem QWEN3_235B 8192 8 matched none 1 1 moonep 341.70 1.00x
1158
+
1159
+ tierbmem QWEN3_30B 4096 8 matched none 1 1 moonep 435.30 1.00x
1160
+
1161
+ tierbprobeDSV2_LITE_L24 8192 8 matched none 0 1 moonep 673.94 1.00x
1162
+
1163
+ tierbprobeQWEN2_57B 4096 8 matched none 0 1 moonep 238.91 1.00x
1164
+
1165
+ tierbprobeQWEN3_235B 8192 8 matched none 0 1 moonep 384.76 1.00x
1166
+
1167
+ tierbprobeQWEN3_30B 4096 8 matched none 0 1 moonep 824.23 1.00x
1168
+
1169
+ 8 probe runs -> /workspace/tierb-fixed-moonep-full/probe_summary_all.tsv
1170
+ run arm cap rawExpI preRankI postRankI drop% reqCap
1171
+ --------------------------------------------------------------------------------------------------------------------
1172
+ moonep/tierbprobe/tierbprobe_gbs8/DSV2_LITE_L24__moo natural none 7.5443* 2.1316 1.1058
1173
+ moonep/tierbprobe/tierbprobe_gbs8/DSV2_LITE_L24__moo forced none 1.0286 1.0061 1.0026
1174
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN2_57B__moonep_ natural none 7.4132* 2.3019 1.0979
1175
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN2_57B__moonep_ forced none 1.0348 1.0073 1.0033
1176
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN3_235B__moonep natural none 15.9227* 2.5851 1.0712
1177
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN3_235B__moonep forced none 1.0386 1.0053 1.0025
1178
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN3_30B__moonep_ natural none 15.7078* 2.4672 1.0635
1179
+ moonep/tierbprobe/tierbprobe_gbs8/QWEN3_30B__moonep_ forced none 1.0565 1.0077 1.0036
1180
+
1181
+ * At least one globally aggregated snapshot reached the E/K expert-Imax ceiling.
1182
+ Inspect raw_expert_imax_mean and hottest_expert_token_fraction_mean before generalising.
1183
+
1184
+ ECHO note: echo_logged_pre_rank_metric_valid=0 and echo_logged_load_reduction_valid=0 are intentional.
1185
+ The branch's built-in fields mix source-rank and destination-rank axes; this tool reconstructs both globally.
1186
+ 8 memory-dump runs -> /workspace/tierb-fixed-moonep-full/memory_summary_all.tsv
1187
+ model backend cg allocation capacity status peak GiB dumps
1188
+ ----------------------------------------------------------------------------------
1189
+ DSV2_LITE_L24 moonep 1 none none OK 130.2 48/48
1190
+ DSV2_LITE_L24 moonep 0 none none OK 122.2 40/40
1191
+ QWEN2_57B moonep 1 none none OK 94.5 48/48
1192
+ QWEN2_57B moonep 0 none none OK 93.6 40/40
1193
+ QWEN3_235B moonep 1 none none OK 117.5 48/48
1194
+ QWEN3_235B moonep 0 none none OK 117.3 40/40
1195
+ QWEN3_30B moonep 1 none none OK 114.0 48/48
1196
+ QWEN3_30B moonep 0 none none OK 113.7 40/40
1197
+
1198
+ ECHO/UltraEP note: graph-on uses static capacity + device-initiated GEMM while graph-off uses dynamic allocation + host-derived sizes. Their difference is not labelled as pure graph cost.
1199
+ Already logged in to Hugging Face as: AbhishekGhosh1998