granularity results tiera-rerun-fixed-moonep @ 2026-08-30T10:33:04Z from sp-aiinfra-interactive-aghosh-7jtjl-8xb200-moe-cg-0
Browse files
tiera-rerun-fixed-moonep/STATUS.txt
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
study_root: /workspace/tiera-rerun-fixed-moonep
|
| 2 |
gbs: 16
|
| 3 |
-
updated: 2026-08-30T10:26:
|
| 4 |
-
current:
|
| 5 |
|
| 6 |
completed steps:
|
| 7 |
echo__a1__gbs16
|
|
|
|
| 1 |
study_root: /workspace/tiera-rerun-fixed-moonep
|
| 2 |
gbs: 16
|
| 3 |
+
updated: 2026-08-30T10:26:19+00:00
|
| 4 |
+
current: done
|
| 5 |
|
| 6 |
completed steps:
|
| 7 |
echo__a1__gbs16
|
tiera-rerun-fixed-moonep/TIERA_RERUN.md
ADDED
|
@@ -0,0 +1,339 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Tier A rerun β fixed MoonEP, measured capacity
|
| 2 |
+
|
| 3 |
+
Driver: `./granularity/run_tiera_rerun.sh`
|
| 4 |
+
Study root: `/workspace/tiera-rerun-fixed-moonep`
|
| 5 |
+
HF: `MoE-CUDA-Graph-Analysis/tierb-b200-gbs8` under `tiera-rerun-fixed-moonep/`
|
| 6 |
+
Hardware: 8ΓB200, **GBS 16**, seq 4096, EP 8.
|
| 7 |
+
|
| 8 |
+
This campaign re-measures the whole Tier A ladder. It supersedes the Tier A
|
| 9 |
+
tables in `RESULTS.md`.
|
| 10 |
+
|
| 11 |
+
## 1. Why the published Tier A ladder had to be re-run
|
| 12 |
+
|
| 13 |
+
Two independent defects, either of which alone invalidates the MoonEP column.
|
| 14 |
+
|
| 15 |
+
**The MoonEP numbers measured the wrong computation.** The published ladder used
|
| 16 |
+
the pre-fix MoonEP integration, where DDP applied its `1/dp_size` expert-gradient
|
| 17 |
+
scaling during backward, *before* MoonEP's `reduce_grad` delivered the duplicated
|
| 18 |
+
experts' contribution β so that contribution ended up 8Γ too large. The stack
|
| 19 |
+
still trained, still reported a plausible loss, and still produced timings. It
|
| 20 |
+
separately held two copies of every expert weight. Both its throughput and its
|
| 21 |
+
memory are therefore unusable. Root cause and fix:
|
| 22 |
+
`MOONEP_GRAD_BUG_AND_FIX.md`.
|
| 23 |
+
|
| 24 |
+
**Capacity was hardcoded, never measured.** Every Tier A cell ran at
|
| 25 |
+
`CAPACITY:-1.25`, a value nothing verified. `RESULTS.md` Β§3 of the caveats
|
| 26 |
+
records that *"8 runs mention capacity saturation (UltraEP probe cells dropping
|
| 27 |
+
tokens)"*. A run that drops assignments does **less expert work**, so it looks
|
| 28 |
+
faster for the wrong reason. Per the repository rule, a run that dropped tokens
|
| 29 |
+
is not a timing result.
|
| 30 |
+
|
| 31 |
+
## 2. Capacity policy
|
| 32 |
+
|
| 33 |
+
The first choice was **one capacity shared by every cell**, so that nothing along
|
| 34 |
+
the ladder differs except granularity. **That turned out to be impossible**, and
|
| 35 |
+
the measurement that settles it is recorded below.
|
| 36 |
+
|
| 37 |
+
MoonEP takes no capacity factor and is excluded from the choice; it is tagged
|
| 38 |
+
`capnone` throughout.
|
| 39 |
+
|
| 40 |
+
### What the probe measured
|
| 41 |
+
|
| 42 |
+
Validated zero-drop capacity per rung and backend, GBS 16, seq 4096:
|
| 43 |
+
|
| 44 |
+
| rung | ECHO | UltraEP | rung max |
|
| 45 |
+
| --- | ---: | ---: | ---: |
|
| 46 |
+
| **G1** | **1.859375** | **2.40625** | **2.40625** |
|
| 47 |
+
| G2 | 1.08203125 | 1.0625 | 1.08203125 |
|
| 48 |
+
| G3 | 1.072265625 | 1.0625 | 1.072265625 |
|
| 49 |
+
| G4 | 1.068359375 | 1.095703125 | 1.095703125 |
|
| 50 |
+
|
| 51 |
+
**G1 is a factor-of-two outlier and every other rung sits barely above 1.0.**
|
| 52 |
+
G1 has only 8 experts at top-2 across 8 ranks, so there is almost nothing to
|
| 53 |
+
average imbalance over; G4's 128 experts average it away almost completely.
|
| 54 |
+
|
| 55 |
+
That the cause is imbalance rather than shape is directly confirmed by the
|
| 56 |
+
probe's force-balanced control: with `--moe-router-force-load-balancing`, G1
|
| 57 |
+
needs only **1.01** on both backends. The capacity is being spent entirely on
|
| 58 |
+
routing skew.
|
| 59 |
+
|
| 60 |
+
### Why a single global capacity is impossible
|
| 61 |
+
|
| 62 |
+
The global max is 2.40625, driven entirely by G1. A direct memory gate on the
|
| 63 |
+
two extreme rungs, CUDA graphs on, settles whether it can be used everywhere:
|
| 64 |
+
|
| 65 |
+
| gate cell | capacity | peak | verdict |
|
| 66 |
+
| --- | ---: | ---: | --- |
|
| 67 |
+
| G4/echo | 2.40625 | 182,627 MiB | **OOM** (ceiling 182,630) |
|
| 68 |
+
| G1/echo | 2.40625 | 161,863 MiB | OK |
|
| 69 |
+
|
| 70 |
+
So no single capacity satisfies both "no drops anywhere" and "fits everywhere".
|
| 71 |
+
The campaign uses **one capacity per rung**, recorded in `tiera_capacity.env`
|
| 72 |
+
with `TIERA_CAPACITY_GLOBAL_FEASIBLE=0` so the driver does not spend a full
|
| 73 |
+
timing stage rediscovering the OOM.
|
| 74 |
+
|
| 75 |
+
Within a rung every backend still shares one capacity, so the **backend
|
| 76 |
+
comparison stays valid**. Only the cross-rung trend must be read alongside the
|
| 77 |
+
capacity column. This is a deliberate, narrow relaxation of the "never mix
|
| 78 |
+
capacity within one comparison" rule, justified because each rung is its own
|
| 79 |
+
backend comparison.
|
| 80 |
+
|
| 81 |
+
### The consequence for the published ladder
|
| 82 |
+
|
| 83 |
+
The published Tier A ran **every** rung at a hardcoded 1.25. Against the
|
| 84 |
+
measured requirements:
|
| 85 |
+
|
| 86 |
+
| rung | required | ran at | verdict |
|
| 87 |
+
| --- | ---: | ---: | --- |
|
| 88 |
+
| **G1** | 1.86 / 2.41 | 1.25 | **both backends dropped tokens** |
|
| 89 |
+
| G2 | 1.08 / 1.06 | 1.25 | fine |
|
| 90 |
+
| G3 | 1.07 / 1.06 | 1.25 | fine |
|
| 91 |
+
| G4 | 1.07 / 1.10 | 1.25 | fine |
|
| 92 |
+
|
| 93 |
+
The defect is **isolated to G1** β and G1 is precisely the rung the published
|
| 94 |
+
headline rests on ("ECHO wins only at the coarsest rung"). A run that drops
|
| 95 |
+
assignments does less expert work and therefore looks faster, so the published
|
| 96 |
+
G1 row cannot support that claim. G2βG4 were never capacity-limited and their
|
| 97 |
+
relative ordering is unaffected.
|
| 98 |
+
|
| 99 |
+
### Why the probe needed a second pass at G1
|
| 100 |
+
|
| 101 |
+
The probe's own ladder could not resolve G1. ECHO's ladder tops out at 2.00 and
|
| 102 |
+
G1 needed 1.906 + 5% margin, just above it. UltraEP starts at 1.25 and was still
|
| 103 |
+
dropping at 1.8125 β and a dropping run **under-reports** demand, because
|
| 104 |
+
`actual_valid_recv` is clipped at the buffer, which is why its estimate climbed
|
| 105 |
+
from 1.72 to 2.31 as the capacity rose. G1 was therefore re-probed with
|
| 106 |
+
`TIERB_ECHO_PROBE_CAPACITIES_DESC="3.00 2.75 2.50 2.25"` and
|
| 107 |
+
`TIERB_ULTRAEP_INITIAL_CAP=3.00`, where both backends measured cleanly with zero
|
| 108 |
+
drops.
|
| 109 |
+
|
| 110 |
+
### Why the grid is coarsest-wins
|
| 111 |
+
|
| 112 |
+
A capacity is executable only when `int(C Β· seq Β· mbs Β· topk)` is a multiple of
|
| 113 |
+
128. `topk` varies 16Γ along the ladder, so the rungs do **not** share a grid:
|
| 114 |
+
|
| 115 |
+
| rung | topk | step |
|
| 116 |
+
| --- | ---: | --- |
|
| 117 |
+
| G1 | 2 | 1/128 β coarsest, therefore binding on any shared capacity |
|
| 118 |
+
| G2 | 8 | 1/512 |
|
| 119 |
+
| G3 | 16 | 1/1024 |
|
| 120 |
+
| G4 | 32 | 1/2048 |
|
| 121 |
+
|
| 122 |
+
`tools/tiera_global_capacity.py` enforces this and re-verifies alignment on every
|
| 123 |
+
rung before emitting a value.
|
| 124 |
+
|
| 125 |
+
### Why GBS stays 16
|
| 126 |
+
|
| 127 |
+
ECHO G1 OOMs at GBS 32. The published ladder is GBS 16 and this rerun matches it
|
| 128 |
+
so the two remain comparable.
|
| 129 |
+
|
| 130 |
+
## 3. Branch matrix
|
| 131 |
+
|
| 132 |
+
Each phase demands a different Megatron checkout. The orchestrator switches
|
| 133 |
+
automatically; `run_ladder.sh` asserts the result.
|
| 134 |
+
|
| 135 |
+
| phase | ECHO | UltraEP | MoonEP |
|
| 136 |
+
| --- | --- | --- | --- |
|
| 137 |
+
| probe (stage 7) | `token-drop-analysis` | `ultraep-token-drop-analysis` | *(excluded β no capacity factor)* |
|
| 138 |
+
| timing (stage 3) | `bracing-regions-through-kernels` | `ultraep-bracing-regions-through-kernels` | `moonep_backend_async_grad_reduce` |
|
| 139 |
+
| memory (stage 8) | `better-memory-dump-support` | `ultraep-better-memory-dump-support` | `moonep_backend_async_grad_reduce_better_memory_dump_support` |
|
| 140 |
+
| trace (stage 9) | `bracing-regions-through-kernels` | `ultraep-bracing-regions-through-kernels` | `moonep_backend_async_grad_reduce` |
|
| 141 |
+
|
| 142 |
+
**There is no separate MoonEP "async + bracing" branch, and none is needed.**
|
| 143 |
+
`moonep_backend_bracing_regions_through_kernels` is an *ancestor* of
|
| 144 |
+
`moonep_backend_async_grad_reduce`, so the async branch already carries every
|
| 145 |
+
bracing region β plus two the bracing branch lacks:
|
| 146 |
+
|
| 147 |
+
```
|
| 148 |
+
_(moonep_weight_prefetch, 1100)
|
| 149 |
+
_(moonep_grad_reduce, 1110)
|
| 150 |
+
```
|
| 151 |
+
|
| 152 |
+
Those brace the weight-movement collectives that live in `fused_moonep.py`
|
| 153 |
+
rather than in the layer or dispatcher. `moonep_grad_reduce` is the marker the
|
| 154 |
+
overlap analyzer keys on; without it the async deferral cannot be measured.
|
| 155 |
+
|
| 156 |
+
**The four pre-async MoonEP branches do not carry the gradient fix**
|
| 157 |
+
(`moonep_backend`, `..._bracing_regions_through_kernels`,
|
| 158 |
+
`..._token_routing_analysis`, `..._better_memory_dump_support`). They must never
|
| 159 |
+
be selected. `run_ladder.sh` now refuses to run MoonEP unless
|
| 160 |
+
`external_contributors` is present in `param_and_grad_buffer.py`;
|
| 161 |
+
`STRICT_MOONEP_GRAD_FIX=0` overrides it, deliberately explicitly.
|
| 162 |
+
|
| 163 |
+
## 4. MoonEP now appears in a1 and ponly
|
| 164 |
+
|
| 165 |
+
`RESULTS.md` states that MoonEP *"does not appear in `a1`/`ponly` by
|
| 166 |
+
construction, not by omission: it forces B = E/R and cannot run the one-slot
|
| 167 |
+
policy."*
|
| 168 |
+
|
| 169 |
+
**That is true for G2βG4 only.** MoonEP forces `B = E/R`, which conflicts with
|
| 170 |
+
the one-slot policy only when `E/R > 1`. G1 and PONLY are both `E=8`, so at
|
| 171 |
+
`R=8` the forced value already **is** one slot:
|
| 172 |
+
|
| 173 |
+
| config | E | E/R at R=8 | one-slot policy |
|
| 174 |
+
| --- | ---: | ---: | --- |
|
| 175 |
+
| G1 | 8 | 1 | β
runnable |
|
| 176 |
+
| PONLY | 8 | 1 | β
runnable |
|
| 177 |
+
| G2 | 32 | 4 | β |
|
| 178 |
+
| G3 | 64 | 8 | β |
|
| 179 |
+
| G4 | 128 | 16 | β |
|
| 180 |
+
|
| 181 |
+
`phase_a1`/`phase_ponly` carried a blanket backend-level skip that threw those
|
| 182 |
+
cells away. `sweep_cell` already makes the correct per-config decision and warns
|
| 183 |
+
when it skips, so the phases now defer to it. This campaign therefore produces
|
| 184 |
+
**MoonEP PONLY numbers at both sequence lengths, and MoonEP `a1` at G1** β data
|
| 185 |
+
the published ladder never collected.
|
| 186 |
+
|
| 187 |
+
Note that at G1 the `one` and `matched` policies are the same configuration
|
| 188 |
+
(B=1) for *every* backend, so `a1@G1` and `a2@G1` are the same measurement taken
|
| 189 |
+
twice. That is a consistency check, not two independent results.
|
| 190 |
+
|
| 191 |
+
## 5. A tagging correction
|
| 192 |
+
|
| 193 |
+
`train_moe.sh` already downgrades `CAPACITY_MODE=static` to `none` for MoonEP,
|
| 194 |
+
so passing a capacity through never changed the run. But the Tier A phases still
|
| 195 |
+
put that capacity into the run-directory tag and the results table, which made
|
| 196 |
+
MoonEP rows look as though they had been measured at a capacity they ignore β
|
| 197 |
+
and would have re-tagged byte-identical MoonEP cells as new measurements when
|
| 198 |
+
the global capacity changed. Tier B already tags them `capnone`; Tier A now
|
| 199 |
+
agrees.
|
| 200 |
+
|
| 201 |
+
## 6. What this campaign collects
|
| 202 |
+
|
| 203 |
+
| stage | phase | content |
|
| 204 |
+
| --- | --- | --- |
|
| 205 |
+
| 7 | `tierbprobe` | zero-drop capacity per rung Γ backend, CG off, instrumented branches |
|
| 206 |
+
| 3 | `a1`, `ponly`, `a2` | throughput, CG on, 3 repeats, all three backends |
|
| 207 |
+
| 8 | `tierbmem` | allocator snapshots at the chosen capacity, CG on and off |
|
| 208 |
+
| 9 | `tierbtrace` | nsys traces with bracing markers, all 8 GPUs verified |
|
| 209 |
+
|
| 210 |
+
Tier A previously recorded `peak_mib` only and never captured allocator
|
| 211 |
+
snapshots, so stage 8 is new for this tier.
|
| 212 |
+
|
| 213 |
+
## 7. Reading the results
|
| 214 |
+
|
| 215 |
+
**Post-fix MoonEP is a new implementation baseline, not an optimization delta.**
|
| 216 |
+
The pre-fix stack computed something different, so a before/after speedup across
|
| 217 |
+
the fix is meaningless. Do not subtract the old Tier A MoonEP column from the
|
| 218 |
+
new one.
|
| 219 |
+
|
| 220 |
+
**Do not compare across capacity modes.** If `CAPACITY_DECISION.txt` records
|
| 221 |
+
`capacity_mode=perrung`, the cross-rung trend must be read together with the
|
| 222 |
+
capacity column.
|
| 223 |
+
|
| 224 |
+
**Verify zero drops before quoting any timing.** That is the whole reason this
|
| 225 |
+
campaign exists.
|
| 226 |
+
|
| 227 |
+
## 8. Results
|
| 228 |
+
|
| 229 |
+
All numbers: GBS 16, seq 4096 (except PONLY 8192), mean of 3 unprofiled repeats,
|
| 230 |
+
zero token drops, CUDA graphs on. 81 timing rows, none failed.
|
| 231 |
+
|
| 232 |
+
### a2 β capacity-matched, B = E/R
|
| 233 |
+
|
| 234 |
+
| rung | capacity | ECHO | UltraEP | MoonEP |
|
| 235 |
+
| --- | ---: | ---: | ---: | ---: |
|
| 236 |
+
| G1 | 2.40625 | **466.8** | 892.7 | 480.6 |
|
| 237 |
+
| G2 | 1.08203125 | **415.4** | 492.3 | 432.2 |
|
| 238 |
+
| G3 | 1.072265625 | **437.2** | 459.7 | 559.9 |
|
| 239 |
+
| G4 | 1.095703125 | 486.7 | **482.9** | 782.8 |
|
| 240 |
+
|
| 241 |
+
### Peak memory (MiB, a2)
|
| 242 |
+
|
| 243 |
+
| rung | ECHO | UltraEP | MoonEP |
|
| 244 |
+
| --- | ---: | ---: | ---: |
|
| 245 |
+
| G1 | 161,723 | 129,475 | **96,849** |
|
| 246 |
+
| G2 | 145,691 | 112,315 | **97,665** |
|
| 247 |
+
| G3 | 147,933 | 113,951 | **99,409** |
|
| 248 |
+
| G4 | 168,485 | 132,437 | **106,031** |
|
| 249 |
+
|
| 250 |
+
### a1 β one-slot policy
|
| 251 |
+
|
| 252 |
+
| rung | ECHO | UltraEP | MoonEP |
|
| 253 |
+
| --- | ---: | ---: | ---: |
|
| 254 |
+
| G1 | **467.3** | 890.4 | 479.1 |
|
| 255 |
+
| G2 | **394.2** | 466.6 | β |
|
| 256 |
+
| G3 | **412.7** | 448.0 | β |
|
| 257 |
+
| G4 | **460.7** | 467.6 | β |
|
| 258 |
+
|
| 259 |
+
MoonEP's G1 entry is new. Its dashes at G2βG4 are structural (`E/R > 1`), not
|
| 260 |
+
missing data. As a consistency check, MoonEP G1 in `a1` (479.1) and in `a2`
|
| 261 |
+
(480.6) are the same configuration measured twice and agree to 0.3%.
|
| 262 |
+
|
| 263 |
+
### ponly β payload-only diagnostic
|
| 264 |
+
|
| 265 |
+
| seq | ECHO | UltraEP | MoonEP |
|
| 266 |
+
| --- | ---: | ---: | ---: |
|
| 267 |
+
| 4096 | 175.0 | 176.8 | **165.2** |
|
| 268 |
+
| 8192 | 316.5 | **292.7** | 300.6 |
|
| 269 |
+
|
| 270 |
+
Also new: the published ladder has no MoonEP `ponly` row at all.
|
| 271 |
+
|
| 272 |
+
PONLY was not probed separately; it inherits G1's capacity because it has the
|
| 273 |
+
same E=8 and K=2, and capacity demand here is driven by routing imbalance over E
|
| 274 |
+
and K. Inheriting a *higher* capacity cannot cause drops, so this is safe, but it
|
| 275 |
+
is an inheritance rather than a measurement.
|
| 276 |
+
|
| 277 |
+
### MoonEP async grad-reduce overlap
|
| 278 |
+
|
| 279 |
+
From the stage-9 traces, 8 devices:
|
| 280 |
+
|
| 281 |
+
| rung | overlap | reduce, ms/iter |
|
| 282 |
+
| --- | ---: | ---: |
|
| 283 |
+
| G1 | 53.5% | 58.44 |
|
| 284 |
+
| G2 | 97.2% | 27.94 |
|
| 285 |
+
| G3 | 95.7% | 16.46 |
|
| 286 |
+
| G4 | 94.1% | 10.79 |
|
| 287 |
+
|
| 288 |
+
Overlap effectiveness falls sharply as experts get larger. At G1 each expert is
|
| 289 |
+
`H'=14336` wide and the gradient reduce is 58 ms per iteration β too much to
|
| 290 |
+
hide behind one layer's backward, so nearly half of it stays exposed. By G4 the
|
| 291 |
+
expert is 16Γ narrower, the reduce costs 10.8 ms, and 94% of it disappears.
|
| 292 |
+
This is a plausible part of why MoonEP's G1 disadvantage is smallest and its G4
|
| 293 |
+
disadvantage largest, though the two move in opposite directions and the
|
| 294 |
+
exposed-reduce term alone does not explain the G4 result.
|
| 295 |
+
|
| 296 |
+
## 9. What changed versus the published ladder
|
| 297 |
+
|
| 298 |
+
**Two published conclusions do not survive.**
|
| 299 |
+
|
| 300 |
+
*"ECHO wins only at the coarsest rung; UltraEP wins everywhere finer."* Not
|
| 301 |
+
reproduced. ECHO now wins G1, G2 and G3; UltraEP wins only G4, by 0.8%
|
| 302 |
+
(482.9 vs 486.7). The published claim rested on a G1 row where both backends
|
| 303 |
+
were dropping tokens.
|
| 304 |
+
|
| 305 |
+
*"UltraEP is 30β35 GB lighter than ECHO at every rung."* Still true of UltraEP
|
| 306 |
+
versus ECHO, but it is no longer the headline: **MoonEP is now the lightest
|
| 307 |
+
backend at every rung**, by 30β60 GB against UltraEP and 50β65 GB against ECHO.
|
| 308 |
+
Its published memory (154β166 GB) came from a stack that held two copies of
|
| 309 |
+
every expert weight; the DDP-owned symmetric-parameter implementation brings it
|
| 310 |
+
to 97β106 GB.
|
| 311 |
+
|
| 312 |
+
**MoonEP's throughput ranking changed too**, from slowest everywhere to second
|
| 313 |
+
at G1/G2. It still degrades fastest along the ladder.
|
| 314 |
+
|
| 315 |
+
| rung | MoonEP published | MoonEP now |
|
| 316 |
+
| --- | ---: | ---: |
|
| 317 |
+
| G1 | 696.2 | 480.6 |
|
| 318 |
+
| G2 | 656.1 | 432.2 |
|
| 319 |
+
| G3 | 769.7 | 559.9 |
|
| 320 |
+
| G4 | 984.5 | 782.8 |
|
| 321 |
+
|
| 322 |
+
These are **not** a speedup measurement. The published stack computed something
|
| 323 |
+
different, so the difference across the fix is not an optimization delta. Treat
|
| 324 |
+
the new column as a first correct baseline.
|
| 325 |
+
|
| 326 |
+
## 10. Audit
|
| 327 |
+
|
| 328 |
+
| check | result |
|
| 329 |
+
| --- | --- |
|
| 330 |
+
| probe selection for all 8 (rung, backend) pairs | yes, after G1 re-probe |
|
| 331 |
+
| capacities executable on every rung | verified by the tool |
|
| 332 |
+
| `CAPACITY_DECISION.txt` records mode and reason | `capacity_mode=perrung` |
|
| 333 |
+
| timing rows, 3 repeats, zero failures | 81 OK, 0 failed |
|
| 334 |
+
| zero token drops | yes, capacities chosen for it |
|
| 335 |
+
| `a1` skipped exactly G2/G3/G4 for MoonEP | yes, with reasons logged |
|
| 336 |
+
| MoonEP PONLY at seq 4096 and 8192 | yes |
|
| 337 |
+
| memory dumps, all rungs Γ backends | 12/12 markers |
|
| 338 |
+
| nsys reports | 12/12, markers on all 8 GPUs |
|
| 339 |
+
| MoonEP async overlap nonzero | 53β97% |
|