File size: 2,983 Bytes
dfb775d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | # Day 2 Build-in-Public post — May 5 2026
**Theme:** The 60-second AOT autotune in action on MI300X.
## X thread
```
1/ Day 2: the autotune layer that makes mindXtrain win the Application of
Technology axis.
60 seconds on MI300X, three probes:
- attention: Composable Kernel vs Triton SDPA
- gemm: hipBLASLt heuristic check
- rccl: 1-GPU vs 8-GPU xGMI
Output: a static AOT plan. #AMDDevHackathon
```
```
2/ Today's measurement:
CK forward, (8, 4096, 32, 128): <X> ms
Triton forward, same shape: <Y> ms
→ CK wins by <Z>%, plan picks ck
Hand-tuned ASM kernels via @AIatAMD's AITER beat Triton at this size.
This decision is locked into the run, not re-decided every step.
```
```
3/ Why AOT-only matters for production training:
JIT autotune (Triton on cold start, torch.compile max-autotune, MIOpen
find-mode) makes the same workload non-deterministic across runs. AMD's
AOTriton + offline-tuned hipBLASLt cache + AOT plan = reproducible.
Hash-equal across machines. cypherpunk2048 standard.
```
```
4/ Code is small. autotune/ is one orchestrator + three probes + a
Pydantic AutotunePlan. The training layer reads plan.json, sets env
vars + flags, then accelerate launches Axolotl.
GitHub: <repo URL>
Tomorrow: the actual LoRA fine-tune of amd/Instella-3B on MI300X.
@Alibaba_Qwen
```
## LinkedIn post
```
Day 2 of the AMD × lablab.ai Developer Hackathon — the autotune layer is
live on MI300X.
A 60-second probe runs before each training job:
1. Attention: torch's scaled_dot_product_attention timed across four
representative shapes on both Composable Kernel (default) and AOTriton.
Pick the faster. Today: <CK ms> vs <Triton ms> on Qwen3-8B's shape.
2. GEMM: per the AMD ROCm 7.2.1 release notes, hipBLASLt 0.10's default
heuristic for gfx942 BF16/FP16 GEMMs is within 5% of hand-tuned for
the LoRA-rank-16-to-64 / hidden-2048-to-8192 shapes mindXtrain hits.
Plan locks it in. Heuristic enumeration is post-hackathon work.
3. RCCL: 1-GPU is no-op; 8-GPU sets NCCL_MIN_NCHANNELS=112 and
GPU_MAX_HW_QUEUES=1 in the plan's env block. The 2/4-GPU paths
raise — MI300X xGMI bandwidth between subsets of 2/4 GPUs is
asymmetric and silently bottlenecks FSDP shards.
Output is a static AutotunePlan JSON, BLAKE3-hashed into the custmodel
manifest. No JIT autotune in production. Same plan = same kernels =
reproducible runs.
This is the cypherpunk2048 reproducibility standard applied to the
ROCm reality. The training layer reads the plan and sets the env vars
and Axolotl flags before subprocess-launching accelerate. Nothing
re-tunes during the loop.
Tomorrow: full LoRA fine-tune of amd/Instella-3B on MI300X using the
plan from today.
#AMDDevHackathon
```
## Asset checklist
- [ ] `autotune_plan.json` from a real MI300X run
- [ ] Side-by-side timing table: CK vs Triton across 4 shapes
- [ ] `rocminfo` output showing gfx942 + 192 GB
- [ ] Recording of the 60-second probe streaming output (used in the demo video)
|