PERFORMANCE.md

# relu kernel — measured performance to date

Consolidated view of every backend/platform measured so far. Detailed
methodology per platform: `OPTIMIZATION.md` (4090), `H100_BENCHMARK.md`
(H100), `DTYPE_THROUGHPUT.md` (dtype scaling), `INVESTIGATION_L2.md`
(timing pitfalls), `README.md` (CPU/XPU). All GPU numbers are nsys
GPU-kernel durations, interleaved 1:1, correctness-gated (bit-identical
to `F.relu`; NaN → 0 matches upstream).

## CUDA — RTX 4090 (Ada, sm_89, tuned here) — corrected in v3

**v3 retraction + fix** (see `L2_BAND_CLIFF.md`): the previously published
fp16/bf16 "1.85x/1.81x" and fp32 "1.20x DRAM" vs torch were artifacts of a
fixed-order shared-buffer interleave, and v2 additionally had a real
launch-config cliff (0.34–0.65x vs torch) for working sets in
(32 MiB, 72 MiB L2]. v3 numbers, fair buffer handling:

| comparison | 1024² (L2) | 32 MiB–L2 band | DRAM-bound |
|---|:---:|:---:|:---:|
| vs torch.relu fp32 | ~parity (0.95–1.04x) | **0.99–1.06x** (was 0.34–0.65x) | ~parity–1.09x |
| vs torch fp16 / bf16 | **1.17–1.27x** | ~parity | ~parity |
| vs torch int8 | **1.35–1.37x** | **1.14x** | ~parity |
| vs upstream Hub (fp32) | 0.96–1.22x (run-sensitive) | — | parity (both at wall) |

DRAM wall ~944 GB/s (93.7% of 1008 peak), dtype-invariant — every config
hits it; upstream can't do fp16/bf16/int8 at all, which remains the main
4090 advantage over the Hub kernel.

## CUDA — H100 80GB HBM3 (sm_90, incl. same-day retune) [2026-07-30]

L2-resident vs-torch numbers below are from separate-buffer symmetric
pairs (the shared-buffer 3-way suite's 1.79x-class readings were
position-biased — see `L2_BAND_CLIFF.md`); DRAM-bound numbers are
position-insensitive and unchanged.

| comparison | 1024² (L2) | 4096² | 8192² |
|---|:---:|:---:|:---:|
| vs upstream Hub kernel (fp32) | 1.2–1.5x (run-sensitive) | **1.44x** | **1.24x** |
| vs torch fp32 | **1.52x** | **1.10x** | 1.01x |
| vs torch fp16 | **1.38x** | **1.20x** | 0.99x |
| vs torch bf16 | **1.38x** | **1.16x** | 1.01x |
| vs torch int8 | **1.23x** | **1.61x** | **1.16x** |

sm_90 retune: +9.3% (pairwise A/B) at sub-16 MB working sets via a
Hopper-only unroll-2/stream-store/block-256 branch; larger sizes keep the
4090 config — the sweep's claimed 33–52 MB win did not survive in-harness
verification (see `H100_BENCHMARK.md` Round 2 for the methodology finding
on standalone-sweep and interleave-position bias).

Ours sits at the HBM3 wall (~3.0 TB/s, ~90% of 3,350 peak); upstream Hub
never reaches it (2.2–2.4 TB/s) — the DRAM-bound margin *grows* on H100.
Robustness: on flat 1-D tensors upstream launches grid=(1,1,1) and runs
**140x slower** than ours (its grid heuristic is shape-dependent; ours is
a shape-agnostic grid-stride launch).

## CPU — i9-13900K (AVX2 + OpenMP + non-temporal stores)

fp32/fp16/bf16/int8 all sit at the memory-bandwidth ceiling (thread/ILP
sweep confirmed bandwidth-, not compute-, limited).

## XPU — Intel UHD 770 iGPU (SYCL, 16 B/work-item), vs torch 2.12+xpu

| dtype | fp32 | fp16 | bf16 | int8 |
|---|:---:|:---:|:---:|:---:|
| ours vs torch | 1.01x | 0.96x | **1.16x** | **1.06x** |

Both bound by the same shared-DDR5 wall; vectorization gave 2.6x over the
naive SYCL version. Consumer iGPU caveats in `../AITESTING.md`.

## Dtype scaling (memory-bound law, both GPUs)

Element throughput scales 1/sizeof(dtype): bf16/fp16 ≈ 2x fp32, int8/fp8
≈ 4x fp32 intrinsically; up to ~8–15x at L2-boundary working sets
(`DTYPE_THROUGHPUT.md`). Upstream Hub kernel is fp32-only; this build
ships fp32/fp16/bf16/int8 on all three backends.