relu kernel — measured performance to date
Consolidated view of every backend/platform measured so far. Detailed
methodology per platform: OPTIMIZATION.md (4090), H100_BENCHMARK.md
(H100), DTYPE_THROUGHPUT.md (dtype scaling), INVESTIGATION_L2.md
(timing pitfalls), README.md (CPU/XPU). All GPU numbers are nsys
GPU-kernel durations, interleaved 1:1, correctness-gated (bit-identical
to F.relu; NaN → 0 matches upstream).
CUDA — RTX 4090 (Ada, sm_89, tuned here) — corrected in v3
v3 retraction + fix (see L2_BAND_CLIFF.md): the previously published
fp16/bf16 "1.85x/1.81x" and fp32 "1.20x DRAM" vs torch were artifacts of a
fixed-order shared-buffer interleave, and v2 additionally had a real
launch-config cliff (0.34–0.65x vs torch) for working sets in
(32 MiB, 72 MiB L2]. v3 numbers, fair buffer handling:
| comparison | 1024² (L2) | 32 MiB–L2 band | DRAM-bound |
|---|---|---|---|
| vs torch.relu fp32 | ~parity (0.95–1.04x) | 0.99–1.06x (was 0.34–0.65x) | ~parity–1.09x |
| vs torch fp16 / bf16 | 1.17–1.27x | ~parity | ~parity |
| vs torch int8 | 1.35–1.37x | 1.14x | ~parity |
| vs upstream Hub (fp32) | 0.96–1.22x (run-sensitive) | — | parity (both at wall) |
DRAM wall ~944 GB/s (93.7% of 1008 peak), dtype-invariant — every config hits it; upstream can't do fp16/bf16/int8 at all, which remains the main 4090 advantage over the Hub kernel.
CUDA — H100 80GB HBM3 (sm_90, incl. same-day retune) [2026-07-30]
L2-resident vs-torch numbers below are from separate-buffer symmetric
pairs (the shared-buffer 3-way suite's 1.79x-class readings were
position-biased — see L2_BAND_CLIFF.md); DRAM-bound numbers are
position-insensitive and unchanged.
| comparison | 1024² (L2) | 4096² | 8192² |
|---|---|---|---|
| vs upstream Hub kernel (fp32) | 1.2–1.5x (run-sensitive) | 1.44x | 1.24x |
| vs torch fp32 | 1.52x | 1.10x | 1.01x |
| vs torch fp16 | 1.38x | 1.20x | 0.99x |
| vs torch bf16 | 1.38x | 1.16x | 1.01x |
| vs torch int8 | 1.23x | 1.61x | 1.16x |
sm_90 retune: +9.3% (pairwise A/B) at sub-16 MB working sets via a
Hopper-only unroll-2/stream-store/block-256 branch; larger sizes keep the
4090 config — the sweep's claimed 33–52 MB win did not survive in-harness
verification (see H100_BENCHMARK.md Round 2 for the methodology finding
on standalone-sweep and interleave-position bias).
Ours sits at the HBM3 wall (~3.0 TB/s, ~90% of 3,350 peak); upstream Hub never reaches it (2.2–2.4 TB/s) — the DRAM-bound margin grows on H100. Robustness: on flat 1-D tensors upstream launches grid=(1,1,1) and runs 140x slower than ours (its grid heuristic is shape-dependent; ours is a shape-agnostic grid-stride launch).
CPU — i9-13900K (AVX2 + OpenMP + non-temporal stores)
fp32/fp16/bf16/int8 all sit at the memory-bandwidth ceiling (thread/ILP sweep confirmed bandwidth-, not compute-, limited).
XPU — Intel UHD 770 iGPU (SYCL, 16 B/work-item), vs torch 2.12+xpu
| dtype | fp32 | fp16 | bf16 | int8 |
|---|---|---|---|---|
| ours vs torch | 1.01x | 0.96x | 1.16x | 1.06x |
Both bound by the same shared-DDR5 wall; vectorization gave 2.6x over the
naive SYCL version. Consumer iGPU caveats in ../AITESTING.md.
Dtype scaling (memory-bound law, both GPUs)
Element throughput scales 1/sizeof(dtype): bf16/fp16 ≈ 2x fp32, int8/fp8
≈ 4x fp32 intrinsically; up to ~8–15x at L2-boundary working sets
(DTYPE_THROUGHPUT.md). Upstream Hub kernel is fp32-only; this build
ships fp32/fp16/bf16/int8 on all three backends.