Kernels
superexpai commited on
Commit
a7f6a13
·
verified ·
1 Parent(s): a6e5c43

Add benchmarks/ for the official CLI harness; README harness notes (MK, powered by Claude)

Browse files
Files changed (2) hide show
  1. README.md +18 -2
  2. benchmarks/benchmark.py +102 -0
README.md CHANGED
@@ -141,8 +141,24 @@ fp16/bf16 creation bug on this GPU).
141
 
142
  ## Benchmarks
143
 
144
- Benchmarking script is available for this kernel. Run
145
- `kernels benchmark SuperexponentialAI/relu --version 2`.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
146
 
147
  Full methodology and results: see `OPTIMIZATION.md`, `DTYPE_THROUGHPUT.md`,
148
  and `INVESTIGATION_L2.md` in this repository.
 
141
 
142
  ## Benchmarks
143
 
144
+ Official-harness scripts live in `benchmarks/`: `ReluBenchmark` mirrors
145
+ upstream's fp32 base/large workloads (directly comparable), and
146
+ `ReluDtypeBenchmark` adds the fp16/bf16/int8 workloads upstream lacks.
147
+
148
+ - Local checkout: `kernels benchmark ./relu-build` (all workloads verify
149
+ against `F.relu`).
150
+ - Remote (`kernels benchmark SuperexponentialAI/relu --version 2`)
151
+ requires this repo to be hosted as a `kernel`-type repo; it is
152
+ currently `model`-type (kernel-repo creation is restricted for this
153
+ account), so use the local form until it migrates. The same caveat
154
+ applies to the `get_kernel` snippet above — load via
155
+ `snapshot_download` + `sys.path` in the meantime.
156
+
157
+ The harness times host-side wall clock per iteration; on boost-clocked
158
+ GPUs at L2-resident sizes this understates the kernel's true speedup
159
+ relative to the single-shot reference timing. The nsys
160
+ GPU-kernel-duration numbers above are the trustworthy comparison
161
+ (`INVESTIGATION_L2.md` explains the mechanism).
162
 
163
  Full methodology and results: see `OPTIMIZATION.md`, `DTYPE_THROUGHPUT.md`,
164
  and `INVESTIGATION_L2.md` in this repository.
benchmarks/benchmark.py ADDED
@@ -0,0 +1,102 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn.functional as F
3
+
4
+ from kernels.benchmark import Benchmark
5
+
6
+
7
+ class ReluBenchmark(Benchmark):
8
+ """Same workloads/names as upstream kernels-community/relu (fp32
9
+ base/large) so results are directly comparable across the two repos."""
10
+
11
+ seed: int = 42
12
+
13
+ def setup(self):
14
+ self.x = torch.randn(1024, 1024, device=self.device, dtype=torch.float32)
15
+ self.out = torch.empty_like(self.x)
16
+
17
+ def benchmark_base(self):
18
+ self.out = self.kernel.relu(self.x)
19
+
20
+ def verify_base(self) -> torch.Tensor:
21
+ return F.relu(self.x)
22
+
23
+ def setup_large(self):
24
+ self.x = torch.randn(4096, 4096, device=self.device, dtype=torch.float32)
25
+ self.out = torch.empty_like(self.x)
26
+
27
+ def benchmark_large(self):
28
+ self.out = self.kernel.relu(self.x)
29
+
30
+ def verify_large(self) -> torch.Tensor:
31
+ return F.relu(self.x)
32
+
33
+
34
+ class ReluDtypeBenchmark(Benchmark):
35
+ """Dtypes this build ships beyond upstream's fp32-only: fp16, bf16, int8.
36
+ base = 1024^2, large = 4096^2, matching ReluBenchmark's shapes."""
37
+
38
+ seed: int = 42
39
+
40
+ def _float_setup(self, side, dtype):
41
+ self.x = torch.randn(side, side, device=self.device, dtype=dtype)
42
+ self.out = torch.empty_like(self.x)
43
+
44
+ def setup_fp16_base(self):
45
+ self._float_setup(1024, torch.float16)
46
+
47
+ def setup_fp16_large(self):
48
+ self._float_setup(4096, torch.float16)
49
+
50
+ def setup_bf16_base(self):
51
+ self._float_setup(1024, torch.bfloat16)
52
+
53
+ def setup_bf16_large(self):
54
+ self._float_setup(4096, torch.bfloat16)
55
+
56
+ def _int8_setup(self, side):
57
+ self.x = torch.randint(
58
+ -128, 128, (side, side), device=self.device, dtype=torch.int8
59
+ )
60
+ self.out = torch.empty_like(self.x)
61
+
62
+ def setup_int8_base(self):
63
+ self._int8_setup(1024)
64
+
65
+ def setup_int8_large(self):
66
+ self._int8_setup(4096)
67
+
68
+ def benchmark_fp16_base(self):
69
+ self.out = self.kernel.relu(self.x)
70
+
71
+ def benchmark_fp16_large(self):
72
+ self.out = self.kernel.relu(self.x)
73
+
74
+ def benchmark_bf16_base(self):
75
+ self.out = self.kernel.relu(self.x)
76
+
77
+ def benchmark_bf16_large(self):
78
+ self.out = self.kernel.relu(self.x)
79
+
80
+ def benchmark_int8_base(self):
81
+ self.out = self.kernel.relu(self.x)
82
+
83
+ def benchmark_int8_large(self):
84
+ self.out = self.kernel.relu(self.x)
85
+
86
+ def verify_fp16_base(self) -> torch.Tensor:
87
+ return F.relu(self.x)
88
+
89
+ def verify_fp16_large(self) -> torch.Tensor:
90
+ return F.relu(self.x)
91
+
92
+ def verify_bf16_base(self) -> torch.Tensor:
93
+ return F.relu(self.x)
94
+
95
+ def verify_bf16_large(self) -> torch.Tensor:
96
+ return F.relu(self.x)
97
+
98
+ def verify_int8_base(self) -> torch.Tensor:
99
+ return F.relu(self.x)
100
+
101
+ def verify_int8_large(self) -> torch.Tensor:
102
+ return F.relu(self.x)