Kernels
wyldecat Claude Opus 4.6 (1M context) commited on
Commit
2dce952
·
1 Parent(s): 05a75f1

fix: disable CUDA graphs in Newton-Schulz for cpu_offload compatibility

Browse files

CUDA graphs bake in memory addresses at capture time. When cpu_offload
frees optimizer state storage via resize_(0) between steps, the
CUDA graph replay accesses stale addresses, causing illegal memory access.

Remove triton.cudagraphs option and cudagraph_mark_step_begin() from
Newton-Schulz compile. torch.compile is preserved for kernel optimization.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

torch-ext/optimizer/newton_schulz.py CHANGED
@@ -213,10 +213,8 @@ def zeropower_via_newtonschulz5(G, steps=5):
213
  if key not in _ns_per_shape:
214
  _ns_per_shape[key] = torch.compile(_zeropower_via_newtonschulz5,
215
  options={
216
- "triton.cudagraphs": True,
217
  "shape_padding": False
218
  })
219
- torch.compiler.cudagraph_mark_step_begin()
220
  return _ns_per_shape[key](G, steps).clone()
221
 
222
 
@@ -229,8 +227,6 @@ def zeropower_via_newtonschulz5_batched(G, steps=5):
229
  _ns_per_shape[key] = torch.compile(
230
  _zeropower_via_newtonschulz5_batched,
231
  options={
232
- "triton.cudagraphs": True,
233
  "shape_padding": False
234
  })
235
- torch.compiler.cudagraph_mark_step_begin()
236
  return _ns_per_shape[key](G, steps).clone()
 
213
  if key not in _ns_per_shape:
214
  _ns_per_shape[key] = torch.compile(_zeropower_via_newtonschulz5,
215
  options={
 
216
  "shape_padding": False
217
  })
 
218
  return _ns_per_shape[key](G, steps).clone()
219
 
220
 
 
227
  _ns_per_shape[key] = torch.compile(
228
  _zeropower_via_newtonschulz5_batched,
229
  options={
 
230
  "shape_padding": False
231
  })
 
232
  return _ns_per_shape[key](G, steps).clone()