Instructions to use Motif-Technologies/optimizer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Kernels
How to use Motif-Technologies/optimizer with Kernels:
# !pip install kernels from kernels import get_kernel # a version (or an explicit revision) is required; see the "Files and versions" tab for the available ones kernel = get_kernel("Motif-Technologies/optimizer", version=1) - Notebooks
- Google Colab
- Kaggle
fix: disable CUDA graphs in Newton-Schulz for cpu_offload compatibility
Browse filesCUDA graphs bake in memory addresses at capture time. When cpu_offload
frees optimizer state storage via resize_(0) between steps, the
CUDA graph replay accesses stale addresses, causing illegal memory access.
Remove triton.cudagraphs option and cudagraph_mark_step_begin() from
Newton-Schulz compile. torch.compile is preserved for kernel optimization.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
torch-ext/optimizer/newton_schulz.py
CHANGED
|
@@ -213,10 +213,8 @@ def zeropower_via_newtonschulz5(G, steps=5):
|
|
| 213 |
if key not in _ns_per_shape:
|
| 214 |
_ns_per_shape[key] = torch.compile(_zeropower_via_newtonschulz5,
|
| 215 |
options={
|
| 216 |
-
"triton.cudagraphs": True,
|
| 217 |
"shape_padding": False
|
| 218 |
})
|
| 219 |
-
torch.compiler.cudagraph_mark_step_begin()
|
| 220 |
return _ns_per_shape[key](G, steps).clone()
|
| 221 |
|
| 222 |
|
|
@@ -229,8 +227,6 @@ def zeropower_via_newtonschulz5_batched(G, steps=5):
|
|
| 229 |
_ns_per_shape[key] = torch.compile(
|
| 230 |
_zeropower_via_newtonschulz5_batched,
|
| 231 |
options={
|
| 232 |
-
"triton.cudagraphs": True,
|
| 233 |
"shape_padding": False
|
| 234 |
})
|
| 235 |
-
torch.compiler.cudagraph_mark_step_begin()
|
| 236 |
return _ns_per_shape[key](G, steps).clone()
|
|
|
|
| 213 |
if key not in _ns_per_shape:
|
| 214 |
_ns_per_shape[key] = torch.compile(_zeropower_via_newtonschulz5,
|
| 215 |
options={
|
|
|
|
| 216 |
"shape_padding": False
|
| 217 |
})
|
|
|
|
| 218 |
return _ns_per_shape[key](G, steps).clone()
|
| 219 |
|
| 220 |
|
|
|
|
| 227 |
_ns_per_shape[key] = torch.compile(
|
| 228 |
_zeropower_via_newtonschulz5_batched,
|
| 229 |
options={
|
|
|
|
| 230 |
"shape_padding": False
|
| 231 |
})
|
|
|
|
| 232 |
return _ns_per_shape[key](G, steps).clone()
|