Jim Burtoft commited on
Commit ·
eacd70d
1
Parent(s): 6709b3f
v1.6: kernel-repo distribution fix
Browse files- Bump metadata.json version 1 -> 2 so kernels' get_kernel(version=2) resolves
the current tree.
- README: document the PyTorch-Native runtime requirement for get_kernel (kernels
backend detection needs torch.neuron / a non-CUDA torch build).
Also (out-of-band): syncing this tree to the kernel-type repo endpoint
(huggingface.co/kernels/jburtoft/kda-neuron-kernels), which get_kernel reads and
which was frozen at an old commit -- the model-repo pushes (v1.1..v1.5) never
reached it.
- README.md +13 -2
- build/torch-neuron/metadata.json +1 -1
README.md
CHANGED
|
@@ -27,10 +27,21 @@ Trainium machine:
|
|
| 27 |
|
| 28 |
```python
|
| 29 |
from kernels import get_kernel
|
| 30 |
-
k = get_kernel("jburtoft/kda-neuron-kernels", version=
|
| 31 |
-
# k.
|
| 32 |
```
|
| 33 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 34 |
## What this package provides
|
| 35 |
|
| 36 |
### Inference (forward)
|
|
|
|
| 27 |
|
| 28 |
```python
|
| 29 |
from kernels import get_kernel
|
| 30 |
+
k = get_kernel("jburtoft/kda-neuron-kernels", version=2, trust_remote_code=True)
|
| 31 |
+
# k.kda_chunk_step_exact(...), k.kda_chunk_step_exact_bwd(...), k.kda_recurrent_fwd(...), etc.
|
| 32 |
```
|
| 33 |
|
| 34 |
+
**Runtime requirement (important):** load this from a **PyTorch Native
|
| 35 |
+
(torch-neuronx)** environment where `torch` is a CPU/Neuron build and `torch.neuron`
|
| 36 |
+
is registered (e.g. the DLAMI venvs `aws_neuronx_venv_pytorch_2_9_nxd_inference`
|
| 37 |
+
or a PyTorch-Native Beta venv). The `kernels` library selects the build variant
|
| 38 |
+
from the active torch backend; in a **CUDA** torch build (some DLAMI base venvs ship
|
| 39 |
+
`torch ...+cuXXX`) it will detect backend `cuda` and refuse the `neuron` variant
|
| 40 |
+
with *"backend (neuron) does not match system backend (cuda)"*. If you hit that,
|
| 41 |
+
switch to a Neuron/PyTorch-Native venv (verify with
|
| 42 |
+
`python -c "from kernels.backends import _backend; print(_backend().name)"` → should
|
| 43 |
+
print `neuron`).
|
| 44 |
+
|
| 45 |
## What this package provides
|
| 46 |
|
| 47 |
### Inference (forward)
|
build/torch-neuron/metadata.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
| 1 |
{
|
| 2 |
"name": "kda-neuron-kernels",
|
| 3 |
"id": "kda_neuron_kernels",
|
| 4 |
-
"version":
|
| 5 |
"license": "Apache-2.0",
|
| 6 |
"backend": {"type": "neuron", "archs": []},
|
| 7 |
"python-depends": []
|
|
|
|
| 1 |
{
|
| 2 |
"name": "kda-neuron-kernels",
|
| 3 |
"id": "kda_neuron_kernels",
|
| 4 |
+
"version": 2,
|
| 5 |
"license": "Apache-2.0",
|
| 6 |
"backend": {"type": "neuron", "archs": []},
|
| 7 |
"python-depends": []
|