Download build/torch-neuron/__init__.py from jburtoft/gte-encoder-neuron-kernels: direct link, hf CLI and curl.
- Browser
- Download file 1.8 kB
-
https://huggingface.co/kernels/jburtoft/gte-encoder-neuron-kernels/resolve/main/build/torch-neuron/__init__.py
- Command line
-
hf download hf://jburtoft/gte-encoder-neuron-kernels/build/torch-neuron/__init__.py
-
curl -L -o __init__.py https://huggingface.co/kernels/jburtoft/gte-encoder-neuron-kernels/resolve/main/build/torch-neuron/__init__.py
1.8 kB
| """NKI kernels for NewBERT (Alibaba-NLP "new-impl") encoders on AWS Trainium2, PyTorch Native. | |
| Covers the gte family built on the NewBERT modeling code (post-LN, rotate-half NTK RoPE, gated GELU MLP), | |
| for example gte-multilingual-reranker-base, gte-multilingual-base, gte-base-en-v1.5, gte-large-en-v1.5. | |
| Entry point: | |
| patch_model(model) -- in-place patch of every layer + the embedding LayerNorm (inference only) | |
| Ops (torch custom ops, namespace gte_enc::): | |
| res_ln, ln -- (residual +) LayerNorm with bn_stats / bn_aggr | |
| gated_mlp_ln -- gated GELU MLP + residual + LayerNorm | |
| attn_block -- QKV + RoPE + masked softmax attention + o_proj + residual + LayerNorm, L <= 512 | |
| Raw NKI kernels (for nki.simulate or other frameworks): res_ln_kernel, gated_mlp_ln_kernel, | |
| gated_mlp_ln_stream_kernel (weights too large for SBUF), attn_block_kernel, plus the host-side packer pack_attn_block. | |
| Set NEURON_LOGICAL_NC_CONFIG before importing; LNC is read once at import. | |
| """ | |
| from .attn_block_kernel import attn_block_kernel | |
| from .attn_block_pack import lane_perm | |
| from .attn_block_pack import pack as pack_attn_block | |
| from .gated_mlp_kernel import gated_mlp_ln_kernel, gated_mlp_ln_stream_kernel, mlp_weights_fit | |
| from .res_ln_kernel import res_ln_kernel | |
| from .torch_ops import ( | |
| ATTN_LENGTHS, | |
| CTE_MIN_LEN, | |
| LNC, | |
| NkiAttnBlock, | |
| NkiLN, | |
| NkiMlpLN, | |
| attn_block, | |
| gated_mlp_ln, | |
| ln, | |
| patch_model, | |
| res_ln, | |
| ) | |
| __version__ = "1.1.0" | |
| __all__ = [ | |
| "patch_model", | |
| "res_ln", "ln", "gated_mlp_ln", "attn_block", | |
| "NkiLN", "NkiMlpLN", "NkiAttnBlock", | |
| "res_ln_kernel", "gated_mlp_ln_kernel", "gated_mlp_ln_stream_kernel", "mlp_weights_fit", "attn_block_kernel", "pack_attn_block", "lane_perm", | |
| "LNC", "ATTN_LENGTHS", "CTE_MIN_LEN", | |
| ] | |