Instructions to use CatSnowdrop/universal_onnxruntime with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use CatSnowdrop/universal_onnxruntime with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
File size: 1,080 Bytes
610d9c2 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 | # -------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License.
# --------------------------------------------------------------------------
class Operators:
ATTENTION = "Attention"
LAYERNORM = "LayerNormalization"
MULTI_HEAD_ATTENTION = "MultiHeadAttention"
PACKEDATTENTION = "PackedAttention"
PACKED_MULTI_HEAD_ATTENTION = "PackedMultiHeadAttention"
REMOVEPADDING = "RemovePadding"
RESTOREPADDING = "RestorePadding"
SKIPLAYERNORM = "SkipLayerNormalization"
class AttentionInputIDs:
INPUT = 0
WEIGHTS = 1
BIAS = 2
MASK_INDEX = 3
PAST = 4
ATTENTION_BIAS = 5
PAST_SEQUENCE_LENGTH = 6
class AttentionOutputIDs:
OUTPUT = 0
PRESENT = 1
class MultiHeadAttentionInputIDs:
QUERY = 0
KEY = 1
VALUE = 2
BIAS = 3
KEY_PADDING_MASK = 4
ATTENTION_BIAS = 5
PAST_KEY = 6
PAST_VALUE = 7
class MultiHeadAttentionOutputIDs:
OUTPUT = 0
PRESENT_KEY = 1
PRESENT_VALUE = 2
|