Instructions to use CatSnowdrop/universal_onnxruntime with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- TensorRT
How to use CatSnowdrop/universal_onnxruntime with TensorRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Download onnxruntime/quantization/operators/base_operator.py from CatSnowdrop/universal_onnxruntime: direct link, hf CLI and curl.
- Browser
- Download file 1.09 kB
-
https://huggingface.co/CatSnowdrop/universal_onnxruntime/resolve/main/onnxruntime/quantization/operators/base_operator.py
- Command line
-
hf download hf://CatSnowdrop/universal_onnxruntime/onnxruntime/quantization/operators/base_operator.py
-
curl -L -o base_operator.py https://huggingface.co/CatSnowdrop/universal_onnxruntime/resolve/main/onnxruntime/quantization/operators/base_operator.py
1.09 kB
| class QuantOperatorBase: | |
| def __init__(self, onnx_quantizer, onnx_node): | |
| self.quantizer = onnx_quantizer | |
| self.node = onnx_node | |
| def should_quantize(self): | |
| if not self.quantizer.should_quantize_node(self.node): | |
| return False | |
| return self.quantizer.is_float_tensor(self.node.input[0]) | |
| def quantize(self): | |
| """ | |
| Given a node which does not support quantization, this method checks whether the input to | |
| this node is quantized and adds a DequantizeLinear node to dequantize this input back to FP32 | |
| parameter node: Current node | |
| parameter new_nodes_list: List of new nodes created before processing current node | |
| return: List of new nodes created | |
| """ | |
| for _, node_input in enumerate(self.node.input): | |
| dequantize_node = self.quantizer._dequantize_value(node_input) | |
| if dequantize_node is not None: | |
| self.quantizer.new_nodes.append(dequantize_node) | |
| # Append the original node | |
| self.quantizer.new_nodes.append(self.node) | |