--- license: mit tags: - onnxruntime - onnx - inference - cuda - tensorrt - openvino - directml - windows --- # Universal ONNX Runtime (UORT) 1.30.1 — Windows x64 package folder A self-contained build of **ONNX Runtime 1.30.1** for **Windows x64 / CPython 3.12** with five execution providers compiled in and every native dependency bundled next to the Python extension: **TensorRT, CUDA, OpenVINO, DirectML and CPU**. This repository publishes the package as a **plain directory tree** — `onnxruntime/` at the repo root, exactly as it has to appear in `Lib\site-packages`. There is no archive to unpack: what you download is already the installable folder. The point of the package is that you install it by copying one folder — no CUDA Toolkit, no cuDNN download, no TensorRT SDK, no `pip install onnxruntime-gpu`. Everything the providers need is in `capi/`, and the package registers that directory with the Windows loader on import. | | | |---|---| | Contents | `onnxruntime/` — **379 files** in **31 subdirectories** | | Stored / transferred size | **5 585 491 685 bytes** (5.20 GiB) | | Python modules | 317 `.py` files | | Native binaries | 52 `.dll` + 1 `.pyd`, all in `capi/` | | `__pycache__` | none (removed before publishing) | | Package version | `uort-python` 1.30.1 | | Platform | Windows 10/11 x64 only | | Python ABI | CPython 3.12 x64 (`python312.dll`) | | Licence | MIT (ONNX Runtime) + third-party terms, see [Licences](#licences) | ## Integrity There is no single archive hash, because there is no archive. Every file is stored as an LFS / Xet object keyed by the **SHA-256 of its content**, and both `hf download` and `git lfs` verify that hash while writing the file — a corrupted transfer is rejected, not silently accepted. Spot-check values if you want to confirm a few key binaries yourself: | File | Bytes | SHA-256 | |---|---|---| | `capi\onnxruntime.dll` | 24 535 040 | `a4599b0168c8b3e485e50c4493559280778f7c533d66bbb3ea372073f316b67f` | | `capi\onnxruntime_pybind11_state.pyd` | 25 851 904 | `e54c4900fc60c521dc597dc0b0c87835a18ea6ee2efd38af78c70ac6f10abcbb` | | `capi\onnxruntime_providers_cuda.dll` | 689 375 232 | `2c744e0d1683518fa88e229ca006304aad6a29b81b9f0229596969c89ba80c12` | | `capi\nvinfer_builder_resource_10.dll` | 1 971 797 024 | `2b8496c57a8434534e8a3c56e732207f1c74bc37ce7300d8f0d3542cdff02e12` | ```powershell (Get-FileHash .\onnxruntime\capi\onnxruntime.dll -Algorithm SHA256).Hash.ToLower() ``` ## Download Grab the whole folder — 379 separate HTTP requests by hand is not a good idea. ```powershell # Hugging Face CLI: chunked, parallel, resumable hf download CatSnowdrop/universal_onnxruntime --local-dir .\uort # result: .\uort\onnxruntime\__init__.py, .\uort\onnxruntime\capi\... # or git (needs git-lfs; Xet accelerates it if hf_xet is installed) git clone https://huggingface.co/CatSnowdrop/universal_onnxruntime uort ``` ```python from huggingface_hub import snapshot_download path = snapshot_download( repo_id="CatSnowdrop/universal_onnxruntime", local_dir="uort", max_workers=8, ) print(path) # -> uort\onnxruntime ``` Single files can be fetched directly; the endpoint supports HTTP `Range`, so a large `.dll` resumes instead of restarting: ```powershell curl.exe -L -C - -o onnxruntime_providers_cuda.dll ` "https://huggingface.co/CatSnowdrop/universal_onnxruntime/resolve/main/onnxruntime/capi/onnxruntime_providers_cuda.dll" ``` `resolve/main` redirects to a commit-pinned CDN path, so re-uploading never invalidates a download that is already running. > **Expect ~5.20 GiB of traffic.** Unlike a zip, DLLs are stored uncompressed, so the folder is > ~1.5× larger on the wire than the equivalent archive (3.48 GiB). In exchange there is no > unpack step and no second copy on disk: peak usage is 5.20 GiB, not 8.8 GiB. ## What's inside ### Execution providers `onnxruntime.get_available_providers()` reports, in this order: ```python ['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'OpenVINOExecutionProvider', 'DmlExecutionProvider', 'CPUExecutionProvider'] ``` | EP | Bundled runtime | Notes | |---|---|---| | `TensorrtExecutionProvider` | TensorRT 10.9, `nvonnxparser`, `nvinfer_dispatch` | engines are built at session creation; requires compute capability ≥ 7.5 | | `CUDAExecutionProvider` | CUDA 12.9 runtime, cuBLAS, cuFFT, cuRAND, cuDNN 9 | driver is the only system prerequisite | | `OpenVINOExecutionProvider` | OpenVINO 2026.2.1 + Intel CPU/GPU/NPU plugins, TBB | CPU, GPU and NPU devices | | `DmlExecutionProvider` | `DirectML.dll` 17.67 MB | any DirectX 12 GPU (NVIDIA/AMD/Intel) | | `CPUExecutionProvider` | MLAS kernels inside `onnxruntime.dll` | always available | GPU kernels are compiled for `sm_61;75;80;86;89;90;100;120`. Note that the **TensorRT** EP, unlike CUDA, does not support 6.1: TensorRT 10 dropped Pascal, so a P10x/GTX 10x0 card can use CUDA or DirectML but not TensorRT. ### Why the package is 5.2 GiB Almost all of the volume is the CUDA / TensorRT / OpenVINO runtimes; the Python side is ~4 MiB. | Group | Main files | Size | |---|---|---| | TensorRT | `nvinfer_builder_resource_10.dll` (1880 MB), `nvinfer_10.dll` (421 MB), `nvinfer_plugin_10.dll` (49 MB) | ~2.30 GiB | | CUDA | `onnxruntime_providers_cuda.dll` (657 MB), `cublasLt64_12.dll` (638 MB), `cudnn_engines_precompiled64_9.dll` (490 MB), `cufft64_11.dll` (274 MB), `cudnn_adv64_9.dll` (269 MB), `cudnn_ops64_9.dll` (121 MB) | ~2.64 GiB | | OpenVINO | `openvino.dll`, `openvino_intel_{cpu,gpu,npu}_plugin.dll`, `openvino_intel_npu_compiler.dll`, frontends, TBB | ~200 MiB | | Core | `onnxruntime.dll` (23 MB), `onnxruntime_pybind11_state.pyd` (25 MB), `onnxruntime_providers_shared.dll`, Python modules | ~50 MiB | | DirectML + MSVC runtime | `DirectML.dll` (18 MB), `msvcp140*`, `vcruntime140*` | ~19 MiB | `nvinfer_builder_resource_10.dll` (1.88 GiB of precompiled CUDA kernels) is loaded **only** when a TensorRT engine has to be built from scratch. Do not delete it unless you never use `TensorrtExecutionProvider`: `nvinfer_10.dll` loads fine without it, but engine construction fails at runtime. ### Layout The tree below is the repository tree; drop the `onnxruntime/` prefix and it is a valid site-packages package. ``` onnxruntime/ ├── __init__.py # public API: InferenceSession, get_available_providers, ... ├── uort.py # create_session(): resilient provider fallback helper ├── LICENSE Privacy.md ThirdPartyNotices.txt ├── capi/ # 63 files, 5.20 GiB — everything native │ ├── onnxruntime.dll # core runtime (C API; also usable via ctypes) │ ├── onnxruntime_pybind11_state.pyd │ ├── onnxruntime_providers_{cuda,tensorrt,openvino,shared}.dll │ ├── nvinfer* / nvonnxparser_10.dll │ ├── cublas* / cudnn* / cufft* / curand* / cudart64_12.dll │ ├── openvino*.dll / tbb*.dll / DirectML.dll / msvcp140*.dll / vcruntime140*.dll │ ├── _uort_dll_dirs.py # registers capi/ with AddDllDirectory + PATH on import │ ├── _pybind_state.py # patched to call _uort_dll_dirs before loading the extension │ ├── build_and_package_info.py │ └── version_info.py ├── backend/ 3 files # ORT EP unit-test harness ├── datasets/ 4 files # three tiny .onnx models used by those tests ├── quantization/ 59 files # Python tooling ├── tools/ 80 files # Python tooling └── transformers/ 165 files # HF transformers integration layer ``` No C/C++ headers are included — for native code use the `onnxruntime.dll` C API together with the public ORT 1.30.1 headers. ## Requirements - **Windows 10/11 x64.** - **CPython 3.12 x64.** The extension links `python312.dll`; 3.11/3.13 raise a DLL load error. - **No NVIDIA/Intel SDKs are needed** — CUDA, cuDNN, TensorRT, OpenVINO and the MSVC runtime are all in `capi/`. - The **GPU driver stays a system prerequisite** (it is not, and cannot be, bundled): - CUDA EP: NVIDIA driver with CUDA 12.x support (≥ 527.41; 551+ recommended for CUDA 12.9). - TensorRT EP: the same driver **and** compute capability ≥ 7.5. - DirectML EP: any DX12-capable GPU with a WDDM 2.x driver. - OpenVINO EP: Intel CPU, or Intel GPU/NPU with a recent graphics driver. - For `import onnxruntime` from a normal Python process nothing else is required; the package calls `os.add_dll_directory(capi)` and prepends it to `PATH` itself. - If you load `capi/onnxruntime.dll` from a **non-Python** host, add `capi` to `PATH` (or use `SetDllDirectoryW`) before loading it. ## Installation The folder is **not a pip wheel** (there is no `*.dist-info`), so `pip` neither sees nor uninstalls it. Copy the folder. ### Option A — into a Python environment ```powershell # the download landed in .\uort\onnxruntime $sp = python -c "import sysconfig; print(sysconfig.get_paths()['purelib'])" Remove-Item "$sp\onnxruntime" -Recurse -Force -ErrorAction SilentlyContinue Copy-Item .\uort\onnxruntime "$sp\onnxruntime" -Recurse -Force # make sure no other ONNX Runtime is installed next to it python -m pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime-directml onnxruntime-openvino ``` ### Option B — into an application folder Copy `onnxruntime\` next to your `python.exe` / `python312.dll`, or into `\_internal\python\Lib\site-packages\onnxruntime`, keeping the directory name exactly `onnxruntime`. This is how the folder is used by the DeepFaceLive build it was made for. ### Verify ```python import onnxruntime as ort print(ort.__version__) # 1.30.1 print(ort.get_available_providers()) # all five providers (compile-time list) print(ort.__file__) # ...\site-packages\onnxruntime\__init__.py ``` Then confirm that a provider really works on this machine by building a session — see below. `get_available_providers()` lists what the **build** contains; whether a device can actually be used depends on the driver, the GPU and the model. ## Usage ### CPU ```python import onnxruntime as ort sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"]) print(sess.get_providers()) ``` ### CUDA ```python cuda = { "device_id": 0, "cudnn_conv_algo_search": "HEURISTIC", # HEURISTIC | EXHAUSTIVE | DEFAULT "arena_extend_strategy": "kSameAsRequested", } sess = ort.InferenceSession("model.onnx", providers=[("CUDAExecutionProvider", cuda), "CPUExecutionProvider"]) ``` ### TensorRT with engine cache (recommended) Without a cache, every session creation rebuilds the engine from ONNX, which can take minutes and requires `nvinfer_builder_resource_10.dll`. ```python import os cache = r"D:\trt-cache" os.makedirs(cache, exist_ok=True) trt = { "device_id": 0, "trt_fp16_enable": True, "trt_engine_cache_enable": True, "trt_engine_cache_path": cache, "trt_timing_cache_enable": True, "trt_max_workspace_size": 4 << 30, } sess = ort.InferenceSession( "model.onnx", providers=[ ("TensorrtExecutionProvider", trt), ("CUDAExecutionProvider", {"device_id": 0}), "CPUExecutionProvider", ], ) ``` Engine files are specific to the GPU architecture, the TensorRT/CUDA versions and the exact input shapes — they are not portable between machines. ### DirectML DirectML must not be combined with CUDA / TensorRT / OpenVINO in one session, and requires the DirectML session options: ```python so = ort.SessionOptions() so.enable_mem_pattern = False so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL sess = ort.InferenceSession( "model.onnx", so, providers=[("DmlExecutionProvider", {"device_id": 0}), "CPUExecutionProvider"], ) ``` Do not call `Run` concurrently on the same DML session. Separate DML sessions may run in parallel (measure the contention). ### OpenVINO ```python ov = {"device_type": "GPU"} # "CPU" | "GPU" | "GPU.0" | "NPU" | "AUTO" sess = ort.InferenceSession("model.onnx", providers=[("OpenVINOExecutionProvider", ov), "CPUExecutionProvider"]) ``` Not every model maps cleanly onto the OpenVINO GPU plugin; large detection models can fail inside `OpenVINO-EP-subgraph` with a very long graph dump. The CPU device is the safest target. ### Resilient helper: `onnxruntime.uort.create_session` The Python binding only retries other providers for `ValueError`/`RuntimeError`. TensorRT raises `onnxruntime.capi.onnxruntime_pybind11_state.Fail`, whose base class is `Exception`, so a GPU that cannot run the engine aborts session creation even when `CUDAExecutionProvider` is in the list. `uort.create_session` tries whole provider lists, best first, and keeps DirectML in a list of its own: ```python from onnxruntime.uort import create_session sess = create_session("model.onnx") # TRT -> CUDA -> DML -> OpenVINO -> CPU sess = create_session("model.onnx", prefer=["DmlExecutionProvider"]) sess = create_session("model.onnx", quiet=True) # no per-attempt diagnostics print(sess.get_providers()) # what actually ran ``` ## Known limitations 1. **Python 3.12 x64 only.** No 3.11/3.13 builds, no Linux, no macOS. 2. **No `dist-info`.** `pip list` will not show the package; a later `pip install onnxruntime` will place a second copy that may shadow this one (site-packages search order). 3. **TensorRT on Pascal and older fails.** Engine building stops with `Target GPU SM 61 is not supported by this TensorRT release`; because the error is `Fail`, the automatic fallback does not catch it. Use the helper above or an explicit `try/except`. 4. **TensorRT builds engines on first use.** Expect a long first session creation unless `trt_engine_cache_enable` is set. 5. **Many files instead of one.** 379 objects per download; a client that fetches them one by one over plain HTTPS pays 379 round-trips. Use `hf download`, `snapshot_download` or `git` (with `git-lfs`) so transfers run in parallel and resume. 6. **`__pycache__` reappears** after the first import — harmless on disk, but exclude it if you re-upload the folder (`hf upload ... --exclude "**/__pycache__/**"`). 7. **`get_available_providers()` is compile-time.** It does not detect a missing driver or GPU; probe by creating a session per provider. 8. **5.20 GiB on disk and in transit**, and no partial packs: even if you only need CPU + DirectML you still download the CUDA and TensorRT runtimes. ## Licences This package redistributes binaries from several vendors. `ThirdPartyNotices.txt` (330 KB) and `LICENSE` (MIT, Microsoft Corporation) are included in the package root. | Component | Licence / terms | |---|---| | ONNX Runtime 1.30.1 core and Python bindings | MIT | | UORT build scripts, `uort.py`, `_uort_dll_dirs.py` additions | MIT | | NVIDIA CUDA runtime, cuBLAS, cuFFT, cuRAND, cuDNN | NVIDIA CUDA Toolkit / cuDNN EULA — redistribution of the runtime libraries is permitted with the notices preserved | | NVIDIA TensorRT (`nvinfer*`, `nvonnxparser`) | NVIDIA TensorRT licence agreement | | Intel OpenVINO, oneDNN, TBB | Apache License 2.0 | | Microsoft DirectML | Microsoft redistributable component | | MSVC runtime (`msvcp140*`, `vcruntime140*`) | Microsoft Visual C++ redistributable terms | This build is **modified** relative to upstream ONNX Runtime (custom packaging, patched `_pybind_state.py`, extra top-level module `uort.py`, bundled provider DLLs), as permitted by the MIT licence. It is not produced, endorsed or supported by Microsoft, NVIDIA or Intel. NVIDIA and Intel libraries are included so that the runtime works out of the box; nothing here grants you rights beyond the vendors' own terms. If you redistribute this package further, keep `LICENSE`, `ThirdPartyNotices.txt` and this section intact. ## Build provenance | Item | Version | |---|---| | ONNX Runtime source | 1.30.1 (pinned release tag, not `main`) | | Package name | `uort-python` | | CUDA | 12.9.86 | | cuDNN | 9 | | TensorRT | 10.9.0.34 | | OpenVINO | 2026.2.1 (`vc_mt`) | | DirectML | `Microsoft.AI.DirectML` (`DirectML.dll` 17.67 MB) | | Compiler | Visual Studio 2022, MSVC 19.44 | | Python | 3.12 x64 (pybind11) | | GPU targets | `sm_61;75;80;86;89;90;100;120` | | Build system | CMake presets, `build.bat --build_shared_lib --enable_pybind` with `--use_cuda --use_tensorrt --use_openvino --use_dml` | ## Repository layout ``` README.md this card onnxruntime/ the package folder — 379 files, 5.20 GiB ``` > Uploading with `hf upload` stores large files as LFS / Xet objects automatically. If you push > with `git` instead, run `git lfs track "*.dll" "*.pyd"` before the first commit, otherwise the > push is rejected for files above the plain-git limit. ## Кратко (по-русски) Это собранный из исходников ONNX Runtime 1.30.1 для Windows x64 / Python 3.12, в который вкомпилированы пять провайдеров — TensorRT, CUDA, OpenVINO, DirectML и CPU — вместе со всеми нативными библиотеками (CUDA 12.9, cuDNN 9, TensorRT 10.9, OpenVINO 2026.2.1, DirectML). Здесь лежит **папка** `onnxruntime` — 379 файлов в 31 подкаталоге, 5,20 ГиБ. Ничего распаковывать не нужно: скачали — и копируете папку в `Lib\site-packages` (это не pip-wheel, `pip` его не видит). Нужен только драйвер GPU; SDK ставить не надо — всё уже внутри. TensorRT работает на видеокартах с compute capability ≥ 7.5 (Pascal и старше — только CUDA или DirectML).