Blame for README.md

---
license: mit
tags:
  - onnxruntime
  - onnx
  - inference
  - cuda
  - tensorrt
  - openvino
  - directml
  - windows
---

# Universal ONNX Runtime (UORT) 1.30.1 — Windows x64 package folder

A self-contained build of **ONNX Runtime 1.30.1** for **Windows x64 / CPython 3.12** with five
execution providers compiled in and every native dependency bundled next to the Python
extension: **TensorRT, CUDA, OpenVINO, DirectML and CPU**.

This repository publishes the package as a **plain directory tree** — `onnxruntime/` at the repo
root, exactly as it has to appear in `Lib\site-packages`. There is no archive to unpack: what you
download is already the installable folder.

The point of the package is that you install it by copying one folder — no CUDA Toolkit, no
cuDNN download, no TensorRT SDK, no `pip install onnxruntime-gpu`. Everything the providers need
is in `capi/`, and the package registers that directory with the Windows loader on import.

| | |
|---|---|
| Contents | `onnxruntime/` — **379 files** in **31 subdirectories** |
| Stored / transferred size | **5 585 491 685 bytes** (5.20 GiB) |
| Python modules | 317 `.py` files |
| Native binaries | 52 `.dll` + 1 `.pyd`, all in `capi/` |
| `__pycache__` | none (removed before publishing) |
| Package version | `uort-python` 1.30.1 |
| Platform | Windows 10/11 x64 only |
| Python ABI | CPython 3.12 x64 (`python312.dll`) |
| Licence | MIT (ONNX Runtime) + third-party terms, see [Licences](#licences) |

## Integrity

There is no single archive hash, because there is no archive. Every file is stored as an
LFS / Xet object keyed by the **SHA-256 of its content**, and both `hf download` and
`git lfs` verify that hash while writing the file — a corrupted transfer is rejected, not
silently accepted.

Spot-check values if you want to confirm a few key binaries yourself:

| File | Bytes | SHA-256 |
|---|---|---|
| `capi\onnxruntime.dll` | 24 535 040 | `a4599b0168c8b3e485e50c4493559280778f7c533d66bbb3ea372073f316b67f` |
| `capi\onnxruntime_pybind11_state.pyd` | 25 851 904 | `e54c4900fc60c521dc597dc0b0c87835a18ea6ee2efd38af78c70ac6f10abcbb` |
| `capi\onnxruntime_providers_cuda.dll` | 689 375 232 | `2c744e0d1683518fa88e229ca006304aad6a29b81b9f0229596969c89ba80c12` |
| `capi\nvinfer_builder_resource_10.dll` | 1 971 797 024 | `2b8496c57a8434534e8a3c56e732207f1c74bc37ce7300d8f0d3542cdff02e12` |

```powershell
(Get-FileHash .\onnxruntime\capi\onnxruntime.dll -Algorithm SHA256).Hash.ToLower()
```

## Download

Grab the whole folder — 379 separate HTTP requests by hand is not a good idea.

```powershell
# Hugging Face CLI: chunked, parallel, resumable
hf download CatSnowdrop/universal_onnxruntime --local-dir .\uort
# result: .\uort\onnxruntime\__init__.py, .\uort\onnxruntime\capi\...

# or git (needs git-lfs; Xet accelerates it if hf_xet is installed)
git clone https://huggingface.co/CatSnowdrop/universal_onnxruntime uort
```

```python
from huggingface_hub import snapshot_download

path = snapshot_download(
    repo_id="CatSnowdrop/universal_onnxruntime",
    local_dir="uort",
    max_workers=8,
)
print(path)   # -> uort\onnxruntime
```

Single files can be fetched directly; the endpoint supports HTTP `Range`, so a large `.dll`
resumes instead of restarting:

```powershell
curl.exe -L -C - -o onnxruntime_providers_cuda.dll `
  "https://huggingface.co/CatSnowdrop/universal_onnxruntime/resolve/main/onnxruntime/capi/onnxruntime_providers_cuda.dll"
```

`resolve/main` redirects to a commit-pinned CDN path, so re-uploading never invalidates a
download that is already running.

> **Expect ~5.20 GiB of traffic.** Unlike a zip, DLLs are stored uncompressed, so the folder is
> ~1.5× larger on the wire than the equivalent archive (3.48 GiB). In exchange there is no
> unpack step and no second copy on disk: peak usage is 5.20 GiB, not 8.8 GiB.

## What's inside

### Execution providers

`onnxruntime.get_available_providers()` reports, in this order:

```python
['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'OpenVINOExecutionProvider',
 'DmlExecutionProvider', 'CPUExecutionProvider']
```

| EP | Bundled runtime | Notes |
|---|---|---|
| `TensorrtExecutionProvider` | TensorRT 10.9, `nvonnxparser`, `nvinfer_dispatch` | engines are built at session creation; requires compute capability ≥ 7.5 |
| `CUDAExecutionProvider` | CUDA 12.9 runtime, cuBLAS, cuFFT, cuRAND, cuDNN 9 | driver is the only system prerequisite |
| `OpenVINOExecutionProvider` | OpenVINO 2026.2.1 + Intel CPU/GPU/NPU plugins, TBB | CPU, GPU and NPU devices |
| `DmlExecutionProvider` | `DirectML.dll` 17.67 MB | any DirectX 12 GPU (NVIDIA/AMD/Intel) |
| `CPUExecutionProvider` | MLAS kernels inside `onnxruntime.dll` | always available |

GPU kernels are compiled for `sm_61;75;80;86;89;90;100;120`. Note that the **TensorRT** EP,
unlike CUDA, does not support 6.1: TensorRT 10 dropped Pascal, so a P10x/GTX 10x0 card can use
CUDA or DirectML but not TensorRT.

### Why the package is 5.2 GiB

Almost all of the volume is the CUDA / TensorRT / OpenVINO runtimes; the Python side is ~4 MiB.

| Group | Main files | Size |
|---|---|---|
| TensorRT | `nvinfer_builder_resource_10.dll` (1880 MB), `nvinfer_10.dll` (421 MB), `nvinfer_plugin_10.dll` (49 MB) | ~2.30 GiB |
| CUDA | `onnxruntime_providers_cuda.dll` (657 MB), `cublasLt64_12.dll` (638 MB), `cudnn_engines_precompiled64_9.dll` (490 MB), `cufft64_11.dll` (274 MB), `cudnn_adv64_9.dll` (269 MB), `cudnn_ops64_9.dll` (121 MB) | ~2.64 GiB |
| OpenVINO | `openvino.dll`, `openvino_intel_{cpu,gpu,npu}_plugin.dll`, `openvino_intel_npu_compiler.dll`, frontends, TBB | ~200 MiB |
| Core | `onnxruntime.dll` (23 MB), `onnxruntime_pybind11_state.pyd` (25 MB), `onnxruntime_providers_shared.dll`, Python modules | ~50 MiB |
| DirectML + MSVC runtime | `DirectML.dll` (18 MB), `msvcp140*`, `vcruntime140*` | ~19 MiB |

`nvinfer_builder_resource_10.dll` (1.88 GiB of precompiled CUDA kernels) is loaded **only** when
a TensorRT engine has to be built from scratch. Do not delete it unless you never use
`TensorrtExecutionProvider`: `nvinfer_10.dll` loads fine without it, but engine construction
fails at runtime.

### Layout

The tree below is the repository tree; drop the `onnxruntime/` prefix and it is a valid
site-packages package.

```
onnxruntime/
├── __init__.py                  # public API: InferenceSession, get_available_providers, ...
├── uort.py                      # create_session(): resilient provider fallback helper
├── LICENSE  Privacy.md  ThirdPartyNotices.txt
├── capi/                        # 63 files, 5.20 GiB — everything native
│   ├── onnxruntime.dll          # core runtime (C API; also usable via ctypes)
│   ├── onnxruntime_pybind11_state.pyd
│   ├── onnxruntime_providers_{cuda,tensorrt,openvino,shared}.dll
│   ├── nvinfer* / nvonnxparser_10.dll
│   ├── cublas* / cudnn* / cufft* / curand* / cudart64_12.dll
│   ├── openvino*.dll / tbb*.dll / DirectML.dll / msvcp140*.dll / vcruntime140*.dll
│   ├── _uort_dll_dirs.py        # registers capi/ with AddDllDirectory + PATH on import
│   ├── _pybind_state.py         # patched to call _uort_dll_dirs before loading the extension
│   ├── build_and_package_info.py
│   └── version_info.py
├── backend/         3 files      # ORT EP unit-test harness
├── datasets/        4 files      # three tiny .onnx models used by those tests
├── quantization/   59 files      # Python tooling
├── tools/          80 files      # Python tooling
└── transformers/  165 files      # HF transformers integration layer
```

No C/C++ headers are included — for native code use the `onnxruntime.dll` C API together with
the public ORT 1.30.1 headers.

## Requirements

- **Windows 10/11 x64.**
- **CPython 3.12 x64.** The extension links `python312.dll`; 3.11/3.13 raise a DLL load error.
- **No NVIDIA/Intel SDKs are needed** — CUDA, cuDNN, TensorRT, OpenVINO and the MSVC runtime are
  all in `capi/`.
- The **GPU driver stays a system prerequisite** (it is not, and cannot be, bundled):
  - CUDA EP: NVIDIA driver with CUDA 12.x support (≥ 527.41; 551+ recommended for CUDA 12.9).
  - TensorRT EP: the same driver **and** compute capability ≥ 7.5.
  - DirectML EP: any DX12-capable GPU with a WDDM 2.x driver.
  - OpenVINO EP: Intel CPU, or Intel GPU/NPU with a recent graphics driver.
- For `import onnxruntime` from a normal Python process nothing else is required; the package
  calls `os.add_dll_directory(capi)` and prepends it to `PATH` itself.
- If you load `capi/onnxruntime.dll` from a **non-Python** host, add `capi` to `PATH` (or use
  `SetDllDirectoryW`) before loading it.

## Installation

The folder is **not a pip wheel** (there is no `*.dist-info`), so `pip` neither sees nor
uninstalls it. Copy the folder.

### Option A — into a Python environment

```powershell
# the download landed in .\uort\onnxruntime
$sp = python -c "import sysconfig; print(sysconfig.get_paths()['purelib'])"
Remove-Item "$sp\onnxruntime" -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item .\uort\onnxruntime "$sp\onnxruntime" -Recurse -Force

# make sure no other ONNX Runtime is installed next to it
python -m pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime-directml onnxruntime-openvino
```

### Option B — into an application folder

Copy `onnxruntime\` next to your `python.exe` / `python312.dll`, or into
`<app>\_internal\python\Lib\site-packages\onnxruntime`, keeping the directory name exactly
`onnxruntime`. This is how the folder is used by the DeepFaceLive build it was made for.

### Verify

```python
import onnxruntime as ort

print(ort.__version__)                 # 1.30.1
print(ort.get_available_providers())   # all five providers (compile-time list)
print(ort.__file__)                    # ...\site-packages\onnxruntime\__init__.py
```

Then confirm that a provider really works on this machine by building a session — see below.
`get_available_providers()` lists what the **build** contains; whether a device can actually be
used depends on the driver, the GPU and the model.

## Usage

### CPU

```python
import onnxruntime as ort

sess = ort.InferenceSession("model.onnx", providers=["CPUExecutionProvider"])
print(sess.get_providers())
```

### CUDA

```python
cuda = {
    "device_id": 0,
    "cudnn_conv_algo_search": "HEURISTIC",   # HEURISTIC | EXHAUSTIVE | DEFAULT
    "arena_extend_strategy": "kSameAsRequested",
}
sess = ort.InferenceSession("model.onnx", providers=[("CUDAExecutionProvider", cuda), "CPUExecutionProvider"])
```

### TensorRT with engine cache (recommended)

Without a cache, every session creation rebuilds the engine from ONNX, which can take minutes
and requires `nvinfer_builder_resource_10.dll`.

```python
import os

cache = r"D:\trt-cache"
os.makedirs(cache, exist_ok=True)

trt = {
    "device_id": 0,
    "trt_fp16_enable": True,
    "trt_engine_cache_enable": True,
    "trt_engine_cache_path": cache,
    "trt_timing_cache_enable": True,
    "trt_max_workspace_size": 4 << 30,
}
sess = ort.InferenceSession(
    "model.onnx",
    providers=[
        ("TensorrtExecutionProvider", trt),
        ("CUDAExecutionProvider", {"device_id": 0}),
        "CPUExecutionProvider",
    ],
)
```

Engine files are specific to the GPU architecture, the TensorRT/CUDA versions and the exact
input shapes — they are not portable between machines.

### DirectML

DirectML must not be combined with CUDA / TensorRT / OpenVINO in one session, and requires the
DirectML session options:

```python
so = ort.SessionOptions()
so.enable_mem_pattern = False
so.execution_mode = ort.ExecutionMode.ORT_SEQUENTIAL

sess = ort.InferenceSession(
    "model.onnx", so,
    providers=[("DmlExecutionProvider", {"device_id": 0}), "CPUExecutionProvider"],
)
```

Do not call `Run` concurrently on the same DML session. Separate DML sessions may run in
parallel (measure the contention).

### OpenVINO

```python
ov = {"device_type": "GPU"}   # "CPU" | "GPU" | "GPU.0" | "NPU" | "AUTO"
sess = ort.InferenceSession("model.onnx", providers=[("OpenVINOExecutionProvider", ov), "CPUExecutionProvider"])
```

Not every model maps cleanly onto the OpenVINO GPU plugin; large detection models can fail
inside `OpenVINO-EP-subgraph` with a very long graph dump. The CPU device is the safest target.

### Resilient helper: `onnxruntime.uort.create_session`

The Python binding only retries other providers for `ValueError`/`RuntimeError`. TensorRT
raises `onnxruntime.capi.onnxruntime_pybind11_state.Fail`, whose base class is `Exception`, so
a GPU that cannot run the engine aborts session creation even when `CUDAExecutionProvider` is
in the list. `uort.create_session` tries whole provider lists, best first, and keeps DirectML
in a list of its own:

```python
from onnxruntime.uort import create_session

sess = create_session("model.onnx")                            # TRT -> CUDA -> DML -> OpenVINO -> CPU
sess = create_session("model.onnx", prefer=["DmlExecutionProvider"])
sess = create_session("model.onnx", quiet=True)                # no per-attempt diagnostics
print(sess.get_providers())                                    # what actually ran
```

## Known limitations

1. **Python 3.12 x64 only.** No 3.11/3.13 builds, no Linux, no macOS.
2. **No `dist-info`.** `pip list` will not show the package; a later `pip install onnxruntime`
   will place a second copy that may shadow this one (site-packages search order).
3. **TensorRT on Pascal and older fails.** Engine building stops with
   `Target GPU SM 61 is not supported by this TensorRT release`; because the error is
   `Fail`, the automatic fallback does not catch it. Use the helper above or an explicit
   `try/except`.
4. **TensorRT builds engines on first use.** Expect a long first session creation unless
   `trt_engine_cache_enable` is set.
5. **Many files instead of one.** 379 objects per download; a client that fetches them one by
   one over plain HTTPS pays 379 round-trips. Use `hf download`, `snapshot_download` or `git`
   (with `git-lfs`) so transfers run in parallel and resume.
6. **`__pycache__` reappears** after the first import — harmless on disk, but exclude it if you
   re-upload the folder (`hf upload ... --exclude "**/__pycache__/**"`).
7. **`get_available_providers()` is compile-time.** It does not detect a missing driver or GPU;
   probe by creating a session per provider.
8. **5.20 GiB on disk and in transit**, and no partial packs: even if you only need CPU +
   DirectML you still download the CUDA and TensorRT runtimes.

## Licences

This package redistributes binaries from several vendors. `ThirdPartyNotices.txt` (330 KB) and
`LICENSE` (MIT, Microsoft Corporation) are included in the package root.

| Component | Licence / terms |
|---|---|
| ONNX Runtime 1.30.1 core and Python bindings | MIT |
| UORT build scripts, `uort.py`, `_uort_dll_dirs.py` additions | MIT |
| NVIDIA CUDA runtime, cuBLAS, cuFFT, cuRAND, cuDNN | NVIDIA CUDA Toolkit / cuDNN EULA — redistribution of the runtime libraries is permitted with the notices preserved |
| NVIDIA TensorRT (`nvinfer*`, `nvonnxparser`) | NVIDIA TensorRT licence agreement |
| Intel OpenVINO, oneDNN, TBB | Apache License 2.0 |
| Microsoft DirectML | Microsoft redistributable component |
| MSVC runtime (`msvcp140*`, `vcruntime140*`) | Microsoft Visual C++ redistributable terms |

This build is **modified** relative to upstream ONNX Runtime (custom packaging, patched
`_pybind_state.py`, extra top-level module `uort.py`, bundled provider DLLs), as permitted by
the MIT licence. It is not produced, endorsed or supported by Microsoft, NVIDIA or Intel.

NVIDIA and Intel libraries are included so that the runtime works out of the box; nothing here
grants you rights beyond the vendors' own terms. If you redistribute this package further, keep
`LICENSE`, `ThirdPartyNotices.txt` and this section intact.

## Build provenance

| Item | Version |
|---|---|
| ONNX Runtime source | 1.30.1 (pinned release tag, not `main`) |
| Package name | `uort-python` |
| CUDA | 12.9.86 |
| cuDNN | 9 |
| TensorRT | 10.9.0.34 |
| OpenVINO | 2026.2.1 (`vc_mt`) |
| DirectML | `Microsoft.AI.DirectML` (`DirectML.dll` 17.67 MB) |
| Compiler | Visual Studio 2022, MSVC 19.44 |
| Python | 3.12 x64 (pybind11) |
| GPU targets | `sm_61;75;80;86;89;90;100;120` |
| Build system | CMake presets, `build.bat --build_shared_lib --enable_pybind` with `--use_cuda --use_tensorrt --use_openvino --use_dml` |

## Repository layout

```
README.md          this card
onnxruntime/       the package folder — 379 files, 5.20 GiB
```

> Uploading with `hf upload` stores large files as LFS / Xet objects automatically. If you push
> with `git` instead, run `git lfs track "*.dll" "*.pyd"` before the first commit, otherwise the
> push is rejected for files above the plain-git limit.

## Кратко (по-русски)

Это собранный из исходников ONNX Runtime 1.30.1 для Windows x64 / Python 3.12, в который
вкомпилированы пять провайдеров — TensorRT, CUDA, OpenVINO, DirectML и CPU — вместе со всеми
нативными библиотеками (CUDA 12.9, cuDNN 9, TensorRT 10.9, OpenVINO 2026.2.1, DirectML).

Здесь лежит **папка** `onnxruntime` — 379 файлов в 31 подкаталоге, 5,20 ГиБ.
Ничего распаковывать не нужно: скачали — и копируете папку в `Lib\site-packages` (это не
pip-wheel, `pip` его не видит). Нужен только драйвер GPU; SDK ставить не надо — всё уже внутри.
TensorRT работает на видеокартах с compute capability ≥ 7.5 (Pascal и старше — только CUDA
или DirectML).