Upload 379 files

---
license: mit
tags:
- onnxruntime
- onnx
- inference
- cuda
- tensorrt
- openvino
- directml
- windows
---

# Universal ONNX Runtime (UORT) 1.30.1 — Windows x64 package folder

A self-contained build of **ONNX Runtime 1.30.1** for **Windows x64 / CPython 3.12** with five
execution providers compiled in and every native dependency bundled next to the Python
extension: **TensorRT, CUDA, OpenVINO, DirectML and CPU**.

This repository publishes the package as a **plain directory tree** — `onnxruntime/` at the repo
root, exactly as it has to appear in `Lib\site-packages`. There is no archive to unpack: what you
download is already the installable folder.

The point of the package is that you install it by copying one folder — no CUDA Toolkit, no
cuDNN download, no TensorRT SDK, no `pip install onnxruntime-gpu`. Everything the providers need
is in `capi/`, and the package registers that directory with the Windows loader on import.

| | |
|---|---|
| Contents | `onnxruntime/` — **379 files** in **31 subdirectories** |
| Stored / transferred size | **5 585 491 685 bytes** (5.20 GiB) |
| Python modules | 317 `.py` files |
| Native binaries | 52 `.dll` + 1 `.pyd`, all in `capi/` |
| `__pycache__` | none (removed before publishing) |
| Package version | `uort-python` 1.30.1 |
| Platform | Windows 10/11 x64 only |
| Python ABI | CPython 3.12 x64 (`python312.dll`) |
| Licence | MIT (ONNX Runtime) + third-party terms, see [Licences](#licences) |

## Integrity

There is no single archive hash, because there is no archive. Every file is stored as an
LFS / Xet object keyed by the **SHA-256 of its content**, and both `hf download` and
`git lfs` verify that hash while writing the file — a corrupted transfer is rejected, not
silently accepted.

Spot-check values if you want to confirm a few key binaries yourself:

| File | Bytes | SHA-256 |
|---|---|---|
| `capi\onnxruntime.dll` | 24 535 040 | `a4599b0168c8b3e485e50c4493559280778f7c533d66bbb3ea372073f316b67f` |
| `capi\onnxruntime_pybind11_state.pyd` | 25 851 904 | `e54c4900fc60c521dc597dc0b0c87835a18ea6ee2efd38af78c70ac6f10abcbb` |
| `capi\onnxruntime_providers_cuda.dll` | 689 375 232 | `2c744e0d1683518fa88e229ca006304aad6a29b81b9f0229596969c89ba80c12` |
| `capi\nvinfer_builder_resource_10.dll` | 1 971 797 024 | `2b8496c57a8434534e8a3c56e732207f1c74bc37ce7300d8f0d3542cdff02e12` |

```powershell
(Get-FileHash .\onnxruntime\capi\onnxruntime.dll -Algorithm SHA256).Hash.ToLower()
```

## Download

Grab the whole folder — 379 separate HTTP requests by hand is not a good idea.

```powershell
# Hugging Face CLI: chunked, parallel, resumable
hf download CatSnowdrop/universal_onnxruntime --local-dir .\uort
# result: .\uort\onnxruntime\__init__.py, .\uort\onnxruntime\capi\...

# or git (needs git-lfs; Xet accelerates it if hf_xet is installed)
git clone https://huggingface.co/CatSnowdrop/universal_onnxruntime uort
```

```python
from huggingface_hub import snapshot_download

path = snapshot_download(
repo_id="CatSnowdrop/universal_onnxruntime",
local_dir="uort",
max_workers=8,
)
print(path) # -> uort\onnxruntime
```

Single files can be fetched directly; the endpoint supports HTTP `Range`, so a large `.dll`
resumes instead of restarting:

```powershell
curl.exe -L -C - -o onnxruntime_providers_cuda.dll `
"https://huggingface.co/CatSnowdrop/universal_onnxruntime/resolve/main/onnxruntime/capi/onnxruntime_providers_cuda.dll"
```

`resolve/main` redirects to a commit-pinned CDN path, so re-uploading never invalidates a
download that is already running.

> **Expect ~5.20 GiB of traffic.** Unlike a zip, DLLs are stored uncompressed, so the folder is
> ~1.5× larger on the wire than the equivalent archive (3.48 GiB). In exchange there is no
> unpack step and no second copy on disk: peak usage is 5.20 GiB, not 8.8 GiB.

## What's inside

### Execution providers

`onnxruntime.get_available_providers()` reports, in this order:

```python
['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'OpenVINOExecutionProvider',
'DmlExecutionProvider', 'CPUExecutionProvider']
```

| EP | Bundled runtime | Notes |
|---|---|---|
| `TensorrtExecutionProvider` | TensorRT 10.9, `nvonnxparser`, `nvinfer_dispatch` | engines are built at session creation; requires compute capability ≥ 7.5 |
| `CUDAExecutionProvider` | CUDA 12.9 runtime, cuBLAS, cuFFT, cuRAND, cuDNN 9 | driver is the only system prerequisite |
| `OpenVINOExecutionProvider` | OpenVINO 2026.2.1 + Intel CPU/GPU/NPU plugins, TBB | CPU, GPU and NPU devices |
| `DmlExecutionProvider` | `DirectML.dll` 17.67 MB | any DirectX 12 GPU (NVIDIA/AMD/Intel) |
| `CPUExecutionProvider` | MLAS kernels inside `onnxruntime.dll` | always available |

GPU kernels are compiled for `sm_61;75;80;86;89;90;100;120`. Note that the **TensorRT** EP,
unlike CUDA, does not support 6.1: TensorRT 10 dropped Pascal, so a P10x/GTX 10x0 card can use
CUDA or DirectML but not TensorRT.

### Why the package is 5.2 GiB

Almost all of the volume is the CUDA / TensorRT / OpenVINO runtimes; the Python side is ~4 MiB.

| Group | Main files | Size |
|---|---|---|
| TensorRT | `nvinfer_builder_resource_10.dll` (1880 MB), `nvinfer_10.dll` (421 MB), `nvinfer_plugin_10.dll` (49 MB) | ~2.30 GiB |
| CUDA | `onnxruntime_providers_cuda.dll` (657 MB), `cublasLt64_12.dll` (638 MB), `cudnn_engines_precompiled64_9.dll` (490 MB), `cufft64_11.dll` (274 MB), `cudnn_adv64_9.dll` (269 MB), `cudnn_ops64_9.dll` (121 MB) | ~2.64 GiB |
| OpenVINO | `openvino.dll`, `openvino_intel_{cpu,gpu,npu}_plugin.dll`, `openvino_intel_npu_compiler.dll`, frontends, TBB | ~200 MiB |
| Core | `onnxruntime.dll` (23 MB), `onnxruntime_pybind11_state.pyd` (25 MB), `onnxruntime_providers_shared.dll`, Python modules | ~50 MiB |
| DirectML + MSVC runtime | `DirectML.dll` (18 MB), `msvcp140*`, `vcruntime140*` | ~19 MiB |

`nvinfer_builder_resource_10.dll` (1.88 GiB of precompiled CUDA kernels) is loaded **only** when
a TensorRT engine has to be built from scratch. Do not delete it unless you never use
`TensorrtExecutionProvider`: `nvinfer_10.dll` loads fine without it, but engine construction
fails at runtime.

### Layout

The tree below is the repository tree; drop the `onnxruntime/` prefix and it is a valid
site-packages package.

```
onnxruntime/
├── __init__.py # public API: InferenceSession, get_available_providers, ...
├── uort.py # create_session(): resilient provider fallback helper
├── LICENSE Privacy.md ThirdPartyNotices.txt
├── capi/ # 63 files, 5.20 GiB — everything native
│ ├── onnxruntime.dll # core runtime (C API; also usable via ctypes)
│ ├── onnxruntime_pybind11_state.pyd
│ ├── onnxruntime_providers_{cuda,tensorrt,openvino,shared}.dll
│ ├── nvinfer* / nvonnxparser_10.dll
│ ├── cublas* / cudnn* / cufft* / curand* / cudart64_12.dll
│ ├── openvino*.dll / tbb*.dll / DirectML.dll / msvcp140*.dll / vcruntime140*.dll
│ ├── _uort_dll_dirs.py # registers capi/ with AddDllDirectory + PATH on import
│ ├── _pybind_state.py # patched to call _uort_dll_dirs before loading the extension
│ ├── build_and_package_info.py
│ └── version_info.py
├── backend/ 3 files # ORT EP unit-test harness
├── datasets/ 4 files # three tiny .onnx models used by those tests
├── quantization/ 59 files # Python tooling
├── tools/ 80 files # Python tooling
└── transformers/ 165 files # HF transformers integration layer
```

No C/C++ headers are included — for native code use the `onnxruntime.dll` C API together with
the public ORT 1.30.1 headers.

## Requirements

- **Windows 10/11 x64.**
- **CPython 3.12 x64.** The extension links `python312.dll`; 3.11/3.13 raise a DLL load error.
- **No NVIDIA/Intel SDKs are needed** — CUDA, cuDNN, TensorRT, OpenVINO and the MSVC runtime are
all in `capi/`.
- The **GPU driver stays a system prerequisite** (it is not, and cannot be, bundled):
- CUDA EP: NVIDIA driver with CUDA 12.x support (≥ 527.41; 551+ recommended for CUDA 12.9).
- TensorRT EP: the same driver **and** compute capability ≥ 7.5.
- DirectML EP: any DX12-capable GPU with a WDDM 2.x driver.
- OpenVINO EP: Intel CPU, or Intel GPU/NPU with a recent graphics driver.
- For `import onnxruntime` from a normal Python process nothing else is required; the package
calls `os.add_dll_directory(capi)` and prepends it to `PATH` itself.
- If you load `capi/onnxruntime.dll` from a **non-Python** host, add `capi` to `PATH` (or use
`SetDllDirectoryW`) before loading it.

## Installation

The folder is **not a pip wheel** (there is no `*.dist-info`), so `pip` neither sees nor
uninstalls it. Copy the folder.

### Option A — into a Python environment

```powershell
# the download landed in .\uort\onnxruntime
$sp = python -c "import sysconfig; print(sysconfig.get_paths()['purelib'])"
Remove-Item "$sp\onnxruntime" -Recurse -Force -ErrorAction SilentlyContinue
Copy-Item .\uort\onnxruntime "$sp\onnxruntime" -Recurse -Force

# make sure no other ONNX Runtime is installed next to it
python -m pip uninstall -y onnxruntime onnxruntime-gpu onnxruntime-directml onnxruntime-openvino
```

### Option B — into an application folder

Copy `onnxruntime\` next to your `python.exe` / `python312.dll`, or into
`<app>\_internal\python\Lib\site-packages\onnxruntime`, keeping the directory name exactly
`onnxruntime`. This is how the folder is used by the DeepFaceLive build it was made for.

### Verify

```python
import onnxruntime as ort

print(ort.__version__) # 1.30.1
print(ort.get_available_providers()) # all five providers (compile-time list)
print(ort.__file__) # ...\site-packages\onnxruntime\__init__.py
```

Then confirm that a provider really works on this machine by building a session — see below.
`get_available_providers()` lists what the **build** contains; whether a device can actually be
used depends on the driver, the GPU and the model.

## Usage

### CPU

```python
import onnxruntime as ort

sess = ort.I

.gitattributes

@@ -33,3 +33,49 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/concrt140.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cublas64_12.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cublasLt64_12.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudart64_12.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_adv64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_cnn64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_engines_precompiled64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_engines_runtime_compiled64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_graph64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_heuristic64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn_ops64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cudnn64_9.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/cufft64_11.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/curand64_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/DirectML.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/msvcp140_2.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/msvcp140.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/nvinfer_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/nvinfer_builder_resource_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/nvinfer_dispatch_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/nvinfer_plugin_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/nvonnxparser_10.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/onnxruntime_providers_cuda.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/onnxruntime_providers_openvino.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/onnxruntime_providers_tensorrt.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/onnxruntime_pybind11_state.pyd filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/onnxruntime.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_auto_batch_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_auto_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_c.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_hetero_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_intel_cpu_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_intel_gpu_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_intel_npu_compiler_loader.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_intel_npu_compiler.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_intel_npu_plugin.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_ir_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_onnx_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_paddle_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_pytorch_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_tensorflow_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino_tensorflow_lite_frontend.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/openvino.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/tbb12.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/tbbmalloc.dll filter=lfs diff=lfs merge=lfs -text
+onnxruntime/capi/vcruntime140.dll filter=lfs diff=lfs merge=lfs -text

onnxruntime/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) Microsoft Corporation
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

onnxruntime/Privacy.md

@@ -0,0 +1,34 @@
+# Privacy
+
+## Data Collection
+The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft's privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.
+
+***
+
+### Official Builds
+ONNX Runtime collects trace events with the goal of improving product quality. On Windows, it uses the platform's built-in ETW telemetry system; on supported Linux architectures, macOS, Android, and iOS, it uses the cross-platform 1DS telemetry SDK that is built into ONNX Runtime. Targets without a supported telemetry provider, including WebAssembly, tvOS, visionOS, Mac Catalyst, AIX, and RISC-V, do not include telemetry. In all cases, collection is subject to user consent and handled following Microsoft's privacy practices.
+
+Telemetry is turned **ON** by default in the official builds ([see here](../README.md#binaries)). Both providers are accessed through ONNX Runtime's common telemetry interface (see [telemetry.h](../onnxruntime/core/platform/telemetry.h)).
+
+### Private Builds
+The build driver enables telemetry by default for supported native platforms. The standard Windows `build.bat` wrapper explicitly passes `--no_telemetry`, so private builds made with that wrapper perform no data collection. Targets without a supported provider and builds that disable C++ exceptions automatically exclude telemetry. For information on how to disable telemetry in other builds, see [Disabling Telemetry](#disabling-telemetry) below.
+
+#### Technical Details
+
+**Windows.** The Windows provider uses the [TraceLogging](https://docs.microsoft.com/en-us/windows/win32/tracelogging/trace-logging-about) API for its implementation. This enables ONNX Runtime trace events to be collected by the operating system, and based on user consent, this data may be periodically sent to Microsoft servers following GDPR and privacy regulations for anonymity and data access controls. Windows ML and ONNX Runtime C APIs allow Trace Logging to be turned on/off (see [API pages](../README.md#api-documentation) for details); there are equivalent APIs in the C#, Python, and Java language bindings as well.
+
+**Non-Windows (Linux, macOS, Android, iOS).** These platforms use the cross-platform 1DS SDK (cpp_client_telemetry) to send the same trace events to Microsoft's telemetry backend over HTTPS. Based on user consent, this data is handled following GDPR and privacy regulations for anonymity and data access controls. ONNX Runtime C APIs allow 1DS to be turned on/off (see [API pages](../README.md#api-documentation) for details); there are equivalent APIs in the C#, Python, and Java language bindings as well.
+
+For ways to disable telemetry, see the [Disabling Telemetry](#disabling-telemetry) section below.
+
+### Disabling Telemetry
+
+Telemetry can be disabled in any of these ways:
+
+- **Disable it at build time.** Pass `--no_telemetry` to `build.py` or `build.sh`. This omits the 1DS provider from non-Windows builds and disables the Microsoft telemetry configuration on Windows. The standard Windows `build.bat` wrapper does this automatically. Unsupported targets and exception-free builds never include telemetry.
+- **Disable all telemetry at runtime (non-Windows).** Set `ORT_DISABLE_TELEMETRY=1` before ONNX Runtime initializes. This prevents the uploader, events, and persistent device identifier from being created for the process lifetime.
+- **Disable non-essential events via the API.** The C API (and the C#, Python, and Java bindings) can suppress non-essential telemetry. ONNX Runtime may already have emitted a minimal initialization event before the API can be called. On **Windows**, ETW events are recorded only when an external trace session is collecting.
+
+Telemetry-enabled static Linux builds use static curl and mbedTLS. FetchContent packages include those archives;
+vcpkg builds resolve them from the same vcpkg installation. A static consumer that links another curl or mbedTLS
+copy into the same final binary must build ORT with `--no_telemetry` to avoid ordinary static-symbol collisions.

onnxruntime/__init__.py

@@ -0,0 +1,500 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""
+ONNX Runtime is a performance-focused scoring engine for Open Neural Network Exchange (ONNX) models.
+For more information on ONNX Runtime, please see `aka.ms/onnxruntime <https://aka.ms/onnxruntime/>`_
+or the `Github project <https://github.com/microsoft/onnxruntime/>`_.
+"""
+
+import contextlib
+
+__version__ = "1.30.1"
+__author__ = "Microsoft"
+
+# we need to do device version validation (for example to check Cuda version for an onnxruntime-training package).
+# in order to know whether the onnxruntime package is for training it needs
+# to do import onnxruntime.training.ortmodule first.
+# onnxruntime.capi._pybind_state is required before import onnxruntime.training.ortmodule.
+# however, import onnxruntime.capi._pybind_state will already raise an exception if a required Cuda version
+# is not found.
+# here we need to save the exception and continue with Cuda version validation in order to post
+# meaningful messages to the user.
+# the saved exception is raised after device version validation.
+try:
+    from onnxruntime.capi._pybind_state import (
+        ExecutionMode,  # noqa: F401
+        ExecutionOrder,  # noqa: F401
+        GraphOptimizationLevel,  # noqa: F401
+        LoraAdapter,  # noqa: F401
+        ModelMetadata,  # noqa: F401
+        NodeArg,  # noqa: F401
+        OrtAllocatorType,  # noqa: F401
+        OrtArenaCfg,  # noqa: F401
+        OrtCompileApiFlags,  # noqa: F401
+        OrtDeviceMemoryType,  # noqa: F401
+        OrtEpAssignedNode,  # noqa: F401
+        OrtEpAssignedSubgraph,  # noqa: F401
+        OrtEpDevice,  # noqa: F401
+        OrtExecutionProviderDevicePolicy,  # noqa: F401
+        OrtExternalInitializerInfo,  # noqa: F401
+        OrtHardwareDevice,  # noqa: F401
+        OrtHardwareDeviceType,  # noqa: F401
+        OrtMemoryInfo,  # noqa: F401
+        OrtMemoryInfoDeviceType,  # noqa: F401
+        OrtMemType,  # noqa: F401
+        OrtSparseFormat,  # noqa: F401
+        OrtSyncStream,  # noqa: F401
+        RunOptions,  # noqa: F401
+        SessionIOBinding,  # noqa: F401
+        SessionOptions,  # noqa: F401
+        create_and_register_allocator,  # noqa: F401
+        create_and_register_allocator_v2,  # noqa: F401
+        disable_telemetry_events,  # noqa: F401
+        enable_telemetry_events,  # noqa: F401
+        get_all_providers,  # noqa: F401
+        get_available_providers,  # noqa: F401
+        get_build_info,
+        get_device,  # noqa: F401
+        get_ep_devices,  # noqa: F401
+        get_version_string,  # noqa: F401
+        has_collective_ops,  # noqa: F401
+        register_execution_provider_library,
+        set_default_logger_severity,  # noqa: F401
+        set_default_logger_verbosity,  # noqa: F401
+        set_global_thread_pool_sizes,  # noqa: F401
+        set_seed,  # noqa: F401
+        unregister_execution_provider_library,  # noqa: F401
+    )
+
+    import_capi_exception = None
+except Exception as e:
+    import_capi_exception = e
+
+from onnxruntime.capi import onnxruntime_validation
+
+if import_capi_exception:
+    raise import_capi_exception
+
+from onnxruntime.capi.onnxruntime_inference_collection import (
+    AdapterFormat,  # noqa: F401
+    InferenceSession,  # noqa: F401
+    IOBinding,  # noqa: F401
+    ModelCompiler,  # noqa: F401
+    OrtDevice,  # noqa: F401
+    OrtDeviceVendorId,  # noqa: F401
+    OrtValue,  # noqa: F401
+    SparseTensor,  # noqa: F401
+    copy_tensors,  # noqa: F401
+)
+
+# TODO: thiagofc: Temporary experimental namespace for new PyTorch front-end
+try:  # noqa: SIM105
+    from . import experimental  # noqa: F401
+except ImportError:
+    pass
+
+
+package_name, version, cuda_version = onnxruntime_validation.get_package_name_and_version_info()
+
+if version:
+    __version__ = version
+
+onnxruntime_validation.check_distro_info()
+
+
+def _get_cuda_plugin_ep_library_path() -> str | None:
+    import os  # noqa: PLC0415
+    import sys  # noqa: PLC0415
+
+    if ", cuda-plugin-ep=" not in get_build_info():
+        return None
+
+    if sys.platform == "win32":
+        library_name = "onnxruntime_providers_cuda.dll"
+    elif sys.platform == "darwin":
+        library_name = "libonnxruntime_providers_cuda.dylib"
+    else:
+        library_name = "libonnxruntime_providers_cuda.so"
+
+    library_path = os.path.join(os.path.dirname(__file__), "capi", library_name)
+    return library_path if os.path.isfile(library_path) else None
+
+
+def _register_bundled_cuda_plugin_ep(warn_on_failure: bool = False):
+    library_path = _get_cuda_plugin_ep_library_path()
+    if not library_path:
+        return
+
+    try:
+        register_execution_provider_library("CUDAExecutionProvider", library_path)
+    except Exception as e:
+        if "already registered" not in str(e).lower():
+            if warn_on_failure:
+                import warnings  # noqa: PLC0415
+
+                warnings.warn(f"Failed to register bundled CUDA plugin EP from {library_path}: {e}")
+
+
+_register_bundled_cuda_plugin_ep(warn_on_failure=True)
+
+
+def _get_package_version(package_name: str):
+    from importlib.metadata import PackageNotFoundError, version  # noqa: PLC0415
+
+    try:
+        package_version = version(package_name)
+    except PackageNotFoundError:
+        package_version = None
+    return package_version
+
+
+def _get_package_root(package_name: str, directory_name: str | None = None):
+    from importlib.metadata import PackageNotFoundError, distribution  # noqa: PLC0415
+
+    root_directory_name = directory_name or package_name
+    try:
+        dist = distribution(package_name)
+        files = dist.files or []
+
+        for file in files:
+            if file.name.endswith("__init__.py") and root_directory_name in file.parts:
+                return file.locate().parent
+
+        # Fallback to the first __init__.py
+        if not directory_name:
+            for file in files:
+                if file.name.endswith("__init__.py"):
+                    return file.locate().parent
+    except PackageNotFoundError:
+        # package not found, do nothing
+        pass
+
+    return None
+
+
+def _extract_cuda_major_version(version_str: str) -> str:
+    """Extract CUDA major version from version string (e.g., '12.1' -> '12').
+
+    Args:
+        version_str: CUDA version string to parse
+
+    Returns:
+        Major version as string, or "12" if parsing fails
+    """
+    return version_str.split(".", maxsplit=1)[0] if version_str else "12"
+
+
+def _get_cufft_version(cuda_major_version: str) -> str:
+    """Get cufft library version based on CUDA major version.
+
+    Args:
+        cuda_major_version: CUDA major version as string (e.g., "12", "13")
+
+    Returns:
+        cufft version as string
+    """
+    # cufft versions: CUDA 12.x -> 11, CUDA 13.x -> 12
+    return "12" if int(cuda_major_version) >= 13 else "11"
+
+
+def _get_nvidia_dll_paths(is_windows: bool, cuda: bool = True, cudnn: bool = True):
+    # Dynamically determine CUDA major version from build info.
+    # build_cuda_version defaults to the version this package was built with; it is a parameter for testability.
+    cuda_major_version = _extract_cuda_major_version(cuda_version)
+    cufft_version = _get_cufft_version(cuda_major_version)
+
+    # Starting with CUDA 13, NVIDIA consolidated the per-component CUDA Toolkit wheels
+    # (cublas, cufft, cuda_runtime, cuda_nvrtc, curand, ...) into a single "nvidia/cu{major}"
+    # package and dropped the "-cuNN" suffix from those package names. On Windows the DLLs
+    # moved into an architecture sub-folder ("bin/<arch>", e.g. "bin/x86_64"); on Linux the
+    # libraries are placed directly in "lib" (the wheel itself is architecture specific, so
+    # there is no arch sub-folder). cuDNN keeps its own "nvidia/cudnn" package and layout.
+    use_consolidated_layout = cuda_major_version.isdigit() and int(cuda_major_version) >= 13
+
+    if use_consolidated_layout:
+        cuda_dir = f"cu{cuda_major_version}"
+        if is_windows:
+            import platform  # noqa: PLC0415
+
+            arch = "arm64" if platform.machine().lower() in ("arm64", "aarch64") else "x86_64"
+            cuda_dll_paths = [
+                ("nvidia", cuda_dir, "bin", arch, f"cublasLt64_{cuda_major_version}.dll"),
+                ("nvidia", cuda_dir, "bin", arch, f"cublas64_{cuda_major_version}.dll"),
+                ("nvidia", cuda_dir, "bin", arch, f"cufft64_{cufft_version}.dll"),
+                ("nvidia", cuda_dir, "bin", arch, f"cudart64_{cuda_major_version}.dll"),
+            ]
+        else:  # Linux
+            # cublas64 depends on cublasLt64, so cublasLt64 should be loaded first.
+            cuda_dll_paths = [
+                ("nvidia", cuda_dir, "lib", f"libcublasLt.so.{cuda_major_version}"),
+                ("nvidia", cuda_dir, "lib", f"libcublas.so.{cuda_major_version}"),
+                ("nvidia", cuda_dir, "lib", f"libnvrtc.so.{cuda_major_version}"),
+                ("nvidia", cuda_dir, "lib", "libcurand.so.10"),
+                ("nvidia", cuda_dir, "lib", f"libcufft.so.{cufft_version}"),
+                ("nvidia", cuda_dir, "lib", f"libcudart.so.{cuda_major_version}"),
+            ]
+    elif is_windows:
+        # CUDA 12 and earlier: each component ships its own "nvidia/<component>" package.
+        # Path is relative to site-packages directory.
+        cuda_dll_paths = [
+            ("nvidia", "cublas", "bin", f"cublasLt64_{cuda_major_version}.dll"),
+            ("nvidia", "cublas", "bin", f"cublas64_{cuda_major_version}.dll"),
+            ("nvidia", "cufft", "bin", f"cufft64_{cufft_version}.dll"),
+            ("nvidia", "cuda_runtime", "bin", f"cudart64_{cuda_major_version}.dll"),
+        ]
+    else:  # Linux
+        # cublas64 depends on cublasLt64, so cublasLt64 should be loaded first.
+        cuda_dll_paths = [
+            ("nvidia", "cublas", "lib", f"libcublasLt.so.{cuda_major_version}"),
+            ("nvidia", "cublas", "lib", f"libcublas.so.{cuda_major_version}"),
+            ("nvidia", "cuda_nvrtc", "lib", f"libnvrtc.so.{cuda_major_version}"),
+            ("nvidia", "curand", "lib", "libcurand.so.10"),
+            ("nvidia", "cufft", "lib", f"libcufft.so.{cufft_version}"),
+            ("nvidia", "cuda_runtime", "lib", f"libcudart.so.{cuda_major_version}"),
+        ]
+
+    # cuDNN keeps its own "nvidia/cudnn" package layout in both old and consolidated schemes.
+    if is_windows:
+        cudnn_dll_paths = [
+            ("nvidia", "cudnn", "bin", "cudnn_engines_runtime_compiled64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_engines_precompiled64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_heuristic64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_ops64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_adv64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_graph64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn64_9.dll"),
+            ("nvidia", "cudnn", "bin", "cudnn_engines_tensor_ir64_9.dll"),
+        ]
+    else:  # Linux
+        # Do not load cudnn sub DLLs (they will be dynamically loaded later) to be consistent with PyTorch in Linux.
+        cudnn_dll_paths = [
+            ("nvidia", "cudnn", "lib", "libcudnn.so.9"),
+        ]
+
+    return (cuda_dll_paths if cuda else []) + (cudnn_dll_paths if cudnn else [])
+
+
+def print_debug_info():
+    """Print information to help debugging."""
+    import importlib.util  # noqa: PLC0415
+    import os  # noqa: PLC0415
+    import platform  # noqa: PLC0415
+    from importlib.metadata import distributions  # noqa: PLC0415
+
+    print(f"{package_name} version: {__version__}")
+    if cuda_version:
+        print(f"CUDA version used in build: {cuda_version}")
+    print("platform:", platform.platform())
+
+    print("\nPython package, version and location:")
+    ort_packages = []
+    for dist in distributions():
+        package = dist.metadata["Name"]
+        if package == "onnxruntime" or package.startswith(("onnxruntime-", "ort-")):
+            # Exclude packages whose root directory name is not onnxruntime.
+            location = _get_package_root(package, "onnxruntime")
+            if location and (package not in ort_packages):
+                ort_packages.append(package)
+                print(f"{package}=={dist.version} at {location}")
+
+    if len(ort_packages) > 1:
+        print(
+            "\033[33mWARNING: multiple onnxruntime packages are installed to the same location. "
+            "Please 'pip uninstall` all above packages, then `pip install` only one of them.\033[0m"
+        )
+
+    if cuda_version:
+        # Print version of installed packages that is related to CUDA or cuDNN DLLs.
+        cuda_major = _extract_cuda_major_version(cuda_version)
+
+        # Starting with CUDA 13, NVIDIA dropped the "-cuNN" suffix from the per-component
+        # CUDA Toolkit packages (cuDNN keeps its suffixed package name).
+        cuda_pkg_suffix = "" if (cuda_major.isdigit() and int(cuda_major) >= 13) else f"-cu{cuda_major}"
+
+        packages = [
+            "torch",
+            f"nvidia-cuda-runtime{cuda_pkg_suffix}",
+            f"nvidia-cudnn-cu{cuda_major}",
+            f"nvidia-cublas{cuda_pkg_suffix}",
+            f"nvidia-cufft{cuda_pkg_suffix}",
+            f"nvidia-curand{cuda_pkg_suffix}",
+            f"nvidia-cuda-nvrtc{cuda_pkg_suffix}",
+            f"nvidia-nvjitlink{cuda_pkg_suffix}",
+        ]
+        for package in packages:
+            directory_name = "nvidia" if package.startswith("nvidia-") else None
+            version = _get_package_version(package)
+            if version:
+                print(f"{package}=={version} at {_get_package_root(package, directory_name)}")
+            else:
+                print(f"{package} not installed")
+
+    if platform.system() == "Windows":
+        print(f"\nEnvironment variable:\nPATH={os.environ.get('PATH', '(unset)')}")
+    elif platform.system() == "Linux":
+        print(f"\nEnvironment variable:\nLD_LIBRARY_PATH={os.environ.get('LD_LIBRARY_PATH', '(unset)')}")
+
+    if importlib.util.find_spec("psutil"):
+
+        def is_target_dll(path: str):
+            target_keywords = ["vcruntime140", "msvcp140"]
+            if cuda_version:
+                target_keywords = ["cufft", "cublas", "cudart", "nvrtc", "curand", "cudnn", *target_keywords]
+            return any(keyword in path for keyword in target_keywords)
+
+        import psutil  # noqa: PLC0415
+
+        p = psutil.Process(os.getpid())
+
+        print("\nList of loaded DLLs:")
+        for lib in p.memory_maps():
+            if is_target_dll(lib.path.lower()):
+                print(lib.path)
+
+        if cuda_version:
+            if importlib.util.find_spec("cpuinfo") and importlib.util.find_spec("py3nvml"):
+                from .transformers.machine_info import get_device_info  # noqa: PLC0415
+
+                print("\nDevice information:")
+                print(get_device_info())
+            else:
+                print("please `pip install py-cpuinfo py3nvml` to show device information.")
+    else:
+        print("please `pip install psutil` to show loaded DLLs.")
+
+
+def preload_dlls(cuda: bool = True, cudnn: bool = True, msvc: bool = True, directory=None):
+    """Preload CUDA 12.x+ and optional cuDNN 9.x DLLs in Windows or Linux, and MSVC runtime DLLs in Windows.
+
+       When the installed PyTorch is compatible (using same major version of CUDA and cuDNN),
+       there is no need to call this function if `import torch` is done before `import onnxruntime`.
+
+    Args:
+        cuda (bool, optional): enable loading CUDA DLLs. Defaults to True.
+        cudnn (bool, optional): enable loading cuDNN DLLs. Defaults to True. Missing cuDNN DLLs are ignored.
+        msvc (bool, optional): enable loading MSVC DLLs in Windows. Defaults to True.
+        directory(str, optional): a directory contains CUDA or cuDNN DLLs. It can be an absolute path,
+           or a path relative to the directory of this file.
+           If directory is None (default value), the search order: the lib directory of compatible PyTorch in Windows,
+            nvidia site packages, default DLL loading paths.
+           If directory is empty string (""), the search order: nvidia site packages, default DLL loading paths.
+           If directory is a path, the search order: the directory, default DLL loading paths.
+    """
+    import ctypes  # noqa: PLC0415
+    import os  # noqa: PLC0415
+    import platform  # noqa: PLC0415
+    import sys  # noqa: PLC0415
+
+    if platform.system() not in ["Windows", "Linux"]:
+        return
+
+    is_windows = platform.system() == "Windows"
+    if is_windows and msvc:
+        try:
+            ctypes.CDLL("vcruntime140.dll")
+            ctypes.CDLL("msvcp140.dll")
+            if platform.machine() != "ARM64":
+                ctypes.CDLL("vcruntime140_1.dll")
+        except OSError:
+            print("Microsoft Visual C++ Redistributable is not installed, this may lead to the DLL load failure.")
+            print("It can be downloaded at https://aka.ms/vs/17/release/vc_redist.x64.exe.")
+
+    # Check if CUDA version is supported (12.x or 13.x+)
+    ort_cuda_major = None
+    if cuda_version:
+        try:
+            ort_cuda_major = int(cuda_version.split(".")[0])
+            if ort_cuda_major < 12 and (cuda or cudnn):
+                print(
+                    f"\033[33mWARNING: {package_name} is built with CUDA {cuda_version}, which is not supported for preloading. "
+                    f"CUDA 12.x or newer is required. Call preload_dlls with cuda=False and cudnn=False.\033[0m"
+                )
+                return
+        except ValueError:
+            print(
+                f"\033[33mWARNING: Unable to parse CUDA version '{cuda_version}'. "
+                "Skipping DLL preloading. Call preload_dlls with cuda=False and cudnn=False.\033[0m"
+            )
+            return
+    elif cuda or cudnn:
+        # No CUDA version info available but CUDA/cuDNN preloading requested
+        return
+
+    is_cuda_cudnn_imported_by_torch = False
+
+    if is_windows:
+        torch_version = _get_package_version("torch")
+        # Check if torch CUDA version matches onnxruntime CUDA version
+        torch_cuda_major = None
+        if torch_version and "+cu" in torch_version:
+            with contextlib.suppress(ValueError):
+                # Extract CUDA version from torch (e.g., "2.0.0+cu121" -> 12)
+                cu_part = torch_version.split("+cu")[1]
+                torch_cuda_major = int(cu_part[:2])  # First 2 digits are major version
+
+        is_torch_cuda_compatible = (
+            torch_cuda_major == ort_cuda_major if (torch_cuda_major and ort_cuda_major) else False
+        )
+
+        if "torch" in sys.modules:
+            is_cuda_cudnn_imported_by_torch = is_torch_cuda_compatible
+            if torch_cuda_major and ort_cuda_major and torch_cuda_major != ort_cuda_major:
+                print(
+                    f"\033[33mWARNING: The installed PyTorch {torch_version} uses CUDA {torch_cuda_major}.x, "
+                    f"but {package_name} is built with CUDA {ort_cuda_major}.x. "
+                    f"Please install PyTorch for CUDA {ort_cuda_major}.x to be compatible.\033[0m"
+                )
+
+        if is_torch_cuda_compatible and directory is None:
+            torch_root = _get_package_root("torch", "torch")
+            if torch_root:
+                directory = os.path.join(torch_root, "lib")
+
+    base_directory = directory or ".."
+    if not os.path.isabs(base_directory):
+        base_directory = os.path.join(os.path.dirname(__file__), base_directory)
+    base_directory = os.path.normpath(base_directory)
+    if not os.path.isdir(base_directory):
+        raise RuntimeError(f"Invalid parameter of directory={directory}. The directory does not exist!")
+
+    if is_cuda_cudnn_imported_by_torch:
+        # In Windows, PyTorch has loaded CUDA and cuDNN DLLs during `import torch`, no need to load them again.
+        print("Skip loading CUDA and cuDNN DLLs since torch is imported.")
+        return
+
+    # Try load DLLs from nvidia site packages.
+    dll_paths = _get_nvidia_dll_paths(is_windows, cuda, cudnn)
+    optional_dll_filenames = {relative_path[-1] for relative_path in _get_nvidia_dll_paths(is_windows, False, cudnn)}
+    loaded_dlls = []
+    for relative_path in dll_paths:
+        dll_path = (
+            os.path.join(base_directory, relative_path[-1])
+            if directory
+            else os.path.join(base_directory, *relative_path)
+        )
+        if os.path.isfile(dll_path):
+            try:
+                _ = ctypes.CDLL(dll_path)
+                loaded_dlls.append(relative_path[-1])
+            except Exception as e:
+                if relative_path[-1] not in optional_dll_filenames:
+                    print(f"Failed to load {dll_path}: {e}")
+
+    # Try load DLLs with default path settings.
+    has_failure = False
+    for relative_path in dll_paths:
+        dll_filename = relative_path[-1]
+        if dll_filename not in loaded_dlls:
+            try:
+                _ = ctypes.CDLL(dll_filename)
+            except Exception as e:
+                if dll_filename not in optional_dll_filenames:
+                    has_failure = True
+                    print(f"Failed to load {dll_filename}: {e}")
+
+    if has_failure:
+        print("Please follow https://onnxruntime.ai/docs/install/#cuda-and-cudnn to install CUDA.")
+
+    _register_bundled_cuda_plugin_ep(warn_on_failure=True)

onnxruntime/backend/__init__.py

@@ -0,0 +1,6 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+from .backend import is_compatible, prepare, run, supports_device  # noqa: F401

onnxruntime/backend/backend.py

@@ -0,0 +1,214 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""
+Implements ONNX's backend API.
+"""
+
+import os
+import unittest
+
+import packaging.version
+from onnx import ModelProto, helper, version  # noqa: F401
+from onnx.backend.base import Backend
+from onnx.checker import check_model
+
+from onnxruntime import InferenceSession, SessionOptions, get_available_providers, get_device
+from onnxruntime.backend.backend_rep import OnnxRuntimeBackendRep
+
+# Allowlist of SessionOptions attributes that are safe to set via the backend API.
+# Dangerous attributes intentionally excluded:
+#   optimized_model_filepath  — triggers Model::Save(), overwrites arbitrary files
+#   profile_file_prefix       — writes profiling JSON to arbitrary path
+#   enable_profiling          — causes uncontrolled file writes to cwd
+_ALLOWED_SESSION_OPTIONS = frozenset(
+    {
+        "enable_cpu_mem_arena",
+        "enable_mem_pattern",
+        "enable_mem_reuse",
+        "execution_mode",
+        "execution_order",
+        "graph_optimization_level",
+        "inter_op_num_threads",
+        "intra_op_num_threads",
+        "log_severity_level",
+        "log_verbosity_level",
+        "logid",
+        "use_deterministic_compute",
+        "use_per_session_threads",
+    }
+)
+
+
+class OnnxRuntimeBackend(Backend):
+    """
+    Implements
+    `ONNX's backend API <https://github.com/onnx/onnx/blob/main/docs/ImplementingAnOnnxBackend.md>`_
+    with *ONNX Runtime*.
+    The backend is mostly used when you need to switch between
+    multiple runtimes with the same API.
+    `Importing models from ONNX to Caffe2 <https://github.com/onnx/tutorials/blob/master/tutorials/OnnxCaffe2Import.ipynb>`_
+    shows how to use *caffe2* as a backend for a converted model.
+    Note: This is not the official Python API.
+    """
+
+    allowReleasedOpsetsOnly = bool(os.getenv("ALLOW_RELEASED_ONNX_OPSET_ONLY", "1") == "1")  # noqa: N815
+
+    @classmethod
+    def is_compatible(cls, model, device=None, **kwargs):
+        """
+        Return whether the model is compatible with the backend.
+
+        :param model: unused
+        :param device: None to use the default device or a string (ex: `'CPU'`)
+        :return: boolean
+        """
+        if device is None:
+            device = get_device()
+        return cls.supports_device(device)
+
+    @classmethod
+    def is_opset_supported(cls, model):
+        """
+        Return whether the opset for the model is supported by the backend.
+        When By default only released onnx opsets are allowed by the backend
+        To test new opsets env variable ALLOW_RELEASED_ONNX_OPSET_ONLY should be set to 0
+
+        :param model: Model whose opsets needed to be verified.
+        :return: boolean and error message if opset is not supported.
+        """
+        if cls.allowReleasedOpsetsOnly:
+            for opset in model.opset_import:
+                domain = opset.domain if opset.domain else "ai.onnx"
+                try:
+                    key = (domain, opset.version)
+                    if key not in helper.OP_SET_ID_VERSION_MAP:
+                        error_message = (
+                            "Skipping this test as only released onnx opsets are supported."
+                            "To run this test set env variable ALLOW_RELEASED_ONNX_OPSET_ONLY to 0."
+                            f" Got Domain '{domain}' version '{opset.version}'."
+                        )
+                        return False, error_message
+                except AttributeError:
+                    # for some CI pipelines accessing helper.OP_SET_ID_VERSION_MAP
+                    # is generating attribute error. TODO investigate the pipelines to
+                    # fix this error. Falling back to a simple version check when this error is encountered
+                    if (domain == "ai.onnx" and opset.version > 12) or (domain == "ai.ommx.ml" and opset.version > 2):
+                        error_message = (
+                            "Skipping this test as only released onnx opsets are supported."
+                            "To run this test set env variable ALLOW_RELEASED_ONNX_OPSET_ONLY to 0."
+                            f" Got Domain '{domain}' version '{opset.version}'."
+                        )
+                        return False, error_message
+        return True, ""
+
+    @classmethod
+    def supports_device(cls, device):
+        """
+        Check whether the backend is compiled with particular device support.
+        In particular it's used in the testing suite.
+        """
+        if device == "CUDA":
+            device = "GPU"
+        return "-" + device in get_device() or device + "-" in get_device() or device == get_device()
+
+    @classmethod
+    def prepare(cls, model, device=None, **kwargs):
+        """
+        Load the model and creates an :class:`onnxruntime.backend.backend_rep.OnnxRuntimeBackendRep`
+        ready to be used as a backend.
+
+        :param model: the model to prepare — accepts a file path (str), serialized
+            model (bytes), :class:`onnx.ModelProto`, :class:`onnxruntime.InferenceSession`,
+            or :class:`onnxruntime.backend.backend_rep.OnnxRuntimeBackendRep` (returned as-is)
+        :param device: requested device for the computation,
+            None means the default one which depends on
+            the compilation settings
+        :param kwargs: only a safe subset of :class:`onnxruntime.SessionOptions` attributes are
+            accepted; see ``_ALLOWED_SESSION_OPTIONS`` for the list
+        :return: :class:`onnxruntime.backend.backend_rep.OnnxRuntimeBackendRep`
+        """
+        if isinstance(model, OnnxRuntimeBackendRep):
+            return model
+        elif isinstance(model, InferenceSession):
+            return OnnxRuntimeBackendRep(model)
+        elif isinstance(model, (str, bytes)):
+            options = SessionOptions()
+            for k, v in kwargs.items():
+                if k in _ALLOWED_SESSION_OPTIONS:
+                    setattr(options, k, v)
+                elif hasattr(options, k):
+                    raise RuntimeError(
+                        f"SessionOptions attribute '{k}' is not permitted via the backend API. "
+                        f"Allowed attributes: {', '.join(sorted(_ALLOWED_SESSION_OPTIONS))}"
+                    )
+                # else: silently ignore unknown keys
+
+            excluded_providers = os.getenv("ORT_ONNX_BACKEND_EXCLUDE_PROVIDERS", default="").split(",")
+            providers = [x for x in get_available_providers() if (x not in excluded_providers)]
+
+            inf = InferenceSession(model, sess_options=options, providers=providers)
+            # backend API is primarily used for ONNX test/validation. As such, we should disable session.run() fallback
+            # which may hide test failures.
+            inf.disable_fallback()
+            if device is not None and not cls.supports_device(device):
+                raise RuntimeError(f"Incompatible device expected '{device}', got '{get_device()}'")
+            return cls.prepare(inf, device, **kwargs)
+        else:
+            # type: ModelProto
+            # check_model serializes the model anyways, so serialize the model once here
+            # and reuse it below in the cls.prepare call to avoid an additional serialization
+            # only works with onnx >= 1.10.0 hence the version check
+            onnx_version = packaging.version.parse(version.version) or packaging.version.Version("0")
+            onnx_supports_serialized_model_check = onnx_version.release >= (1, 10, 0)
+            bin_or_model = model.SerializeToString() if onnx_supports_serialized_model_check else model
+            check_model(bin_or_model)
+            opset_supported, error_message = cls.is_opset_supported(model)
+            if not opset_supported:
+                raise unittest.SkipTest(error_message)
+            # Now bin might be serialized, if it's not we need to serialize it otherwise we'll have
+            # an infinite recursive call
+            bin = bin_or_model
+            if not isinstance(bin, (str, bytes)):
+                bin = bin.SerializeToString()
+            return cls.prepare(bin, device, **kwargs)
+
+    @classmethod
+    def run_model(cls, model, inputs, device=None, **kwargs):
+        """
+        Compute the prediction.
+
+        :param model: the model to run — accepts a file path (str), serialized
+            model (bytes), :class:`onnx.ModelProto`, :class:`onnxruntime.InferenceSession`,
+            or :class:`onnxruntime.backend.backend_rep.OnnxRuntimeBackendRep`
+        :param inputs: inputs
+        :param device: requested device for the computation,
+            None means the default one which depends on
+            the compilation settings
+        :param kwargs: ``run_model()`` forwards kwargs to both ``prepare()`` and ``rep.run()``.
+            ``prepare()`` validates and applies ``_ALLOWED_SESSION_OPTIONS`` only when creating
+            a new session from a model path or bytes; if ``model`` is already an
+            ``InferenceSession`` or ``OnnxRuntimeBackendRep``, session-option kwargs are
+            silently ignored. ``rep.run()`` always validates against ``_ALLOWED_RUN_OPTIONS``
+            and raises ``RuntimeError`` for known-but-blocked run attributes.
+            Logging-related kwargs (``log_severity_level``, ``log_verbosity_level``, ``logid``)
+            appear in both allowlists.
+        :return: predictions
+        """
+        rep = cls.prepare(model, device, **kwargs)
+        return rep.run(inputs, **kwargs)
+
+    @classmethod
+    def run_node(cls, node, inputs, device=None, outputs_info=None, **kwargs):
+        """
+        This method is not implemented as it is much more efficient
+        to run a whole model than every node independently.
+        """
+        raise NotImplementedError("It is much more efficient to run a whole model than every node independently.")
+
+
+is_compatible = OnnxRuntimeBackend.is_compatible
+prepare = OnnxRuntimeBackend.prepare
+run = OnnxRuntimeBackend.run_model
+supports_device = OnnxRuntimeBackend.supports_device

onnxruntime/backend/backend_rep.py

@@ -0,0 +1,76 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""
+Implements ONNX's backend API.
+"""
+
+from onnx.backend.base import BackendRep
+
+from onnxruntime import RunOptions
+
+# Allowlist of RunOptions attributes that are safe to set via the backend API.
+# 'terminate' excluded: setting it True would deny the current inference call.
+# 'training_mode' excluded: silently switches inference behavior in training builds.
+_ALLOWED_RUN_OPTIONS = frozenset(
+    {
+        "log_severity_level",
+        "log_verbosity_level",
+        "logid",
+        "only_execute_path_to_fetches",
+    }
+)
+
+
+class OnnxRuntimeBackendRep(BackendRep):
+    """
+    Wraps an :class:`onnxruntime.InferenceSession` to implement ONNX's
+    :class:`onnx.backend.base.BackendRep` interface for running predictions.
+    """
+
+    def __init__(self, session):
+        """
+        :param session: :class:`onnxruntime.InferenceSession`
+        """
+        self._session = session
+
+    def run(self, inputs, **kwargs):  # type: (Any, **Any) -> Tuple[Any, ...]
+        """
+        Computes the prediction.
+        See :meth:`onnxruntime.InferenceSession.run`.
+
+        :param inputs: a list of input arrays (one per model input) or a single
+            array when the model has exactly one input
+        :param kwargs: only a safe subset of :class:`onnxruntime.RunOptions` attributes are
+            accepted; see ``_ALLOWED_RUN_OPTIONS`` for the list
+        :return: list of output arrays
+        """
+
+        options = RunOptions()
+        for k, v in kwargs.items():
+            if k in _ALLOWED_RUN_OPTIONS:
+                setattr(options, k, v)
+            elif hasattr(options, k):
+                raise RuntimeError(
+                    f"RunOptions attribute '{k}' is not permitted via the backend API. "
+                    f"Allowed attributes: {', '.join(sorted(_ALLOWED_RUN_OPTIONS))}"
+                )
+            # else: silently ignore unknown keys
+
+        if isinstance(inputs, list):
+            inps = {}
+            for i, inp in enumerate(self._session.get_inputs()):
+                inps[inp.name] = inputs[i]
+            outs = self._session.run(None, inps, options)
+            if isinstance(outs, list):
+                return outs
+            else:
+                output_names = [o.name for o in self._session.get_outputs()]
+                return [outs[name] for name in output_names]
+        else:
+            inp = self._session.get_inputs()
+            if len(inp) != 1:
+                raise RuntimeError(f"Model expect {len(inp)} inputs")
+            inps = {inp[0].name: inputs}
+            return self._session.run(None, inps, options)

onnxruntime/capi/DirectML.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c9e6d822561c6c41b90e6994b3e8857cf1d66dbfb1e0c4c799c7c89b4e92da1
+size 18527776

onnxruntime/capi/__init__.py

@@ -0,0 +1,4 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------

onnxruntime/capi/_ld_preload.py

@@ -0,0 +1,7 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+
+# This file can be modified by setup.py when building a manylinux2010 wheel
+# When modified, it will preload some libraries needed for the python C extension

onnxruntime/capi/_pybind_state.py

@@ -0,0 +1,43 @@
+# --- UORT bundle ------------------------------------------------------------
+# Registers the directory of this module with the Windows loader before the
+# pybind11 extension is imported. Added by scripts\runtime\stage_python_bundle.ps1.
+try:
+    from ._uort_dll_dirs import add_dll_dirs as _uort_add_dll_dirs
+
+    _uort_add_dll_dirs()
+except Exception:  # noqa: BLE001 - a plain package layout has no _uort_dll_dirs
+    pass
+# ----------------------------------------------------------------------------
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""
+Ensure that dependencies are available and then load the extension module.
+"""
+import os
+import platform
+import warnings
+
+from . import _ld_preload  # noqa: F401
+
+if platform.system() == "Windows":
+    from . import version_info
+
+    # If on Windows, check if this import error is caused by the user not installing the 2019 VC Runtime
+    # The VC Redist installer usually puts the VC Runtime dlls in the System32 folder, but it may also be found
+    # in some other locations.
+    # TODO, we may want to try to load the VC Runtime dlls instead of checking if the hardcoded file path
+    # is valid, and raise ImportError if the load fails
+    if version_info.vs2019 and platform.architecture()[0] == "64bit":
+        system_root = os.getenv("SystemRoot") or "C:\\Windows"
+        if not os.path.isfile(os.path.join(system_root, "System32", "vcruntime140_1.dll")):
+            warnings.warn("Please install the 2019 Visual C++ runtime and then try again. "
+                          "If you've installed the runtime in a non-standard location "
+                          "(other than %SystemRoot%\\System32), "
+                          "make sure it can be found by setting the correct path.")
+
+
+
+from .onnxruntime_pybind11_state import *  # noqa
+

onnxruntime/capi/_uort_dll_dirs.py

@@ -0,0 +1,32 @@
+"""UORT: make the DLLs bundled next to this module visible to the Windows loader.
+
+The bundle keeps every native dependency of the pybind11 module (CUDA, cuDNN,
+TensorRT, OpenVINO) in the same directory as the module. CPython already
+searches that directory for the direct imports of the extension module, but
+cuDNN, cuFFT, TensorRT and OpenVINO resolve several libraries themselves with
+plain LoadLibrary, which only looks at the process directory and PATH.
+Registering the directory both ways covers all of them: AddDllDirectory works
+for LoadLibraryEx with LOAD_LIBRARY_SEARCH_USER_DIRS, PATH for plain
+LoadLibrary.
+"""
+
+from __future__ import annotations
+
+import os
+
+_DIR = os.path.dirname(os.path.abspath(__file__))
+
+
+def add_dll_dirs() -> str:
+    """Register the bundle directory; returns the directory that was added."""
+    if os.name != "nt":
+        return _DIR
+    try:
+        os.add_dll_directory(_DIR)
+    except (AttributeError, OSError):
+        pass
+    current = os.environ.get("PATH", "")
+    parts = [p for p in current.split(os.pathsep) if p]
+    if _DIR not in parts:
+        os.environ["PATH"] = _DIR + os.pathsep + current
+    return _DIR

onnxruntime/capi/build_and_package_info.py

@@ -0,0 +1,3 @@
+package_name = 'uort-python'
+__version__ = '1.30.1'
+cuda_version = '12.9'

onnxruntime/capi/concrt140.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2405355f0a58067b258f8df33c327e3a3d716eaac5a3a5aebb757842d85bd376
+size 324208

onnxruntime/capi/convert_npz_to_onnx_adapter.py

@@ -0,0 +1,48 @@
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+
+# This script helps converting .npz files to .onnx_adapter files
+
+import argparse
+import os
+import sys
+
+import numpy as np
+
+import onnxruntime as ort
+
+
+def get_args() -> argparse:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--npz_file_path", type=str, required=True)
+    parser.add_argument("--output_file_path", type=str, required=True)
+    parser.add_argument("--adapter_version", type=int, required=True)
+    parser.add_argument("--model_version", type=int, required=True)
+    return parser.parse_args()
+
+
+def export_lora_parameters(
+    npz_file_path: os.PathLike, adapter_version: int, model_version: int, output_file_path: os.PathLike
+):
+    """The function converts lora parameters in npz to onnx_adapter format"""
+    adapter_format = ort.AdapterFormat()
+    adapter_format.set_adapter_version(adapter_version)
+    adapter_format.set_model_version(model_version)
+    name_to_ort_value = {}
+    with np.load(npz_file_path) as data:
+        for name, np_arr in data.items():
+            ort_value = ort.OrtValue.ortvalue_from_numpy(np_arr)
+            name_to_ort_value[name] = ort_value
+
+    adapter_format.set_parameters(name_to_ort_value)
+    adapter_format.export_adapter(output_file_path)
+
+
+def main() -> int:
+    args = get_args()
+    export_lora_parameters(args.npz_file_path, args.adapter_version, args.model_version, args.output_file_path)
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

onnxruntime/capi/cublas64_12.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:90052a83efd1b57a8e3616a6590b335855f81b814a4f16eecb7b5bf6d1b1d4eb
+size 102518272

onnxruntime/capi/cublasLt64_12.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c3a05ea244c937314afec09f87b91f814c7e27977681f6c67eb51bb06ced3a4a
+size 668669952

onnxruntime/capi/cudart64_12.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:760c38928bbe5759f7b31ed6692599eb7ec83cedd5702e84c2b72028a89837e1
+size 583680

onnxruntime/capi/cudnn64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9edbcdff73b0af070eb160b2ce66e59feca04aa017351d8eedcc5e8e149967d2
+size 266288

onnxruntime/capi/cudnn_adv64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f202e8ff36209ac4292fe2ef79d4539487495b3b948f0c978cc63298f43a1850
+size 282445872

onnxruntime/capi/cudnn_cnn64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3f03dec3b81f137f049e1d1c06ff469248eef54ff9424b3b9045fc0f69559d1a
+size 4618272

onnxruntime/capi/cudnn_engines_precompiled64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ebc2c1f74366b029a4350f7f9a3b109ccd5aa9b4ae02527950793dd29f7b74fb
+size 513926688

onnxruntime/capi/cudnn_engines_runtime_compiled64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:93ab6351ca0f2b6ca842f83d284e56c427525778c14657229801a13cf8121bd8
+size 20201008

onnxruntime/capi/cudnn_graph64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a933efeb280a3252297f92d79f7e83003bcbc6f665872d11645d1dbbab9cb598
+size 2420256

onnxruntime/capi/cudnn_heuristic64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:458175f12771a35de0ea39c04250d727c2424d42a7ffe0bc987ae81204748fba
+size 56823328

onnxruntime/capi/cudnn_ops64_9.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:851d8f4191a24b322b0be738d191a3159455f0da4db47a19f5cc7ad6e55edf61
+size 126508576

onnxruntime/capi/cufft64_11.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5b90655ca9cdf5d91cbdf2c72b995ec937eed0aef2d6954370c711b45c929b35
+size 287136768

onnxruntime/capi/curand64_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4d6d16c9941c4f7a7e49d5c43df25d408a2a8503d8cc1eb9808e0e0c2cde36d4
+size 79197696

onnxruntime/capi/msvcp140.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0f885b509a685d2bbfa652fed26b5fb31d88fbdab0a978c641d1c7b8aa460aa9
+size 557728

onnxruntime/capi/msvcp140_2.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ea06f0ee098b4823cb79599df3780e7f23cce52c19aac31d2a0d47efe33a5e9
+size 280200

onnxruntime/capi/nvinfer_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:55c7ec9272a4b74830d3caaae9106f277fc7df37690067f69fe1fdfd1dbc52ff
+size 440983072

onnxruntime/capi/nvinfer_builder_resource_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2b8496c57a8434534e8a3c56e732207f1c74bc37ce7300d8f0d3542cdff02e12
+size 1971797024

onnxruntime/capi/nvinfer_dispatch_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4100a0e46f96088a3e79664c1d719beb462ce1f067b4cadf24feeebc7dd693e1
+size 624672

onnxruntime/capi/nvinfer_plugin_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:420aee53c0c2b4b35e76335daba57e0cabd7e22f2b946e47ed77dfb3019ca5a9
+size 51002400

onnxruntime/capi/nvonnxparser_10.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d4ca07f3f392f6f39f1738ed5e615910448d1f98b657551a01c7575881584a2a
+size 2993696

onnxruntime/capi/onnxruntime.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a4599b0168c8b3e485e50c4493559280778f7c533d66bbb3ea372073f316b67f
+size 24535040

onnxruntime/capi/onnxruntime_collect_build_info.py

@@ -0,0 +1,47 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+import ctypes
+import sys
+import warnings
+
+
+def find_cudart_versions(build_env=False, build_cuda_version=None):
+    # ctypes.CDLL and ctypes.util.find_library load the latest installed library.
+    # it may not the the library that would be loaded by onnxruntime.
+    # for example, in an environment with Cuda 11.1 and subsequently
+    # conda cudatoolkit 10.2.89 installed. ctypes will find cudart 10.2. however,
+    # onnxruntime built with Cuda 11.1 will find and load cudart for Cuda 11.1.
+    # for the above reason, we need find all versions in the environment and
+    # only give warnings if the expected cuda version is not found.
+    # in onnxruntime build environment, we expected only one Cuda version.
+    if not sys.platform.startswith("linux"):
+        warnings.warn("find_cudart_versions only works on Linux")
+        return None
+
+    cudart_possible_versions = {None, build_cuda_version}
+
+    def get_cudart_version(find_cudart_version=None):
+        cudart_lib_filename = "libcudart.so"
+        if find_cudart_version:
+            cudart_lib_filename = cudart_lib_filename + "." + find_cudart_version
+
+        try:
+            cudart = ctypes.CDLL(cudart_lib_filename)
+            cudart.cudaRuntimeGetVersion.restype = int
+            cudart.cudaRuntimeGetVersion.argtypes = [ctypes.POINTER(ctypes.c_int)]
+            version = ctypes.c_int()
+            status = cudart.cudaRuntimeGetVersion(ctypes.byref(version))
+            if status != 0:
+                return None
+        except Exception:
+            return None
+
+        return version.value
+
+    # use set to avoid duplications
+    cudart_found_versions = {get_cudart_version(cudart_version) for cudart_version in cudart_possible_versions}
+
+    # convert to list and remove None
+    return [ver for ver in cudart_found_versions if ver]

onnxruntime/capi/onnxruntime_inference_collection.py

@@ -0,0 +1,1930 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+from __future__ import annotations
+
+import collections
+import collections.abc
+import os
+import typing
+import warnings
+import weakref
+from collections.abc import Callable, Sequence
+from enum import IntEnum
+from typing import Any
+
+import numpy as np
+
+from onnxruntime.capi import _pybind_state as C
+
+if typing.TYPE_CHECKING:
+    import numpy.typing as npt
+
+
+def get_ort_device_type(device_type: str) -> int:
+    if device_type == "cuda":
+        return C.OrtDevice.cuda()
+    elif device_type == "cann":
+        return C.OrtDevice.cann()
+    elif device_type == "cpu":
+        return C.OrtDevice.cpu()
+    elif device_type == "dml":
+        return C.OrtDevice.dml()
+    elif device_type == "webgpu":
+        return C.OrtDevice.webgpu()
+    elif device_type == "gpu":
+        return C.OrtDevice.gpu()
+    elif device_type == "npu":
+        return C.OrtDevice.npu()
+    else:
+        raise Exception("Unsupported device type: " + device_type)
+
+
+class OrtDeviceVendorId(IntEnum):
+    """Vendor IDs aligned with OrtDevice::VendorIds in ortdevice.h."""
+
+    NONE = 0x0000
+    AMD = 0x1002
+    NVIDIA = 0x10DE
+    ARM = 0x13B5
+    MICROSOFT = 0x1414
+    HUAWEI = 0x19E5
+    QUALCOMM = 0x5143
+    INTEL = 0x8086
+
+
+def get_vendor_id_for_device_type(device_type: str) -> OrtDeviceVendorId | None:
+    if device_type == "cuda":
+        return OrtDeviceVendorId.NVIDIA
+    elif device_type == "dml":
+        return OrtDeviceVendorId.MICROSOFT
+    elif device_type == "cann":
+        return OrtDeviceVendorId.HUAWEI
+    elif device_type == "webgpu":
+        return OrtDeviceVendorId.NONE
+    else:
+        return None
+
+
+_GPU_GRAPH_ID_RUN_CONFIG_KEY = "gpu_graph_id"
+# Mirrors InferenceSession::kGraphAnnotationSkip; core skips capture and replay for this ID.
+_GRAPH_ANNOTATION_SKIP = -1
+
+
+def _graph_annotation_id(run_options) -> int:
+    """Return the effective ``gpu_graph_id`` for a run, defaulting to 0 when unset."""
+    if run_options is None:
+        return 0
+    try:
+        entry = run_options.get_run_config_entry(_GPU_GRAPH_ID_RUN_CONFIG_KEY)
+    except RuntimeError:
+        return 0
+    if not entry:
+        return 0
+    try:
+        return int(entry)
+    except ValueError:
+        raise ValueError(f"Run option '{_GPU_GRAPH_ID_RUN_CONFIG_KEY}' must be an integer, got {entry!r}.") from None
+
+
+def _is_ortvalue_session_compatible(ortvalue, target_session) -> bool:
+    """Whether ``target_session`` can share ``ortvalue``'s buffer.
+
+    Sessionless values come from a shared allocator. A session-owned WebGPU buffer is
+    usable by any session on the same WebGPU context.
+    """
+    if ortvalue._session is None or target_session is None or ortvalue._session is target_session:
+        return True
+    return ortvalue._is_webgpu_buffer and ortvalue._session.webgpu_context_id() == target_session.webgpu_context_id()
+
+
+def _validate_ortvalue_session_compatibility(ortvalue, target_session, action) -> None:
+    """Reject an OrtValue that the target session cannot share buffers with.
+
+    ``action`` is the verb used in the message, e.g. ``"used with"`` or ``"bound to"``.
+    """
+    if not _is_ortvalue_session_compatible(ortvalue, target_session):
+        raise ValueError(f"Session-scoped OrtValue must be {action} the session that created it.")
+
+
+class AdapterFormat:
+    """
+    This class is used to create adapter files from python structures
+    """
+
+    def __init__(self, adapter=None) -> None:
+        if adapter is None:
+            self._adapter = C.AdapterFormat()
+        else:
+            self._adapter = adapter
+
+    @staticmethod
+    def read_adapter(file_path: os.PathLike) -> AdapterFormat:
+        return AdapterFormat(C.AdapterFormat.read_adapter(file_path))
+
+    def export_adapter(self, file_path: os.PathLike):
+        """
+        This function writes a file at the specified location
+        in onnxrunitme adapter format containing Lora parameters.
+
+        :param file_path: absolute path for the adapter
+        """
+        self._adapter.export_adapter(file_path)
+
+    def get_format_version(self) -> int:
+        return self._adapter.format_version
+
+    def set_adapter_version(self, adapter_version: int) -> None:
+        self._adapter.adapter_version = adapter_version
+
+    def get_adapter_version(self) -> int:
+        return self._adapter.adapter_version
+
+    def set_model_version(self, model_version: int) -> None:
+        self._adapter.model_version = model_version
+
+    def get_model_version(self) -> int:
+        return self._adapter.model_version
+
+    def set_parameters(self, params: dict[str, OrtValue]) -> None:
+        """Set adapter parameters for export."""
+        self._adapter.parameters = {k: v._ortvalue for k, v in params.items()}
+
+    def get_parameters(self) -> dict[str, OrtValue]:
+        """Get adapter parameters as a dict of name -> OrtValue.
+
+        On read instances, the returned OrtValues are zero-copy views; the
+        backing memory stays alive as long as any returned OrtValue is referenced.
+        """
+        return {k: OrtValue(v) for k, v in self._adapter.parameters.items()}
+
+
+def check_and_normalize_provider_args(
+    providers: Sequence[str | tuple[str, dict[Any, Any]]] | None,
+    provider_options: Sequence[dict[Any, Any]] | None,
+    available_provider_names: Sequence[str],
+):
+    """
+    Validates the 'providers' and 'provider_options' arguments and returns a
+        normalized version.
+
+    :param providers: Optional sequence of providers in order of decreasing
+        precedence. Values can either be provider names or tuples of
+        (provider name, options dict).
+    :param provider_options: Optional sequence of options dicts corresponding
+        to the providers listed in 'providers'.
+    :param available_provider_names: The available provider names.
+
+    :return: Tuple of (normalized 'providers' sequence, normalized
+        'provider_options' sequence).
+
+    'providers' can contain either names or names and options. When any options
+        are given in 'providers', 'provider_options' should not be used.
+
+    The normalized result is a tuple of:
+    1. Sequence of provider names in the same order as 'providers'.
+    2. Sequence of corresponding provider options dicts with string keys and
+        values. Unspecified provider options yield empty dicts.
+    """
+    if providers is None:
+        return [], []
+
+    provider_name_to_options = collections.OrderedDict()
+
+    def set_provider_options(name, options):
+        if name not in available_provider_names:
+            warnings.warn(
+                "Specified provider '{}' is not in available provider names.Available providers: '{}'".format(
+                    name, ", ".join(available_provider_names)
+                )
+            )
+
+        if name in provider_name_to_options:
+            warnings.warn(f"Duplicate provider '{name}' encountered, ignoring.")
+            return
+
+        normalized_options = {str(key): str(value) for key, value in options.items()}
+        provider_name_to_options[name] = normalized_options
+
+    if not isinstance(providers, collections.abc.Sequence):
+        raise ValueError("'providers' should be a sequence.")
+
+    if provider_options is not None:
+        if not isinstance(provider_options, collections.abc.Sequence):
+            raise ValueError("'provider_options' should be a sequence.")
+
+        if len(providers) != len(provider_options):
+            raise ValueError("'providers' and 'provider_options' should be the same length if both are given.")
+
+        if not all(isinstance(provider, str) for provider in providers):
+            raise ValueError("Only string values for 'providers' are supported if 'provider_options' is given.")
+
+        if not all(isinstance(options_for_provider, dict) for options_for_provider in provider_options):
+            raise ValueError("'provider_options' values must be dicts.")
+
+        for name, options in zip(providers, provider_options, strict=False):
+            set_provider_options(name, options)
+
+    else:
+        for provider in providers:
+            if isinstance(provider, str):
+                set_provider_options(provider, {})
+            elif (
+                isinstance(provider, tuple)
+                and len(provider) == 2
+                and isinstance(provider[0], str)
+                and isinstance(provider[1], dict)
+            ):
+                set_provider_options(provider[0], provider[1])
+            else:
+                raise ValueError("'providers' values must be either strings or (string, dict) tuples.")
+
+    return list(provider_name_to_options.keys()), list(provider_name_to_options.values())
+
+
+class Session:
+    """
+    This is the main class used to run a model.
+    """
+
+    def __init__(self, enable_fallback: bool = True):
+        # self._sess is managed by the derived class and relies on bindings from C.InferenceSession
+        self._sess = None
+        self._enable_fallback = enable_fallback
+        # Captured graphs retain buffer signatures without extending the IOBinding lifetime.
+        self._captured_graph_bindings: dict[int, tuple[weakref.ref, tuple]] = {}
+
+    def get_session_options(self) -> C.SessionOptions:
+        "Return the session options. See :class:`onnxruntime.SessionOptions`."
+        return self._sess_options
+
+    def get_inputs(self) -> Sequence[C.NodeArg]:
+        "Return the inputs metadata as a list of :class:`onnxruntime.NodeArg`."
+        return self._inputs_meta
+
+    def get_outputs(self) -> Sequence[C.NodeArg]:
+        "Return the outputs metadata as a list of :class:`onnxruntime.NodeArg`."
+        return self._outputs_meta
+
+    def get_overridable_initializers(self) -> Sequence[C.NodeArg]:
+        "Return the inputs (including initializers) metadata as a list of :class:`onnxruntime.NodeArg`."
+        return self._overridable_initializers
+
+    def get_modelmeta(self) -> C.ModelMetadata:
+        "Return the metadata. See :class:`onnxruntime.ModelMetadata`."
+        return self._model_meta
+
+    def get_input_memory_infos(self) -> Sequence[C.OrtMemoryInfo]:
+        "Return the memory info for the inputs."
+        return self._input_meminfos
+
+    def get_output_memory_infos(self) -> Sequence[C.OrtMemoryInfo]:
+        "Return the memory info for the outputs."
+        return self._output_meminfos
+
+    def get_input_epdevices(self) -> Sequence[C.OrtEpDevice]:
+        "Return the execution providers for the inputs."
+        return self._input_epdevices
+
+    def get_providers(self) -> Sequence[str]:
+        "Return list of registered execution providers."
+        return self._providers
+
+    def get_provider_options(self):
+        "Return registered execution providers' configurations."
+        return self._provider_options
+
+    def get_provider_graph_assignment_info(self) -> Sequence[C.OrtEpAssignedSubgraph]:
+        """
+        Get information about the subgraphs assigned to each execution provider and the nodes within.
+
+        Application must enable the recording of graph assignment information by setting the session configuration
+        for the key "session.record_ep_graph_assignment_info" to "1".
+        """
+        return self._sess.get_provider_graph_assignment_info()
+
+    def set_providers(self, providers=None, provider_options=None) -> None:
+        """
+        Register the input list of execution providers. The underlying session is re-created.
+
+        :param providers: Optional sequence of providers in order of decreasing
+            precedence. Values can either be provider names or tuples of
+            (provider name, options dict). If not provided, then all available
+            providers are used with the default precedence.
+        :param provider_options: Optional sequence of options dicts corresponding
+            to the providers listed in 'providers'.
+
+        'providers' can contain either names or names and options. When any options
+        are given in 'providers', 'provider_options' should not be used.
+
+        The list of providers is ordered by precedence. For example
+        `['CUDAExecutionProvider', 'CPUExecutionProvider']`
+        means execute a node using CUDAExecutionProvider if capable,
+        otherwise execute using CPUExecutionProvider.
+        """
+        # recreate the underlying C.InferenceSession
+        self._reset_session(providers, provider_options)
+
+    def disable_fallback(self) -> None:
+        """
+        Disable session.run() fallback mechanism.
+        """
+        self._enable_fallback = False
+
+    def enable_fallback(self) -> None:
+        """
+        Enable session.Run() fallback mechanism. If session.Run() fails due to an internal Execution Provider failure,
+        reset the Execution Providers enabled for this session.
+        If GPU is enabled, fall back to CUDAExecutionProvider.
+        otherwise fall back to CPUExecutionProvider.
+        """
+        self._enable_fallback = True
+
+    def _validate_input(self, feed_input_names):
+        missing_input_names = []
+        for input in self._inputs_meta:
+            if input.name not in feed_input_names and not input.type.startswith("optional"):
+                missing_input_names.append(input.name)
+        if missing_input_names:
+            raise ValueError(
+                f"Required inputs ({missing_input_names}) are missing from input feed ({feed_input_names})."
+            )
+
+    def _validate_ortvalue_ownership(self, values):
+        for value in values:
+            if not isinstance(value, OrtValue):
+                continue
+            _validate_ortvalue_session_compatibility(value, self._sess, "used with")
+
+    def _validate_graph_capture_run_api(self, run_options=None):
+        if not self._sess.is_webgpu_graph_capture_enabled():
+            return
+        if _graph_annotation_id(run_options) == _GRAPH_ANNOTATION_SKIP:
+            # gpu_graph_id=-1 skips capture, so transient feeds remain valid.
+            return
+        raise ValueError(
+            "WebGPU graph capture requires fixed device OrtValues and run_with_iobinding. "
+            f"Set the '{_GPU_GRAPH_ID_RUN_CONFIG_KEY}' run option to "
+            f"{_GRAPH_ANNOTATION_SKIP} to opt a single run out of capture."
+        )
+
+    def run(self, output_names, input_feed, run_options=None) -> Sequence[np.ndarray | SparseTensor | list | dict]:
+        """
+        Compute the predictions.
+
+        WebGPU graph capture requires fixed device OrtValues bound with
+        :meth:`run_with_iobinding`; do not use this convenience API for captured replay.
+
+        :param output_names: name of the outputs
+        :param input_feed: dictionary ``{ input_name: input_value }``
+        :param run_options: See :class:`onnxruntime.RunOptions`.
+        :return: list of results, every result is either a numpy array,
+            a sparse tensor, a list or a dictionary.
+
+        ::
+
+            sess.run([output_name], {input_name: x})
+        """
+        self._validate_graph_capture_run_api(run_options)
+        self._validate_input(list(input_feed.keys()))
+        self._validate_ortvalue_ownership(input_feed.values())
+        if not output_names:
+            output_names = [output.name for output in self._outputs_meta]
+        try:
+            return self._sess.run(output_names, input_feed, run_options)
+        except C.EPFail as err:
+            if self._enable_fallback:
+                print(f"EP Error: {err!s} using {self._providers}")
+                print(f"Falling back to {self._fallback_providers} and retrying.")
+                self.set_providers(self._fallback_providers)
+                # Fallback only once.
+                self.disable_fallback()
+                self._validate_ortvalue_ownership(input_feed.values())
+                return self._sess.run(output_names, input_feed, run_options)
+            raise
+
+    def run_async(self, output_names, input_feed, callback, user_data, run_options=None):
+        """
+        Compute the predictions asynchronously in a separate cxx thread from ort intra-op threadpool.
+
+        :param output_names: name of the outputs
+        :param input_feed: dictionary ``{ input_name: input_value }``
+        :param callback: python function that accept array of results, and a status string on error.
+            The callback will be invoked by a cxx thread from ort intra-op threadpool.
+        :param run_options: See :class:`onnxruntime.RunOptions`.
+
+        ::
+            class MyData:
+                def __init__(self):
+                    # ...
+                def save_results(self, results):
+                    # ...
+
+            def callback(results: np.ndarray, user_data: MyData, err: str) -> None:
+              if err:
+                 print (err)
+              else:
+                # save results to user_data
+
+            sess.run_async([output_name], {input_name: x}, callback)
+        """
+        self._validate_graph_capture_run_api(run_options)
+        self._validate_input(list(input_feed.keys()))
+        self._validate_ortvalue_ownership(input_feed.values())
+        if not output_names:
+            output_names = [output.name for output in self._outputs_meta]
+        return self._sess.run_async(output_names, input_feed, callback, user_data, run_options)
+
+    def run_with_ort_values(self, output_names, input_dict_ort_values, run_options=None) -> Sequence[OrtValue]:
+        """
+        Compute the predictions.
+
+        :param output_names: name of the outputs
+        :param input_dict_ort_values: dictionary ``{ input_name: input_ort_value }``
+            See ``OrtValue`` class how to create `OrtValue`
+            from numpy array or `SparseTensor`
+        :param run_options: See :class:`onnxruntime.RunOptions`.
+        :return: an array of `OrtValue`
+
+        ::
+
+            sess.run([output_name], {input_name: x})
+        """
+
+        def invoke(sess, output_names, input_dict_ort_values, run_options):
+            input_dict = {}
+            for n, v in input_dict_ort_values.items():
+                input_dict[n] = v._get_c_value()
+            result = sess.run_with_ort_values(input_dict, output_names, run_options)
+            if not isinstance(result, C.OrtValueVector):
+                raise TypeError("run_with_ort_values() must return a instance of type 'OrtValueVector'.")
+            ort_values = [OrtValue(v) for v in result]
+            return ort_values
+
+        self._validate_graph_capture_run_api(run_options)
+        self._validate_input(list(input_dict_ort_values.keys()))
+        self._validate_ortvalue_ownership(input_dict_ort_values.values())
+        if not output_names:
+            output_names = [output.name for output in self._outputs_meta]
+        try:
+            return invoke(self._sess, output_names, input_dict_ort_values, run_options)
+        except C.EPFail as err:
+            if self._enable_fallback:
+                print(f"EP Error: {err!s} using {self._providers}")
+                print(f"Falling back to {self._fallback_providers} and retrying.")
+                self.set_providers(self._fallback_providers)
+                # Fallback only once.
+                self.disable_fallback()
+                self._validate_ortvalue_ownership(input_dict_ort_values.values())
+                return invoke(self._sess, output_names, input_dict_ort_values, run_options)
+            raise
+
+    def end_profiling(self):
+        """
+        End profiling and return results in a file.
+
+        The results are stored in a filename if the option
+        :meth:`onnxruntime.SessionOptions.enable_profiling`.
+        """
+        return self._sess.end_profiling()
+
+    def get_profiling_start_time_ns(self):
+        """
+        Return the nanoseconds of profiling's start time
+        Comparable to time.monotonic_ns() after Python 3.3
+        On some platforms, this timer may not be as precise as nanoseconds
+        For instance, on Windows and MacOS, the precision will be ~100ns
+        """
+        return self._sess.get_profiling_start_time_ns
+
+    def io_binding(self) -> IOBinding:
+        "Return an onnxruntime.IOBinding object`."
+        return IOBinding(self)
+
+    def create_ortvalue_from_shape_and_type(
+        self,
+        shape: Sequence[int],
+        element_type,
+        device_type: str,
+        device_id: int = 0,
+        vendor_id: int | OrtDeviceVendorId = -1,
+    ) -> OrtValue:
+        """Create an OrtValue using this session's allocator.
+
+        The value retains this session and may only be updated or bound through it.
+        A matching allocator and a numeric tensor type are required.
+        """
+        device = OrtDevice.make(device_type, device_id, vendor_id)._get_c_device()
+        if isinstance(element_type, int):
+            ortvalue = self._sess.create_ortvalue_from_shape_and_onnx_type(shape, element_type, device)
+        else:
+            ortvalue = self._sess.create_ortvalue_from_shape_and_type(shape, element_type, device)
+        return OrtValue(ortvalue, session=self._sess)
+
+    def release_captured_graph(self, graph_annotation_id: int = 0) -> None:
+        """Release a captured graph and unpin its IOBinding.
+
+        ``graph_annotation_id`` matches the capture's ``gpu_graph_id`` and defaults to zero.
+        EPs without captured-graph release support treat this as a no-op.
+        """
+        self._sess.release_captured_graph(graph_annotation_id)
+        pinned = self._captured_graph_bindings.pop(graph_annotation_id, None)
+        if pinned is not None:
+            iobinding = pinned[0]()
+            if iobinding is not None:
+                iobinding._pinned_graph_ids.discard(graph_annotation_id)
+
+    def run_with_iobinding(self, iobinding, run_options=None):
+        """
+        Compute the predictions.
+
+        :param iobinding: the iobinding object that has graph inputs/outputs bind.
+        :param run_options: See :class:`onnxruntime.RunOptions`.
+
+        WebGPU capture requires static shapes, disabled memory patterns, fixed WebGPU OrtValues
+        (session-owned or shared), and no CPU compute nodes other than shape-only nodes. Each
+        ``gpu_graph_id`` pins one IOBinding and its buffers until :meth:`release_captured_graph`.
+        Update inputs in place and use :meth:`IOBinding.copy_outputs_to_cpu` for readback;
+        ``gpu_graph_id=-1`` disables capture.
+        """
+        if iobinding._session is not self._sess:
+            raise ValueError("IOBinding must be used with the session that created it.")
+
+        graph_annotation_id = _graph_annotation_id(run_options)
+        capturing = self._sess.is_webgpu_graph_capture_enabled() and graph_annotation_id != _GRAPH_ANNOTATION_SKIP
+        if capturing:
+            iobinding._validate_capture_bindings()
+            signature = iobinding._capture_signature()
+            pinned = self._captured_graph_bindings.get(graph_annotation_id)
+            if pinned is not None:
+                pinned_iobinding, pinned_signature = pinned[0](), pinned[1]
+                if pinned_iobinding is not iobinding:
+                    raise ValueError(
+                        f"WebGPU graph {graph_annotation_id} was captured with a different "
+                        "IOBinding. Replay re-issues the buffers recorded at capture and would "
+                        "silently ignore this binding. Reuse the original IOBinding, or call "
+                        f"release_captured_graph({graph_annotation_id}) first."
+                    )
+                if pinned_signature != signature:
+                    raise ValueError(
+                        f"WebGPU graph {graph_annotation_id} was captured with different I/O "
+                        "buffers. Replay writes to the buffers recorded at capture, so the "
+                        "rebound values would never be read or written. Update the original "
+                        "buffers in place, or call "
+                        f"release_captured_graph({graph_annotation_id}) first."
+                    )
+
+        self._sess.run_with_iobinding(iobinding._iobinding, run_options)
+
+        if capturing and graph_annotation_id not in self._captured_graph_bindings:
+            self._captured_graph_bindings[graph_annotation_id] = (weakref.ref(iobinding), signature)
+            iobinding._pinned_graph_ids.add(graph_annotation_id)
+
+    def set_ep_dynamic_options(self, options: dict[str, str]):
+        """
+        Set dynamic options for execution providers.
+
+        :param options: Dictionary of key-value pairs where both keys and values are strings.
+                        These options will be passed to the execution providers to modify
+                        their runtime behavior.
+        """
+        self._sess.set_ep_dynamic_options(options)
+
+    def get_tuning_results(self):
+        return self._sess.get_tuning_results()
+
+    def set_tuning_results(self, results, *, error_on_invalid=False):
+        return self._sess.set_tuning_results(results, error_on_invalid)
+
+    def run_with_ortvaluevector(self, run_options, feed_names, feeds, fetch_names, fetches, fetch_devices):
+        """
+        Compute the predictions similar to other run_*() methods but with minimal C++/Python conversion overhead.
+
+        :param run_options: See :class:`onnxruntime.RunOptions`.
+        :param feed_names: list of input names.
+        :param feeds: list of input OrtValue.
+        :param fetch_names: list of output names.
+        :param fetches: list of output OrtValue.
+        :param fetch_devices: list of output devices.
+        """
+        # Same capture check as run(), run_with_ort_values() and run_async(): replay re-issues the
+        # buffers recorded at capture, so transient vectors would be silently ignored. Nothing about
+        # a raw vector is unsafe outside capture, including on a WebGPU session.
+        self._validate_graph_capture_run_api(run_options)
+        self._sess.run_with_ortvaluevector(run_options, feed_names, feeds, fetch_names, fetches, fetch_devices)
+
+
+class InferenceSession(Session):
+    """
+    This is the main class used to run a model.
+    """
+
+    def __init__(
+        self,
+        path_or_bytes: str | bytes | os.PathLike,
+        sess_options: C.SessionOptions | None = None,
+        providers: Sequence[str | tuple[str, dict[Any, Any]]] | None = None,
+        provider_options: Sequence[dict[Any, Any]] | None = None,
+        **kwargs,
+    ) -> None:
+        """
+        :param path_or_bytes: Filename or serialized ONNX or ORT format model in a byte string.
+        :param sess_options: Session options.
+        :param providers: Optional sequence of providers in order of decreasing
+            precedence. Values can either be provider names or tuples of
+            (provider name, options dict). If not provided, then all available
+            providers are used with the default precedence.
+        :param provider_options: Optional sequence of options dicts corresponding
+            to the providers listed in 'providers'.
+
+        The model type will be inferred unless explicitly set in the SessionOptions.
+        To explicitly set:
+
+        ::
+
+            so = onnxruntime.SessionOptions()
+            # so.add_session_config_entry('session.load_model_format', 'ONNX') or
+            so.add_session_config_entry('session.load_model_format', 'ORT')
+
+        A file extension of '.ort' will be inferred as an ORT format model.
+        All other filenames are assumed to be ONNX format models.
+
+        'providers' can contain either names or names and options. When any options
+        are given in 'providers', 'provider_options' should not be used.
+
+        The list of providers is ordered by precedence. For example
+        `['CUDAExecutionProvider', 'CPUExecutionProvider']`
+        means execute a node using `CUDAExecutionProvider`
+        if capable, otherwise execute using `CPUExecutionProvider`.
+        """
+        super().__init__(enable_fallback=int(kwargs.get("enable_fallback", 1)) == 1)
+
+        if isinstance(path_or_bytes, (str, os.PathLike)):
+            self._model_path = os.fspath(path_or_bytes)
+            self._model_bytes = None
+        elif isinstance(path_or_bytes, bytes):
+            self._model_path = None
+            self._model_bytes = path_or_bytes  # TODO: This is bad as we're holding the memory indefinitely
+        else:
+            raise TypeError(f"Unable to load from type '{type(path_or_bytes)}'")
+
+        self._sess_options = sess_options
+        self._sess_options_initial = sess_options
+        if "read_config_from_model" in kwargs:
+            self._read_config_from_model = int(kwargs["read_config_from_model"]) == 1
+        else:
+            self._read_config_from_model = os.environ.get("ORT_LOAD_CONFIG_FROM_MODEL") == "1"
+
+        # internal parameters that we don't expect to be used in general so aren't documented
+        disabled_optimizers = kwargs.get("disabled_optimizers")
+
+        try:
+            self._create_inference_session(providers, provider_options, disabled_optimizers)
+        except (ValueError, RuntimeError) as e:
+            if self._enable_fallback:
+                try:
+                    print("*************** EP Error ***************")
+                    print(f"EP Error {e} when using {providers}")
+                    print(f"Falling back to {self._fallback_providers} and retrying.")
+                    print("****************************************")
+                    self._create_inference_session(self._fallback_providers, None)
+                    # Fallback only once.
+                    self.disable_fallback()
+                    return
+                except Exception as fallback_error:
+                    raise fallback_error from e
+            # Fallback is disabled. Raise the original error.
+            raise e
+
+    def _create_inference_session(self, providers, provider_options, disabled_optimizers=None):
+        available_providers = C.get_available_providers()
+
+        # Validate that TensorrtExecutionProvider and NvTensorRTRTXExecutionProvider are not both specified
+        if providers:
+            has_tensorrt = any(
+                provider == "TensorrtExecutionProvider"
+                or (isinstance(provider, tuple) and provider[0] == "TensorrtExecutionProvider")
+                for provider in providers
+            )
+            has_tensorrt_rtx = any(
+                provider == "NvTensorRTRTXExecutionProvider"
+                or (isinstance(provider, tuple) and provider[0] == "NvTensorRTRTXExecutionProvider")
+                for provider in providers
+            )
+            if has_tensorrt and has_tensorrt_rtx:
+                raise ValueError(
+                    "Cannot enable both 'TensorrtExecutionProvider' and 'NvTensorRTRTXExecutionProvider' "
+                    "in the same session."
+                )
+        # Tensorrt and TensorRT RTX can fall back to CUDA if it's explicitly assigned. All others fall back to CPU.
+        if "NvTensorRTRTXExecutionProvider" in available_providers:
+            if (
+                providers
+                and any(
+                    provider == "CUDAExecutionProvider"
+                    or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
+                    for provider in providers
+                )
+                and any(
+                    provider == "NvTensorRTRTXExecutionProvider"
+                    or (isinstance(provider, tuple) and provider[0] == "NvTensorRTRTXExecutionProvider")
+                    for provider in providers
+                )
+            ):
+                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
+            else:
+                self._fallback_providers = ["CPUExecutionProvider"]
+        elif "TensorrtExecutionProvider" in available_providers:
+            if (
+                providers
+                and any(
+                    provider == "CUDAExecutionProvider"
+                    or (isinstance(provider, tuple) and provider[0] == "CUDAExecutionProvider")
+                    for provider in providers
+                )
+                and any(
+                    provider == "TensorrtExecutionProvider"
+                    or (isinstance(provider, tuple) and provider[0] == "TensorrtExecutionProvider")
+                    for provider in providers
+                )
+            ):
+                self._fallback_providers = ["CUDAExecutionProvider", "CPUExecutionProvider"]
+            else:
+                self._fallback_providers = ["CPUExecutionProvider"]
+        else:
+            self._fallback_providers = ["CPUExecutionProvider"]
+
+        # validate providers and provider_options before other initialization
+        providers, provider_options = check_and_normalize_provider_args(
+            providers, provider_options, available_providers
+        )
+
+        # Print a warning if user passed providers to InferenceSession() but the SessionOptions instance
+        # already has provider information (e.g., via add_provider_for_devices()). The providers specified
+        # here will take precedence.
+        if self._sess_options is not None and (providers or provider_options) and self._sess_options.has_providers():
+            warnings.warn(
+                "Specified 'providers'/'provider_options' when creating InferenceSession but SessionOptions has "
+                "already been configured with providers. InferenceSession will only use the providers "
+                "passed to InferenceSession()."
+            )
+
+        session_options = self._sess_options if self._sess_options else C.get_default_session_options()
+
+        self._register_ep_custom_ops(session_options, providers, provider_options, available_providers)
+
+        if self._model_path:
+            sess = C.InferenceSession(session_options, self._model_path, True, self._read_config_from_model)
+        else:
+            sess = C.InferenceSession(session_options, self._model_bytes, False, self._read_config_from_model)
+
+        if disabled_optimizers is None:
+            disabled_optimizers = set()
+        elif not isinstance(disabled_optimizers, set):
+            # convert to set. assumes iterable
+            disabled_optimizers = set(disabled_optimizers)
+
+        # initialize the C++ InferenceSession
+        sess.initialize_session(providers, provider_options, disabled_optimizers)
+
+        self._sess = sess
+        self._sess_options = self._sess.session_options
+        self._inputs_meta = self._sess.inputs_meta
+        self._outputs_meta = self._sess.outputs_meta
+        self._overridable_initializers = self._sess.overridable_initializers
+        self._input_meminfos = self._sess.input_meminfos
+        self._output_meminfos = self._sess.output_meminfos
+        self._input_epdevices = self._sess.input_epdevices
+        self._model_meta = self._sess.model_meta
+        self._providers = self._sess.get_providers()
+        self._provider_options = self._sess.get_provider_options()
+        self._profiling_start_time_ns = self._sess.get_profiling_start_time_ns
+
+    def _release_captured_graphs(self) -> None:
+        """Release every graph captured by the current session handle and unpin its IOBinding.
+
+        A captured graph belongs to the ``C.InferenceSession`` that captured it, so it must be
+        released before that handle is replaced. Otherwise the stale bookkeeping would reject an
+        IOBinding created by the replacement session and a later ``release_captured_graph`` would
+        act on the replacement session while unpinning the old binding.
+        """
+        for graph_annotation_id in sorted(self._captured_graph_bindings):
+            self.release_captured_graph(graph_annotation_id)
+
+    def _reset_session(self, providers, provider_options) -> None:
+        "release underlying session object."
+        # Captured graphs outlive neither the session handle nor its bookkeeping. Release them
+        # first so a failure here leaves the session intact instead of half torn down.
+        self._release_captured_graphs()
+
+        # meta data references session internal structures
+        # so they must be set to None to decrement _sess reference count.
+        self._sess_options = None
+        self._inputs_meta = None
+        self._outputs_meta = None
+        self._overridable_initializers = None
+        self._input_meminfos = None
+        self._output_meminfos = None
+        self._input_epdevices = None
+        self._model_meta = None
+        self._providers = None
+        self._provider_options = None
+        self._profiling_start_time_ns = None
+
+        # create a new C.InferenceSession
+        self._sess = None
+        self._sess_options = self._sess_options_initial
+        self._create_inference_session(providers, provider_options)
+
+    def _register_ep_custom_ops(self, session_options, providers, provider_options, available_providers):
+        for i in range(len(providers)):
+            if providers[i] in available_providers and providers[i] == "TensorrtExecutionProvider":
+                C.register_tensorrt_plugins_as_custom_ops(session_options, provider_options[i])
+            elif (
+                isinstance(providers[i], tuple)
+                and providers[i][0] in available_providers
+                and providers[i][0] == "TensorrtExecutionProvider"
+            ):
+                C.register_tensorrt_plugins_as_custom_ops(session_options, providers[i][1])
+
+            if providers[i] in available_providers and providers[i] == "NvTensorRTRTXExecutionProvider":
+                C.register_nv_tensorrt_rtx_plugins_as_custom_ops(session_options, provider_options[i])
+            elif (
+                isinstance(providers[i], tuple)
+                and providers[i][0] in available_providers
+                and providers[i][0] == "NvTensorrtRTXExecutionProvider"
+            ):
+                C.register_nv_tensorrt_rtx_plugins_as_custom_ops(session_options, providers[i][1])
+
+
+def make_get_initializer_location_func_wrapper(
+    get_initializer_location_func: GetInitializerLocationFunc,
+) -> GetInitializerLocationWrapperFunc:
+    """
+    Wraps a user's "get initializer location" function. The returned wrapper function adheres to the
+    signature expected by ORT.
+
+    Need this wrapper to:
+      - Convert the `initializer_value` parameter from `C.OrtValue` to `onnxruntime.OrtValue`, which is more
+        convenient for the user's function to use.
+      - Allow the user's function to return the original `external_info` parameter (this wrapper makes a copy)
+    """
+
+    def get_initializer_location_func_wrapper(
+        initializer_name: str,
+        initializer_value: C.OrtValue,
+        external_info: C.OrtExternalInitializerInfo | None,
+    ) -> C.OrtExternalInitializerInfo | None:
+        ret_val: C.OrtExternalInitializerInfo | None = get_initializer_location_func(
+            initializer_name, OrtValue(initializer_value), external_info
+        )
+        if ret_val is not None and ret_val == external_info:
+            # User returned `external_info` (const and owned by ORT). ORT expects the returned value to be
+            # a new instance (that it deletes), so make a copy.
+            ret_val = C.OrtExternalInitializerInfo(ret_val.filepath, ret_val.file_offset, ret_val.byte_size)
+        return ret_val
+
+    return get_initializer_location_func_wrapper
+
+
+class ModelCompiler:
+    """
+    This class is used to compile an ONNX model. A compiled ONNX model has EPContext nodes that each
+    encapsulates a subgraph compiled/optimized for a specific execution provider.
+
+    Refer to the EPContext design document for more information about EPContext models:
+    https://onnxruntime.ai/docs/execution-providers/EP-Context-Design.html
+
+        ::
+
+            sess_options = onnxruntime.SessionOptions()
+            sess_options.add_provider("SomeExecutionProvider", {"option1": "value1"})
+            # Alternatively, allow ONNX Runtime to select the provider automatically given a policy:
+            # sess_options.set_provider_selection_policy(onnxrt.OrtExecutionProviderDevicePolicy.PREFER_NPU)
+
+            model_compiler = onnxruntime.ModelCompiler(sess_options, "input_model.onnx")
+            model_compiler.compile_to_file("output_model.onnx")
+    """
+
+    def __init__(
+        self,
+        sess_options: C.SessionOptions,
+        input_model_path_or_bytes: str | os.PathLike | bytes,
+        embed_compiled_data_into_model: bool = False,
+        external_initializers_file_path: str | os.PathLike | None = None,
+        external_initializers_size_threshold: int = 1024,
+        flags: int = C.OrtCompileApiFlags.NONE,
+        graph_optimization_level: C.GraphOptimizationLevel = C.GraphOptimizationLevel.ORT_DISABLE_ALL,
+        get_initializer_location_func: GetInitializerLocationFunc | None = None,
+    ):
+        """
+        Creates a ModelCompiler instance.
+
+        :param sess_options: Session options containing the providers for which the model will be compiled.
+            Refer to SessionOptions.add_provider() and SessionOptions.set_provider_selection_policy().
+        :param input_model_path_or_bytes: The path to the input model file or bytes representing a serialized
+            ONNX model.
+        :param embed_compiled_data_into_model: Defaults to False. Set to True to embed compiled binary data into
+            EPContext nodes in the compiled model.
+        :param external_initializers_file_path: Defaults to None. Set to a path for a file that will store the
+            initializers for non-compiled nodes.
+        :param external_initializers_size_threshold: Defaults to 1024. Ignored if `external_initializers_file_path`
+            is None or empty. Initializers larger than this threshold are stored in the external initializers file.
+        :param flags: Additional boolean options to enable. Set this parameter to a bitwise OR of
+            flags in onnxruntime.OrtCompileApiFlags.
+        :param graph_optimization_level: The graph optimization level.
+            Defaults to onnxruntime.GraphOptimizationLevel.ORT_DISABLE_ALL.
+        :param get_initializer_location_func: Optional function called for every initializer to allow user to specify
+            whether an initializer should be stored within the model or externally. Example:
+            ```
+                def get_initializer_location(
+                    initializer_name: str,
+                    initializer_value: onnxrt.OrtValue,
+                    external_info: onnxrt.OrtExternalInitializerInfo | None,
+                ) -> onnxrt.OrtExternalInitializerInfo | None:
+                    byte_size = initializer_value.tensor_size_in_bytes()
+
+                    if byte_size < 64:
+                        return None  # Store small initializer within compiled model.
+
+                    # Else, write initializer to new external file.
+                    value_np = initializer_value.numpy()
+                    file_offset = ext_init_file.tell()
+                    ext_init_file.write(value_np.tobytes())
+                    return onnxrt.OrtExternalInitializerInfo(initializer_file_path, file_offset, byte_size)
+            ```
+        """
+        input_model_path: str | os.PathLike | None = None
+        input_model_bytes: bytes | None = None
+        if isinstance(input_model_path_or_bytes, (str, os.PathLike)):
+            if not input_model_path_or_bytes:
+                raise ValueError("Input model path is empty")
+            input_model_path = os.fspath(input_model_path_or_bytes)
+        elif isinstance(input_model_path_or_bytes, bytes):
+            if len(input_model_path_or_bytes) == 0:
+                raise ValueError("Input model bytes array is empty")
+            input_model_bytes = input_model_path_or_bytes
+        else:
+            raise TypeError(f"Unable to load from type '{type(input_model_path_or_bytes)}'")
+
+        if external_initializers_file_path:
+            if not isinstance(external_initializers_file_path, (str, os.PathLike)):
+                arg_type = type(external_initializers_file_path)
+                raise TypeError(f"Output external initializer filepath is of unexpected type '{arg_type}'")
+            external_initializers_file_path = os.fspath(external_initializers_file_path)
+        else:
+            external_initializers_file_path = ""
+
+        if get_initializer_location_func is not None:
+            if external_initializers_file_path:
+                raise ValueError(
+                    "Cannot initialize ModelCompiler with both `external_initializers_file_path` "
+                    "and `get_initializer_location_func`"
+                )
+            self.get_initializer_location_func_wrapper = make_get_initializer_location_func_wrapper(
+                get_initializer_location_func
+            )
+        else:
+            self.get_initializer_location_func_wrapper = None
+
+        if input_model_path:
+            self._model_compiler = C.ModelCompiler(
+                sess_options,
+                input_model_path,
+                True,  # is path
+                embed_compiled_data_into_model,
+                external_initializers_file_path,
+                external_initializers_size_threshold,
+                flags,
+                graph_optimization_level,
+                self.get_initializer_location_func_wrapper,
+            )
+        else:
+            self._model_compiler = C.ModelCompiler(
+                sess_options,
+                input_model_bytes,
+                False,  # is bytes
+                embed_compiled_data_into_model,
+                external_initializers_file_path,
+                external_initializers_size_threshold,
+                flags,
+                graph_optimization_level,
+                self.get_initializer_location_func_wrapper,
+            )
+
+    def compile_to_file(self, output_model_path: str | None = None):
+        """
+        Compiles to an output file. If an output file path is not provided,
+        the output file path is generated based on the input model path by replacing
+        '.onnx' with '_ctx.onnx'. Ex: The generated output file is 'model_ctx.onnx' for
+        an input model with path 'model.onnx'.
+
+        Raises an 'InvalidArgument' exception if the compilation options are invalid.
+
+        :param output_model_path: Defaults to None. The path for the output/compiled model.
+        """
+        if output_model_path:
+            if not isinstance(output_model_path, (str, os.PathLike)):
+                raise TypeError(f"Output model's filepath is of unexpected type '{type(output_model_path)}'")
+            output_model_path = os.fspath(output_model_path)
+        self._model_compiler.compile_to_file(output_model_path)
+
+    def compile_to_bytes(self) -> bytes:
+        """
+        Compiles to bytes representing the serialized compiled ONNX model.
+
+        Raises an 'InvalidArgument' exception if the compilation options are invalid.
+
+        :return: A bytes object representing the compiled ONNX model.
+        """
+        return self._model_compiler.compile_to_bytes()
+
+    def compile_to_stream(self, write_function: Callable[[bytes], None]):
+        """
+        Compiles the input model and writes the serialized ONNX bytes to a stream using the provided write function.
+        Raises an 'InvalidArgument' exception if the compilation options are invalid.
+        :param write_function: A callable that accepts a bytes buffer to write.
+        """
+        self._model_compiler.compile_to_stream(write_function)
+
+
+class IOBinding:
+    """
+    This class provides API to bind input/output to a specified device, e.g. GPU.
+    """
+
+    def __init__(self, session: Session):
+        self._session = session._sess
+        self._is_webgpu_session = "WebGpuExecutionProvider" in session.get_providers()
+        self._is_webgpu_graph_capture_enabled = self._session.is_webgpu_graph_capture_enabled()
+        self._iobinding = C.SessionIOBinding(session._sess)
+        self._numpy_obj_references = {}
+        # Capture tracks fixed OrtValues; None marks host or raw-pointer bindings.
+        self._bound_inputs: dict[str, OrtValue | None] = {}
+        self._bound_outputs: dict[str, OrtValue | None] = {}
+        # Captured graph IDs freeze these bindings until release.
+        self._pinned_graph_ids: set[int] = set()
+
+    def _reject_if_pinned(self, action: str) -> None:
+        if self._pinned_graph_ids:
+            pinned = ", ".join(str(graph_id) for graph_id in sorted(self._pinned_graph_ids))
+            raise ValueError(
+                f"Cannot {action} while captured WebGPU graph(s) [{pinned}] still reference this "
+                "IOBinding. Replay re-issues the buffers recorded at capture, so a change here "
+                "would not affect replay and releasing the buffers could invalidate it. Call "
+                "release_captured_graph(id) first."
+            )
+
+    def _capture_signature(self):
+        """Return bound OrtValues whose identity and lifetime must remain fixed during replay."""
+        return (
+            tuple(sorted(self._bound_inputs.items(), key=lambda item: item[0])),
+            tuple(sorted(self._bound_outputs.items(), key=lambda item: item[0])),
+        )
+
+    def _validate_capture_bindings(self) -> None:
+        """Every binding participating in capture must be a fixed WebGPU device OrtValue."""
+        for kind, bound in (("input", self._bound_inputs), ("output", self._bound_outputs)):
+            for name, value in bound.items():
+                if value is None or not value._is_webgpu_buffer:
+                    raise ValueError(
+                        f"WebGPU graph capture requires fixed WebGPU device OrtValues; {kind} "
+                        f"'{name}' is not one. Bind values created by this session or backed by an "
+                        f"environment-registered shared allocator, or set the "
+                        f"'{_GPU_GRAPH_ID_RUN_CONFIG_KEY}' run option to {_GRAPH_ANNOTATION_SKIP} "
+                        "to run without capture."
+                    )
+
+    def bind_cpu_input(self, name, arr_on_cpu):
+        """
+        bind an input to array on CPU
+        :param name: input name
+        :param arr_on_cpu: input values as a python array on CPU
+        """
+        self._reject_if_pinned("rebind inputs")
+
+        # Hold a reference to the numpy object as the bound OrtValue is backed
+        # directly by the data buffer of the numpy object and so the numpy object
+        # must be around until this IOBinding instance is around
+        self._numpy_obj_references[name] = arr_on_cpu
+        self._iobinding.bind_input(name, arr_on_cpu)
+        self._bound_inputs[name] = None
+
+    def bind_input(self, name, device_type, device_id, element_type, shape, buffer_ptr):
+        """
+        :param name: input name
+        :param device_type: e.g. cpu, cuda, cann
+        :param device_id: device id, e.g. 0
+        :param element_type: input element type. It can be either numpy type (like numpy.float32) or an integer for onnx type (like onnx.TensorProto.BFLOAT16)
+        :param shape: input shape
+        :param buffer_ptr: memory pointer to input data
+        """
+        self._reject_if_pinned("rebind inputs")
+        self._iobinding.bind_input(
+            name,
+            OrtDevice.make(device_type, device_id)._get_c_device(),
+            element_type,
+            shape,
+            buffer_ptr,
+        )
+        self._bound_inputs[name] = None
+
+    def bind_ortvalue_input(self, name, ortvalue):
+        """
+        :param name: input name
+        :param ortvalue: OrtValue instance to bind
+        """
+        _validate_ortvalue_session_compatibility(ortvalue, self._session, "bound to")
+        self._reject_if_pinned("rebind inputs")
+        self._iobinding.bind_ortvalue_input(name, ortvalue._ortvalue)
+        self._bound_inputs[name] = ortvalue
+
+    def synchronize_inputs(self):
+        self._iobinding.synchronize_inputs()
+
+    def bind_output(
+        self,
+        name,
+        device_type="cpu",
+        device_id=0,
+        element_type=None,
+        shape=None,
+        buffer_ptr=None,
+    ):
+        """
+        :param name: output name
+        :param device_type: e.g. cpu, cuda, cann, cpu by default
+        :param device_id: device id, e.g. 0
+        :param element_type: output element type. It can be either numpy type (like numpy.float32) or an integer for onnx type (like onnx.TensorProto.BFLOAT16)
+        :param shape: output shape
+        :param buffer_ptr: memory pointer to output data
+        """
+
+        self._reject_if_pinned("rebind outputs")
+
+        # Follow the `if` path when the user has not provided any pre-allocated buffer but still
+        # would like to bind an output to a specific device (e.g. cuda).
+        # Pre-allocating an output buffer may not be an option for the user as :
+        # (1) They may not want to use a custom allocator specific to the device they want to bind the output to,
+        # in which case ORT will allocate the memory for the user
+        # (2) The output has a dynamic shape and hence the size of the buffer may not be fixed across runs
+        if buffer_ptr is None:
+            self._iobinding.bind_output(
+                name,
+                OrtDevice.make(device_type, device_id)._get_c_device(),
+            )
+        else:
+            if element_type is None or shape is None:
+                raise ValueError("`element_type` and `shape` are to be provided if pre-allocated memory is provided")
+            self._iobinding.bind_output(
+                name,
+                OrtDevice.make(device_type, device_id)._get_c_device(),
+                element_type,
+                shape,
+                buffer_ptr,
+            )
+        self._bound_outputs[name] = None
+
+    def bind_ortvalue_output(self, name, ortvalue):
+        """
+        :param name: output name
+        :param ortvalue: OrtValue instance to bind
+        """
+        _validate_ortvalue_session_compatibility(ortvalue, self._session, "bound to")
+        self._reject_if_pinned("rebind outputs")
+        self._iobinding.bind_ortvalue_output(name, ortvalue._ortvalue)
+        self._bound_outputs[name] = ortvalue
+
+    def synchronize_outputs(self):
+        self._iobinding.synchronize_outputs()
+
+    def get_outputs(self):
+        """
+        Returns the output OrtValues from the Run() that preceded the call.
+        The data buffer of the obtained OrtValues may not reside on CPU memory
+        """
+        outputs = self._iobinding.get_outputs()
+        if not isinstance(outputs, C.OrtValueVector):
+            raise TypeError("get_outputs() must return an instance of type 'OrtValueVector'.")
+        result = []
+        for index in range(len(outputs)):
+            ortvalue = outputs[index]
+            result.append(
+                OrtValue(
+                    ortvalue,
+                    session=self._session if ortvalue._is_webgpu_buffer() else None,
+                )
+            )
+        return result
+
+    def get_outputs_as_ortvaluevector(self):
+        """Return the raw OrtValueVector of outputs from the Run() that preceded the call.
+
+        The vector is a reference into this IOBinding (pybind ``reference_internal``), so it keeps the
+        IOBinding alive, which in turn keeps the session alive. Device-resident outputs are therefore
+        safe to hold past the session going out of scope.
+        """
+        return self._iobinding.get_outputs()
+
+    def copy_outputs_to_cpu(self):
+        """Copy output contents to CPU."""
+        return self._iobinding.copy_outputs_to_cpu()
+
+    def clear_binding_inputs(self):
+        self._reject_if_pinned("clear input bindings")
+        self._iobinding.clear_binding_inputs()
+        self._bound_inputs.clear()
+
+    def clear_binding_outputs(self):
+        self._reject_if_pinned("clear output bindings")
+        self._iobinding.clear_binding_outputs()
+        self._bound_outputs.clear()
+
+
+class OrtValue:
+    """
+    A data structure that supports all ONNX data formats (tensors and non-tensors) that allows users
+    to place the data backing these on a device, for example, on a CUDA supported device.
+    This class provides APIs to construct and deal with OrtValues.
+    """
+
+    def __init__(
+        self,
+        ortvalue: C.OrtValue,
+        numpy_obj: np.ndarray | None = None,
+        session: C.InferenceSession | None = None,
+    ):
+        if isinstance(ortvalue, C.OrtValue):
+            self._ortvalue = ortvalue
+            # Hold a ref count to the numpy object if the OrtValue is backed directly
+            # by its data buffer so that it isn't destroyed when the OrtValue is in use
+            self._numpy_obj = numpy_obj
+            # Session-scoped device allocators can be invalidated when their session is destroyed.
+            self._session = session
+        else:
+            # An end user won't hit this error
+            raise ValueError(
+                "`Provided ortvalue` needs to be of type `onnxruntime.capi.onnxruntime_pybind11_state.OrtValue`"
+            )
+
+    @property
+    def _is_webgpu_buffer(self) -> bool:
+        """Whether this value is backed by a WebGPU buffer.
+
+        Resolved from the native OrtValue on every access and deliberately read-only: graph-capture
+        validation and the copy-path checks below refuse host memory, so a settable attribute would
+        let a CPU tensor pass itself off as a device tensor.
+        """
+        return self._ortvalue._is_webgpu_buffer()
+
+    def _get_c_value(self) -> C.OrtValue:
+        return self._ortvalue
+
+    @classmethod
+    def ortvalue_from_numpy(
+        cls, numpy_obj: np.ndarray, /, device_type="cpu", device_id=0, vendor_id: int | OrtDeviceVendorId = -1
+    ) -> OrtValue:
+        """
+        Factory method to construct an OrtValue (which holds a Tensor) from a given Numpy object
+        A copy of the data in the Numpy object is held by the OrtValue only if the device is NOT cpu
+
+        :param numpy_obj: The Numpy object to construct the OrtValue from
+        :param device_type: e.g. cpu, cuda, cann, cpu by default
+        :param device_id: device id, e.g. 0
+        :param vendor_id: The device's PCI vendor id as an int or OrtDeviceVendorId. If provided, the device_type should be "gpu" or "npu".
+        """
+        # Hold a reference to the numpy object (if device_type is 'cpu') as the OrtValue
+        # is backed directly by the data buffer of the numpy object and so the numpy object
+        # must be around until this OrtValue instance is around
+        return cls(
+            C.OrtValue.ortvalue_from_numpy(
+                numpy_obj,
+                OrtDevice.make(device_type, device_id, vendor_id)._get_c_device(),
+            ),
+            numpy_obj if device_type.lower() == "cpu" else None,
+        )
+
+    @classmethod
+    def ortvalue_from_numpy_with_onnx_type(cls, data: np.ndarray, /, onnx_element_type: int) -> OrtValue:
+        """
+        This method creates an instance of OrtValue on top of the numpy array.
+        No data copy is made and the lifespan of the resulting OrtValue should never
+        exceed the lifespan of bytes object. The API attempts to reinterpret
+        the data type which is expected to be the same size. This is useful
+        when we want to use an ONNX data type that is not supported by numpy.
+
+        :param data: numpy.ndarray.
+        :param onnx_element_type: a valid onnx TensorProto::DataType enum value
+        """
+        return cls(C.OrtValue.ortvalue_from_numpy_with_onnx_type(data, onnx_element_type), data)
+
+    @classmethod
+    def ortvalue_from_shape_and_type(
+        cls,
+        shape: Sequence[int],
+        element_type,
+        device_type: str = "cpu",
+        device_id: int = 0,
+        vendor_id: int | OrtDeviceVendorId = -1,
+        memory_info: C.OrtMemoryInfo | None = None,
+    ) -> OrtValue:
+        """
+        Factory method to construct an OrtValue (which holds a Tensor) from given shape and element_type
+
+        :param shape: List of integers indicating the shape of the OrtValue
+        :param element_type: The data type of the elements. It can be either numpy type (like numpy.float32) or an integer for onnx type (like onnx.TensorProto.BFLOAT16).
+        :param device_type: e.g. cpu, cuda, cann, cpu by default
+        :param device_id: device id, e.g. 0
+        :param vendor_id: The device's PCI vendor id as an int or OrtDeviceVendorId. If provided, the device type should be "gpu" or "npu".
+        :param memory_info: An OrtMemoryInfo from an OrtEpDevice (e.g. via ep_device.memory_info(OrtDeviceMemoryType.HOST_ACCESSIBLE)). When provided, the allocator matching this memory info is used directly, which allows allocating HOST_ACCESSIBLE memory for zero-copy numpy interop. The device_type, device_id, and vendor_id parameters are ignored when memory_info is provided.
+        """
+
+        if memory_info is not None:
+            if device_type != "cpu" or device_id != 0 or vendor_id != -1:
+                warnings.warn(
+                    "device_type, device_id, and vendor_id are ignored when memory_info is provided.",
+                    stacklevel=2,
+                )
+            if isinstance(element_type, int):
+                return cls(
+                    C.OrtValue.ortvalue_from_shape_and_onnx_type_for_memory_info(
+                        shape,
+                        element_type,
+                        memory_info,
+                    )
+                )
+            return cls(
+                C.OrtValue.ortvalue_from_shape_and_type_for_memory_info(
+                    shape,
+                    element_type,
+                    memory_info,
+                )
+            )
+
+        device = OrtDevice.make(device_type, device_id, vendor_id)._get_c_device()
+
+        # Integer for onnx element type (see https://onnx.ai/onnx/api/mapping.html).
+        # This is helpful for some data type (like TensorProto.BFLOAT16) that is not available in numpy.
+        if isinstance(element_type, int):
+            return cls(
+                C.OrtValue.ortvalue_from_shape_and_onnx_type(
+                    shape,
+                    element_type,
+                    device,
+                )
+            )
+
+        return cls(
+            C.OrtValue.ortvalue_from_shape_and_type(
+                shape,
+                element_type,
+                device,
+            )
+        )
+
+    @classmethod
+    def ort_value_from_sparse_tensor(cls, sparse_tensor: SparseTensor) -> OrtValue:
+        """
+        The function will construct an OrtValue instance from a valid SparseTensor
+        The new instance of OrtValue will assume the ownership of sparse_tensor
+        """
+        return cls(C.OrtValue.ort_value_from_sparse_tensor(sparse_tensor._get_c_tensor()))
+
+    def as_sparse_tensor(self) -> SparseTensor:
+        """
+        The function will return SparseTensor contained in this OrtValue
+        """
+        return SparseTensor(self._ortvalue.as_sparse_tensor())
+
+    def data_ptr(self) -> int:
+        """
+        Returns the address of the first element in the OrtValue's data buffer
+
+        WebGPU buffers are opaque handles and do not expose a data pointer.
+        """
+        return self._ortvalue.data_ptr()
+
+    def device_name(self) -> str:
+        """
+        Returns the name of the device where the OrtValue's data buffer resides e.g. cpu, cuda, cann, webgpu
+        """
+        return self._ortvalue.device_name().lower()
+
+    def shape(self) -> Sequence[int]:
+        """
+        Returns the shape of the data in the OrtValue
+        """
+        return self._ortvalue.shape()
+
+    def data_type(self) -> str:
+        """
+        Returns the data type of the data in the OrtValue. E.g. 'tensor(int64)'
+        """
+        return self._ortvalue.data_type()
+
+    def element_type(self) -> int:
+        """
+        Returns the proto type of the data in the OrtValue
+        if the OrtValue is a tensor.
+        """
+        return self._ortvalue.element_type()
+
+    def tensor_size_in_bytes(self) -> int:
+        """
+        Returns the size of the data in the OrtValue in bytes
+        if the OrtValue is a tensor.
+        """
+        return self._ortvalue.tensor_size_in_bytes()
+
+    def has_value(self) -> bool:
+        """
+        Returns True if the OrtValue corresponding to an
+        optional type contains data, else returns False
+        """
+        return self._ortvalue.has_value()
+
+    def is_tensor(self) -> bool:
+        """
+        Returns True if the OrtValue contains a Tensor, else returns False
+        """
+        return self._ortvalue.is_tensor()
+
+    def is_sparse_tensor(self) -> bool:
+        """
+        Returns True if the OrtValue contains a SparseTensor, else returns False
+        """
+        return self._ortvalue.is_sparse_tensor()
+
+    def is_tensor_sequence(self) -> bool:
+        """
+        Returns True if the OrtValue contains a Tensor Sequence, else returns False
+        """
+        return self._ortvalue.is_tensor_sequence()
+
+    def numpy(self) -> np.ndarray:
+        """
+        Returns a Numpy object from the OrtValue.
+        Valid only for OrtValues holding Tensors. Throws for OrtValues holding non-Tensors.
+        Use accessors to gain a reference to non-Tensor objects such as SparseTensor
+        WebGPU device values require explicit readback with
+        :meth:`IOBinding.copy_outputs_to_cpu`.
+        """
+        return self._ortvalue.numpy()
+
+    def __array__(self, dtype=None, copy=None) -> np.ndarray:
+        """
+        Supports ``numpy.asarray(ortvalue)`` and ``numpy.array(ortvalue)`` via the
+        `numpy __array__ protocol <https://numpy.org/devdocs/user/basics.interoperability.html>`_.
+
+        Valid only for OrtValues holding Tensors on CPU.
+
+        :param dtype: Optional numpy dtype to cast the result to.
+        :param copy: Optional bool (numpy >= 2.0). If ``False``, a copy will
+            only be made if necessary. If ``True``, a copy is always forced.
+            If ``None`` (default), a copy will be made only if needed.
+        :return: A numpy array with the same data as the OrtValue.
+        """
+        arr = self.numpy()
+
+        if copy is not None:
+            # numpy >= 2.0 added the copy kwarg to np.asarray;
+            # np.array has always accepted it but with weaker semantics pre-2.0.
+            arr = np.array(arr, dtype=dtype, copy=copy)
+        elif dtype is not None:
+            # np.asarray avoids a copy when the dtype already matches,
+            # preserving memory sharing with the underlying OrtValue.
+            arr = np.asarray(arr, dtype=dtype)
+
+        return arr
+
+    def __dlpack__(self, *, stream=None):
+        """
+        Returns a DLPack capsule representing the tensor (part of the
+        `DLPack protocol <https://dmlc.github.io/dlpack/latest/>`_).
+
+        This enables interoperability with other frameworks via
+        ``from_dlpack(ortvalue)`` (e.g. ``torch.from_dlpack``,
+        ``jax.dlpack.from_dlpack``, ``numpy.from_dlpack``).
+
+        The OrtValue must hold a contiguous tensor. No data is copied;
+        the consumer shares memory with this OrtValue, which must remain
+        alive while the capsule is in use.
+        WebGPU OrtValues cannot be exported through DLPack because their data
+        is stored in opaque WebGPU buffers rather than CUDA-addressable memory.
+
+        :param stream: Optional stream on which the tensor data is accessible.
+            Currently unused; included for protocol compliance.
+        :return: A PyCapsule holding a DLManagedTensor.
+        """
+        return self._ortvalue.__dlpack__(stream=stream)
+
+    def __dlpack_device__(self) -> tuple[int, int]:
+        """
+        Returns ``(device_type, device_id)`` indicating where the tensor data
+        resides (part of the `DLPack protocol
+        <https://dmlc.github.io/dlpack/latest/>`_).
+
+        WebGPU OrtValues do not expose a DLPack device.
+
+        :return: Tuple of ``(device_type, device_id)`` as ints following DLPack
+            ``DLDeviceType`` enum values.
+        """
+        return self._ortvalue.__dlpack_device__()
+
+    @classmethod
+    def from_dlpack(cls, data, /) -> OrtValue:
+        """
+        Construct an OrtValue from an object that implements the DLPack protocol.
+
+        Accepts either:
+
+        * An object with ``__dlpack__`` / ``__dlpack_device__`` methods
+          (e.g. a PyTorch tensor, JAX array, or numpy array).
+        * A raw DLPack PyCapsule (legacy path).
+
+        Boolean tensors are automatically detected when the source object
+        exposes a ``dtype`` attribute (numpy, PyTorch, etc.) or is an
+        ``OrtValue``. For raw DLPack capsules where the original dtype cannot
+        be inspected, bool tensors encoded as uint8 by older DLPack versions
+        are not distinguishable from true uint8 tensors and will be imported
+        as uint8.
+
+        No data is copied; the new OrtValue shares memory with the source.
+
+        :param data: A tensor object supporting the DLPack protocol, or a raw
+            DLPack PyCapsule.
+        :return: An OrtValue wrapping the tensor data.
+        """
+        # Detect boolean dtype from the source object before consuming it,
+        # because DLPack encodes bool as uint8 and the capsule alone cannot
+        # distinguish between the two.
+        is_bool = False
+        if isinstance(data, OrtValue):
+            is_bool = data.data_type() == "tensor(bool)"
+        elif hasattr(data, "dtype"):
+            dtype_obj = data.dtype
+            # Use .name when available (numpy, cupy, tensorflow all expose it).
+            # Fall back to str() for frameworks that don't (e.g. PyTorch).
+            dtype_name = getattr(dtype_obj, "name", str(dtype_obj))
+            is_bool = dtype_name in ("bool", "bool_", "torch.bool")
+
+        # If the input supports the __dlpack__ protocol, call it to get the capsule.
+        if hasattr(data, "__dlpack__"):
+            capsule = data.__dlpack__()
+        else:
+            capsule = data
+
+        return cls(C.OrtValue.from_dlpack(capsule, is_bool))
+
+    def update_inplace(self, data) -> None:
+        """
+        Update the OrtValue in place. The source data is copied over to the device
+        memory backing the OrtValue. It can be used to update the input values for
+        an InferenceSession with CUDA graph enabled or other scenarios where the
+        OrtValue needs to be updated while the memory address can not be changed.
+
+        :param data: The source data, which can be a Numpy array or another OrtValue.
+            When an OrtValue is provided, data can be copied between devices (e.g.,
+            GPU to GPU) without going through the CPU.
+        """
+        if isinstance(data, OrtValue):
+            if self._is_webgpu_buffer != data._is_webgpu_buffer:
+                raise ValueError(
+                    "WebGPU OrtValue copies require WebGPU source and destination values; "
+                    "use IOBinding.copy_outputs_to_cpu for readback."
+                )
+            if not _is_ortvalue_session_compatible(data, self._session):
+                raise ValueError("Session-scoped OrtValues must originate from the same session.")
+
+            self._ortvalue.update_inplace(data._ortvalue)
+            return
+
+        if not isinstance(data, np.ndarray):
+            raise TypeError("data must be a numpy.ndarray or an OrtValue.")
+
+        # Every copy path requires contiguous source storage.
+        if not data.flags.c_contiguous:
+            data = np.ascontiguousarray(data)
+
+        self._ortvalue.update_inplace(data)
+
+
+_DEFAULT_WEBGPU_CONTEXT_ID = 0
+
+
+def _session_webgpu_context_id(session: C.InferenceSession) -> int:
+    """Return a session's WebGPU context id, or -1 when it has no WebGPU EP."""
+    return session.webgpu_context_id()
+
+
+def _has_foreign_webgpu_context(
+    values: Sequence[OrtValue],
+    context_id_getter=_session_webgpu_context_id,
+) -> bool:
+    """Whether any value provably belongs to a WebGPU context other than the default one.
+
+    The environment data transfer resolves the default WebGPU context, so copying a buffer that
+    belongs to a caller-supplied context would silently run on the wrong device. A value with no
+    owning session cannot be attributed to a context and is treated as default-context, as is a
+    session with no WebGPU EP.
+
+    TODO: this can only attribute a value that carries session provenance, because every WebGPU
+    allocation reports OrtDevice(GPU, VendorIds::NONE, 0) regardless of its context. Supporting a
+    custom external WebGPU context requires the shared allocator and its data transfer to retain and
+    use that external device instead of resolving WebGpuContextFactory::DefaultContext(), which
+    would also let the transfer itself reject the copy instead of relying on this Python-side guard.
+    """
+    for value in values:
+        if not value._is_webgpu_buffer or value._session is None:
+            continue
+        if context_id_getter(value._session) not in (-1, _DEFAULT_WEBGPU_CONTEXT_ID):
+            return True
+    return False
+
+
+def copy_tensors(src: Sequence[OrtValue], dst: Sequence[OrtValue], stream=None) -> None:
+    """
+    Copy tensor data from source OrtValue sequence to destination OrtValue sequence.
+
+    WebGPU values belonging to the default context are supported in either direction. A value from a
+    caller-supplied WebGPU context is not, because the shared data transfer is bound to the default
+    context.
+    """
+    if _has_foreign_webgpu_context([*src, *dst]):
+        raise ValueError(
+            "copy_tensors cannot copy a WebGPU OrtValue that belongs to a custom WebGPU context "
+            "because the shared data transfer is bound to the default context; use IOBinding "
+            "readback instead."
+        )
+    c_sources = [s._get_c_value() for s in src]
+    c_dsts = [d._get_c_value() for d in dst]
+    C.copy_tensors(c_sources, c_dsts, stream)
+
+
+class OrtDevice:
+    """
+    A data structure that exposes the underlying C++ OrtDevice
+    """
+
+    def __init__(self, c_ort_device):
+        """
+        Internal constructor
+        """
+        if isinstance(c_ort_device, C.OrtDevice):
+            self._ort_device = c_ort_device
+        else:
+            # An end user won't hit this error
+            raise ValueError(
+                "`Provided object` needs to be of type `onnxruntime.capi.onnxruntime_pybind11_state.OrtDevice`"
+            )
+
+    def _get_c_device(self):
+        """
+        Internal accessor to underlying object
+        """
+        return self._ort_device
+
+    @staticmethod
+    def make(ort_device_name, device_id, vendor_id: int | OrtDeviceVendorId = -1):
+        if vendor_id < 0:
+            # Preserve the historical convenience aliases ("cuda", "dml", "cann")
+            # while making them work with plugin EP shared allocators. Those
+            # allocators are keyed by vendor-specific OrtDevice values even when the
+            # Python package itself was built without the corresponding built-in EP.
+            alias_vendor_id = get_vendor_id_for_device_type(ort_device_name)
+            if alias_vendor_id is not None:
+                return OrtDevice(
+                    C.OrtDevice(
+                        get_ort_device_type(ort_device_name),
+                        C.OrtDevice.default_memory(),
+                        int(alias_vendor_id),
+                        device_id,
+                    )
+                )
+
+            # backwards compatibility with generic predefined OrtDevice names
+            return OrtDevice(
+                C.OrtDevice(
+                    get_ort_device_type(ort_device_name),
+                    C.OrtDevice.default_memory(),
+                    device_id,
+                )
+            )
+        else:
+            # generic. use GPU or NPU for ort_device_name and provide a vendor id.
+            # vendor id of 0 is valid in some cases (e.g. webgpu is generic and does not have a vendor id)
+            return OrtDevice(
+                C.OrtDevice(
+                    get_ort_device_type(ort_device_name),
+                    C.OrtDevice.default_memory(),
+                    int(vendor_id),
+                    device_id,
+                )
+            )
+
+    def device_id(self):
+        return self._ort_device.device_id()
+
+    def device_type(self):
+        return self._ort_device.device_type()
+
+    def device_vendor_id(self):
+        return self._ort_device.vendor_id()
+
+    def device_mem_type(self):
+        return self._ort_device.mem_type()
+
+
+class SparseTensor:
+    """
+    A data structure that project the C++ SparseTensor object
+    The class provides API to work with the object.
+    Depending on the format, the class will hold more than one buffer
+    depending on the format
+    """
+
+    def __init__(self, sparse_tensor: C.SparseTensor):
+        """
+        Internal constructor
+        """
+        if isinstance(sparse_tensor, C.SparseTensor):
+            self._tensor = sparse_tensor
+        else:
+            # An end user won't hit this error
+            raise ValueError(
+                "`Provided object` needs to be of type `onnxruntime.capi.onnxruntime_pybind11_state.SparseTensor`"
+            )
+
+    def _get_c_tensor(self) -> C.SparseTensor:
+        return self._tensor
+
+    @classmethod
+    def sparse_coo_from_numpy(
+        cls,
+        dense_shape: npt.NDArray[np.int64],
+        values: np.ndarray,
+        coo_indices: npt.NDArray[np.int64],
+        ort_device: OrtDevice,
+    ) -> SparseTensor:
+        """
+        Factory method to construct a SparseTensor in COO format from given arguments
+
+        :param dense_shape: 1-D  numpy array(int64) or a python list that contains a dense_shape of the sparse tensor
+            must be on cpu memory
+        :param values: a homogeneous, contiguous 1-D numpy array that contains non-zero elements of the tensor
+            of a type.
+        :param coo_indices:  contiguous numpy array(int64) that contains COO indices for the tensor. coo_indices may
+            have a 1-D shape when it contains a linear index of non-zero values and its length must be equal to
+            that of the values. It can also be of 2-D shape, in which has it contains pairs of coordinates for
+            each of the nnz values and its length must be exactly twice of the values length.
+        :param ort_device: - describes the backing memory owned by the supplied nummpy arrays. Only CPU memory is
+            suppored for non-numeric data types.
+
+        For primitive types, the method will map values and coo_indices arrays into native memory and will use
+        them as backing storage. It will increment the reference count for numpy arrays and it will decrement it
+        on GC. The buffers may reside in any storage either CPU or GPU.
+        For strings and objects, it will create a copy of the arrays in CPU memory as ORT does not support those
+        on other devices and their memory can not be mapped.
+        """
+        return cls(C.SparseTensor.sparse_coo_from_numpy(dense_shape, values, coo_indices, ort_device._get_c_device()))
+
+    @classmethod
+    def sparse_csr_from_numpy(
+        cls,
+        dense_shape: npt.NDArray[np.int64],
+        values: np.ndarray,
+        inner_indices: npt.NDArray[np.int64],
+        outer_indices: npt.NDArray[np.int64],
+        ort_device: OrtDevice,
+    ) -> SparseTensor:
+        """
+        Factory method to construct a SparseTensor in CSR format from given arguments
+
+        :param dense_shape: 1-D numpy array(int64) or a python list that contains a dense_shape of the
+            sparse tensor (rows, cols) must be on cpu memory
+        :param values: a  contiguous, homogeneous 1-D numpy array that contains non-zero elements of the tensor
+            of a type.
+        :param inner_indices:  contiguous 1-D numpy array(int64) that contains CSR inner indices for the tensor.
+            Its length must be equal to that of the values.
+        :param outer_indices:  contiguous 1-D numpy array(int64) that contains CSR outer indices for the tensor.
+            Its length must be equal to the number of rows + 1.
+        :param ort_device: - describes the backing memory owned by the supplied nummpy arrays. Only CPU memory is
+            suppored for non-numeric data types.
+
+        For primitive types, the method will map values and indices arrays into native memory and will use them as
+        backing storage. It will increment the reference count and it will decrement then count when it is GCed.
+        The buffers may reside in any storage either CPU or GPU.
+        For strings and objects, it will create a copy of the arrays in CPU memory as ORT does not support those
+        on other devices and their memory can not be mapped.
+        """
+        return cls(
+            C.SparseTensor.sparse_csr_from_numpy(
+                dense_shape,
+                values,
+                inner_indices,
+                outer_indices,
+                ort_device._get_c_device(),
+            )
+        )
+
+    def values(self) -> np.ndarray:
+        """
+        The method returns a numpy array that is backed by the native memory
+        if the data type is numeric. Otherwise, the returned numpy array that contains
+        copies of the strings.
+        """
+        return self._tensor.values()
+
+    def as_coo_view(self):
+        """
+        The method will return coo representation of the sparse tensor which will enable
+        querying COO indices. If the instance did not contain COO format, it would throw.
+        You can query coo indices as:
+
+        ::
+
+            coo_indices = sparse_tensor.as_coo_view().indices()
+
+        which will return a numpy array that is backed by the native memory.
+        """
+        return self._tensor.get_coo_data()
+
+    def as_csrc_view(self):
+        """
+        The method will return CSR(C) representation of the sparse tensor which will enable
+        querying CRS(C) indices. If the instance dit not contain CSR(C) format, it would throw.
+        You can query indices as:
+
+        ::
+
+            inner_ndices = sparse_tensor.as_csrc_view().inner()
+            outer_ndices = sparse_tensor.as_csrc_view().outer()
+
+        returning numpy arrays backed by the native memory.
+        """
+        return self._tensor.get_csrc_data()
+
+    def as_blocksparse_view(self):
+        """
+        The method will return coo representation of the sparse tensor which will enable
+        querying BlockSparse indices. If the instance did not contain BlockSparse format, it would throw.
+        You can query coo indices as:
+
+        ::
+
+            block_sparse_indices = sparse_tensor.as_blocksparse_view().indices()
+
+        which will return a numpy array that is backed by the native memory
+        """
+        return self._tensor.get_blocksparse_data()
+
+    def to_cuda(self, ort_device):
+        """
+        Returns a copy of this instance on the specified cuda device
+
+        :param ort_device: with name 'cuda' and valid gpu device id
+
+        The method will throw if:
+
+        - this instance contains strings
+        - this instance is already on GPU. Cross GPU copy is not supported
+        - CUDA is not present in this build
+        - if the specified device is not valid
+        """
+        return SparseTensor(self._tensor.to_cuda(ort_device._get_c_device()))
+
+    def format(self):
+        """
+        Returns a OrtSparseFormat enumeration
+        """
+        return self._tensor.format
+
+    def dense_shape(self) -> npt.NDArray[np.int64]:
+        """
+        Returns a numpy array(int64) containing a dense shape of a sparse tensor
+        """
+        return self._tensor.dense_shape()
+
+    def data_type(self) -> str:
+        """
+        Returns a string data type of the data in the OrtValue
+        """
+        return self._tensor.data_type()
+
+    def device_name(self) -> str:
+        """
+        Returns the name of the device where the SparseTensor data buffers reside e.g. cpu, cuda
+        """
+        return self._tensor.device_name().lower()
+
+
+# Type hint for user-specified function that allows the user to specify initializer locations when compiling a model.
+GetInitializerLocationFunc = Callable[
+    [str, OrtValue, C.OrtExternalInitializerInfo | None], C.OrtExternalInitializerInfo | None
+]
+
+# Type hint that adheres to the signature expected by ORT.
+GetInitializerLocationWrapperFunc = Callable[
+    [str, C.OrtValue, C.OrtExternalInitializerInfo | None], C.OrtExternalInitializerInfo | None
+]

onnxruntime/capi/onnxruntime_providers_cuda.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2c744e0d1683518fa88e229ca006304aad6a29b81b9f0229596969c89ba80c12
+size 689375232

onnxruntime/capi/onnxruntime_providers_openvino.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:15664f6d9bad3f65705df0ec6275eb029cc7efa2310e0243e30349a9195b71ce
+size 1345024

onnxruntime/capi/onnxruntime_providers_tensorrt.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:67264756a9b187e9329c8a9b6c013bb39dfe3a52eaa4f44bc19be283d68afcac
+size 744448

onnxruntime/capi/onnxruntime_pybind11_state.pyd

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e54c4900fc60c521dc597dc0b0c87835a18ea6ee2efd38af78c70ac6f10abcbb
+size 25851904

onnxruntime/capi/onnxruntime_validation.py

@@ -0,0 +1,154 @@
+# -------------------------------------------------------------------------
+# Copyright (c) Microsoft Corporation. All rights reserved.
+# Licensed under the MIT License.
+# --------------------------------------------------------------------------
+"""
+Check OS requirements for ONNX Runtime Python Bindings.
+"""
+
+import linecache
+import platform
+import warnings
+
+
+def check_distro_info():
+    __my_distro__ = ""
+    __my_distro_ver__ = ""
+    __my_system__ = platform.system().lower()
+
+    __OS_RELEASE_FILE__ = "/etc/os-release"  # noqa: N806
+    __LSB_RELEASE_FILE__ = "/etc/lsb-release"  # noqa: N806
+
+    if __my_system__ == "windows":
+        __my_distro__ = __my_system__
+        __my_distro_ver__ = platform.release().lower()
+
+        if __my_distro_ver__ not in ["10", "11", "2016server", "2019server", "2022server", "2025server"]:
+            warnings.warn(
+                f"Unsupported Windows version ({__my_distro_ver__}). ONNX Runtime supports Windows 10 and above, or Windows Server 2016 and above."
+            )
+    elif __my_system__ == "linux":
+        """Although the 'platform' python module for getting Distro information works well on standard OS images
+        running on real hardware, it is not accurate when running on Azure VMs, Git Bash, Cygwin, etc.
+        The returned values for release and version are unpredictable for virtualized or emulated environments.
+        /etc/os-release and /etc/lsb_release files, on the other hand, are guaranteed to exist and have standard values
+        in all OSes supported by onnxruntime. The former is the current standard file to check OS info and the latter
+        is its predecessor.
+        """
+        # Newer systems have /etc/os-release with relevant distro info
+        __my_distro__ = linecache.getline(__OS_RELEASE_FILE__, 3)[3:-1]
+        __my_distro_ver__ = linecache.getline(__OS_RELEASE_FILE__, 6)[12:-2]
+
+        # Older systems may have /etc/os-release instead
+        if not __my_distro__:
+            __my_distro__ = linecache.getline(__LSB_RELEASE_FILE__, 1)[11:-1]
+            __my_distro_ver__ = linecache.getline(__LSB_RELEASE_FILE__, 2)[16:-1]
+
+        # Instead of trying to parse distro specific files,
+        # warn the user ONNX Runtime may not work out of the box
+        __my_distro__ = __my_distro__.lower()
+        __my_distro_ver__ = __my_distro_ver__.lower()
+    elif __my_system__ == "darwin":
+        __my_distro__ = __my_system__
+        __my_distro_ver__ = platform.release().lower()
+
+        if int(__my_distro_ver__.split(".")[0]) < 11:
+            warnings.warn(
+                f"Unsupported macOS version ({__my_distro_ver__}). ONNX Runtime supports macOS 11.0 or later."
+            )
+    elif __my_system__ == "aix":
+        import subprocess  # noqa: PLC0415
+
+        returned_output = subprocess.check_output("oslevel")
+        __my_distro_ver__str = returned_output.decode("utf-8")
+        __my_distro_ver = __my_distro_ver__str[:3]
+    else:
+        warnings.warn(
+            f"Unsupported platform ({__my_system__}). ONNX Runtime supports Linux, macOS, AIX and Windows platforms, only."
+        )
+
+
+def get_package_name_and_version_info():
+    package_name = ""
+    version = ""
+    cuda_version = ""
+
+    try:
+        from .build_and_package_info import __version__ as version  # noqa: PLC0415
+        from .build_and_package_info import package_name  # noqa: PLC0415
+
+        try:  # noqa: SIM105
+            from .build_and_package_info import cuda_version  # noqa: PLC0415
+        except ImportError:
+            # cuda_version is optional. For example, cpu only package does not have the attribute.
+            pass
+    except Exception as e:
+        warnings.warn("WARNING: failed to collect package name and version info")
+        print(e)
+
+    return package_name, version, cuda_version
+
+
+def check_training_module():
+    import_ortmodule_exception = None
+
+    has_ortmodule = False
+    try:
+        from onnxruntime.training.ortmodule import ORTModule  # noqa: F401, PLC0415
+
+        has_ortmodule = True
+    except ImportError:
+        # ORTModule not present
+        has_ortmodule = False
+    except Exception as e:
+        # this may happen if Cuda is not installed, we want to raise it after
+        # for any exception other than not having ortmodule, we want to continue
+        # device version validation and raise the exception after.
+        try:
+            from onnxruntime.training.ortmodule._fallback import ORTModuleInitException  # noqa: PLC0415
+
+            if isinstance(e, ORTModuleInitException):
+                # ORTModule is present but not ready to run yet
+                has_ortmodule = True
+        except Exception:
+            # ORTModule not present
+            has_ortmodule = False
+
+        if not has_ortmodule:
+            import_ortmodule_exception = e
+
+    # collect onnxruntime package name, version, and cuda version
+    package_name, version, cuda_version = get_package_name_and_version_info()
+
+    if has_ortmodule and cuda_version:
+        try:
+            # collect cuda library build info. the library info may not be available
+            # when the build environment has none or multiple libraries installed
+            try:
+                from .build_and_package_info import cudart_version  # noqa: PLC0415
+            except ImportError:
+                warnings.warn("WARNING: failed to get cudart_version from onnxruntime build info.")
+                cudart_version = None
+
+            def print_build_package_info():
+                warnings.warn(f"onnxruntime training package info: package_name: {package_name}")
+                warnings.warn(f"onnxruntime training package info: __version__: {version}")
+                warnings.warn(f"onnxruntime training package info: cuda_version: {cuda_version}")
+                warnings.warn(f"onnxruntime build info: cudart_version: {cudart_version}")
+
+            # collection cuda library info from current environment.
+            from onnxruntime.capi.onnxruntime_collect_build_info import find_cudart_versions  # noqa: PLC0415
+
+            local_cudart_versions = find_cudart_versions(build_env=False, build_cuda_version=cuda_version)
+            if cudart_version and local_cudart_versions and cudart_version not in local_cudart_versions:
+                print_build_package_info()
+                warnings.warn("WARNING: failed to find cudart version that matches onnxruntime build info")
+                warnings.warn(f"WARNING: found cudart versions: {local_cudart_versions}")
+        except Exception as e:
+            warnings.warn("WARNING: failed to collect onnxruntime version and build info")
+            print(e)
+
+    if import_ortmodule_exception:
+        raise import_ortmodule_exception
+
+    return has_ortmodule, package_name, version, cuda_version

onnxruntime/capi/openvino.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7040243fc8037d65482f974ae2dd1a99394b4fa522e1be05616ef5881f8737ec
+size 16048120

onnxruntime/capi/openvino_auto_batch_plugin.dll

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db26036e4f1335f5ed7b9da3e334f159dc28b79496c2e8cdfe0455fa04121c72
+size 317944