README.md

# ESRGAN Inference (C++)

A C++ command-line tool for running ESRGAN (Enhanced Super-Resolution Generative Adversarial Networks) inference using ONNX Runtime with support for both CPU and AMD NPU (Neural Processing Unit) execution via VitisAI Execution Provider.

## Overview

This tool performs image super-resolution using pre-trained ESRGAN models, taking low-resolution images as input and producing high-resolution upscaled outputs. It supports execution on both CPU and AMD Ryzen AI NPUs for accelerated inference.

## Prerequisites

### Required Software
- **Windows 10/11** (x64 architecture)
- **Visual Studio 2019/2022** with C++17 support
- **CMake 3.5** or higher
- **AMD Ryzen AI SDK 1.4** (for NPU support)
- **OpenCV 4.12** (included in project at `./opencv/build`)

### Required Files
- **ONNX Model**: ESRGAN model file (`esrgan_fp32_qdq.onnx`)
- **Config JSON**: VitisAI configuration file (`esrgan_config.json`)
- **Runtime DLLs**: Ryzen AI runtime libraries (automatically copied from Ryzen AI SDK during build)
- **XCLBIN Files**: NPU binary files (automatically copied from Ryzen AI SDK during build)
- **OpenCV Dependencies**: OpenCV DLLs (included in `opencv/build/x64/vc16/bin/`)

### Hardware Requirements
- For NPU acceleration: AMD Ryzen AI compatible processor (Phoenix 0x1503 or Strix 0x17F0)
- Minimum 8GB RAM recommended
- Sufficient disk space for models and output images

## Building the Project

The build process automatically copies all required dependencies to the output directory, making the executable completely self-contained for distribution.

### Prerequisites for Building
- Ryzen AI SDK 1.4 must be installed and the `RYZEN_AI_INSTALLATION_PATH` environment variable set
- Visual Studio 2019/2022 with C++17 support
- CMake 3.5 or higher

### Option 1: Using the Build Script (Recommended)
1. Set the Ryzen AI SDK environment variable:
   ```cmd
   set RYZEN_AI_INSTALLATION_PATH=C:\Program Files\RyzenAI\1.4.0
   ```

2. Run the build script:
   ```cmd
   compile.bat
   ```

### Option 2: Manual Build
1. Set the environment variable:
   ```cmd
   set RYZEN_AI_INSTALLATION_PATH=C:\Program Files\RyzenAI\1.4.0
   ```

2. Generate build files with CMake:
   ```cmd
   cmake -DCMAKE_CONFIGURATION_TYPES=Release -A x64 -T host=x64 -B build -S . -G "Visual Studio 17 2022"
   ```

3. Build the project:
   ```cmd
   cmake --build .\build --config Release
   ```

## Files Automatically Copied During Build

The CMakeLists.txt configuration automatically copies the following files to `build/Release/`, creating a fully self-contained executable directory:

### Model and Configuration Files
- **esrgan_config.json** - VitisAI configuration file
- **esrgan_fp32_qdq.onnx** - ONNX model file

### NPU Binary Files
- **xclbins/** directory structure containing NPU binaries for different device types:
  - Phoenix NPUs (device ID: 0x1503)  
  - Strix NPUs (device ID: 0x17F0)
  - Copied from: `${RYZEN_AI_INSTALLATION_PATH}/voe-4.0-win_amd64/xclbins/`

### Runtime Dependencies  
- **OpenCV DLLs** from `opencv/build/x64/vc16/bin/` (including `opencv_world412.dll`)
- **Ryzen AI DLLs** from local `RAI_dll/` directory:
  - `onnxruntime.dll` - Core ONNX Runtime engine
  - `onnxruntime_providers_vitisai.dll` - VitisAI execution provider
  - `onnxruntime_vitisai_ep.dll` - VitisAI EP interface layer
  - `onnxruntime_vitis_ai_custom_ops.dll` - Custom operations support
  - `DirectML.dll` - DirectML runtime support
  - `xclbin.dll` - XCLBIN file handling utilities
  - `transaction.dll` - Transaction management
  - `dyn_dispatch_core.dll` - Dynamic dispatch core
- **VitisAI Runtime** files from `${RYZEN_AI_INSTALLATION_PATH}/deployment/voe`


## Project Structure

### Source Files
```
src/
├── main.cpp           # Main application logic
├── npu_util.cpp       # NPU utility functions  
├── npu_util.h         # NPU utility headers
└── cxxopts.hpp        # Command-line argument parsing (header-only library)
```

### Project Root
```
./
├── bird_input.png             # Sample input image
├── CMakeLists.txt             # Build configuration
├── compile.bat                # Build script for Windows
├── esrgan_config.json         # VitisAI EP configuration
├── esrgan_fp32_qdq.onnx      # ONNX model file
├── esrgan_cache/              # VitisAI compilation cache (generated after first run)
├── opencv/                    # Local OpenCV installation
│   └── build/                 # Pre-built OpenCV binaries and headers
└── src/                       # Source code directory
```

### After Build (`build/Release/`)
```
build/Release/
├── esrgan_inference.exe       # Main executable
├── esrgan_fp32_qdq.onnx      # ONNX model (copied from root)
├── esrgan_config.json         # Configuration (copied from root)
├── opencv_world412.dll        # OpenCV runtime (from opencv/build/x64/vc16/bin/)
├── onnxruntime.dll            # ONNX Runtime core (from RAI_dll/)
├── onnxruntime_providers_vitisai.dll     # VitisAI provider (from RAI_dll/)
├── onnxruntime_vitisai_ep.dll           # VitisAI EP interface (from RAI_dll/)
├── onnxruntime_vitis_ai_custom_ops.dll  # Custom ops support (from RAI_dll/)
├── DirectML.dll               # DirectML runtime (from RAI_dll/)
├── xclbin.dll                 # XCLBIN utilities (from RAI_dll/)
├── transaction.dll            # Transaction management (from RAI_dll/)
├── dyn_dispatch_core.dll      # Dynamic dispatch (from RAI_dll/)
├── [additional VitisAI files] # Other runtime components from SDK deployment
└── xclbins/                   # NPU binaries (from Ryzen AI SDK)
    ├── phoenix/               # Phoenix NPU binaries (device ID: 0x1503)
    └── strix/                 # Strix NPU binaries (device ID: 0x17F0)
```

**Note**: The build output directory is completely self-contained and portable. It can be copied to other compatible systems and run without additional installation requirements.

## Usage

### Command-Line Syntax
```cmd
esrgan_inference.exe [OPTIONS]
```

### Required Arguments
- `-m, --model <file>` : ONNX model filename (relative to executable directory)
- `-c, --config <file>` : JSON configuration filename (relative to executable directory)

### Optional Arguments
- `-i, --input_image <file>` : Input image file (default: `input_image.png`)
- `-o, --output_image <file>` : Output image file (default: `output_image.png`)
- `-k, --cache_key <string>` : Cache key for VitisAI EP (default: empty)
- `-d, --cache_dir <string>` : Cache directory for VitisAI EP (default: empty)
- `-x, --xclbin <file>` : XCLBIN filename for NPU (default: auto-selected)
- `-h, --help` : Show help message

### Example Usage

```cmd
esrgan_inference.exe ^
    -m esrgan_fp32_qdq.onnx ^
    -c esrgan_config.json ^
    -i ..\..\bird_input.png ^
    -o bird_output.png ^
    -d ..\..\esrgan_cache ^
    -k esrgan_cache
```

This example demonstrates:
- Using model and config files from the build directory (automatically copied)
- Relative paths for input images (from project root)
- Using the existing VitisAI cache directory for faster subsequent runs

**Note**: The `esrgan_cache/` directory is created automatically during the first NPU inference run. It contains compiled model artifacts that significantly speed up subsequent runs. You can point to this existing cache using the `-d` and `-k` parameters.


## Console Output Example

```
-------------------------------------------------------
Configuration Parameters:
-------------------------------------------------------
Executable Directory: C:\Users\user\Desktop\QuickTest\ESRGAN_Inference_cpp\build\Release
Model Path         : C:\Users\user\Desktop\QuickTest\ESRGAN_Inference_cpp\build\Release\esrgan_fp32_qdq.onnx
Config JSON Path   : C:\Users\user\Desktop\QuickTest\ESRGAN_Inference_cpp\build\Release\esrgan_config.json
Cache Key         : esrgan_cache
Cache Directory    : ..\..\
Input image : ..\..\bird_input.png
Output image : bird_output.png
-------------------------------------------------------

[INFO] Model input name: input
[INFO] Input data type: 1
[INFO] Input dims: [ 1 250 250 3 ]
[INFO] Model output name: output
[INFO] Output data type: 1
[INFO] Output dims: [ 1 1000 1000 3 ]
[INFO] Running inference...

-------------------------------------------------------
Performing compatibility check for VitisAI EP 1.4      
-------------------------------------------------------
 - NPU Device ID     : 0x1503
 - NPU Device Name   : AMD IPU Device
 - NPU Driver Version: 32.0.203.257
Environment compatible for VitisAI EP

[INFO] Writing upscaled image to: bird_output.png
[INFO] Done.
```


MIT License - See source files for full license text.

## Dependencies and Attribution

This project includes and uses the following components:

- **AMD Ryzen AI SDK 1.4** - NPU acceleration support
- **ONNX Runtime** - Model inference engine (runtime DLLs included in `RAI_dll/`)
- **OpenCV 4.12** - Image processing (pre-built binaries included in `opencv/build/`)
- **cxxopts** - Command-line argument parsing (header-only, included in `src/`)

**Note**: This project is largely self-contained. The required ONNX Runtime and VitisAI provider DLLs are included in the `RAI_dll/` directory, and OpenCV binaries are included in the `opencv/` directory. The only external dependency is the Ryzen AI SDK installation for XCLBIN files and additional runtime components.

## Source Code References

- **Main Application**: [`src/main.cpp`](src/main.cpp)
- **NPU Utilities**: [`src/npu_util.cpp`](src/npu_util.cpp), [`src/npu_util.h`](src/npu_util.h)  
- **Command-Line Parsing**: [`src/cxxopts.hpp`](src/cxxopts.hpp)
- **Build Configuration**: [`CMakeLists.txt`](CMakeLists.txt)
- **Build Script**: [`compile.bat`](compile.bat)