---
license: apache-2.0
base_model: []
pipeline_tag: video-classification
tags:
- video-classification
- action-recognition
- optical-flow
- computer-vision
- renesas
- x5h
- onnx
- resnet
- resnet18
---

# ResNet18-OpticalFlow (ONNX) – Renesas X5H

> ⏳ **Model file not yet uploaded.** Benchmark results on this page were published ahead of the model weights — see **Provided Artifacts** below. Download/deployment steps will not work until the file is added to this repository.

## Introduction

This repository hosts a **ResNet18-based 2D optical-flow action-recognition model**, targeting the
**Renesas R-Car X5H** platform for inference on the NPX6 NPU.

- **Model Architecture:** ResNet18 backbone operating on stacked 2D optical-flow frames for video
  action recognition
- **Source Model:** internal export `resnet18_2d_of_hmdb5_32_{xavier,a100}`
  *(no HuggingFace mirror of these weights; see `model.source` in `.metadata.yaml`)*
- **Task:** Video Classification / Action Recognition (2D optical-flow input stream)
- **Dataset:** Likely **HMDB51** — the `hmdb5` fragment in the checkpoint names is almost certainly
  a truncation of "hmdb51"; confirm before publishing
- **Artifacts:** Two artifacts are provided — `xavier-export` and `a100-export`. Their latencies on
  the X5H NPU are nearly identical (see Performance below), and the checkpoint names differ only by
  a reference-platform suffix (`_xavier` vs `_a100`). This strongly suggests **both artifacts are
  the same trained checkpoint**, exported/compiled with a different reference target platform
  (NVIDIA Jetson Xavier vs. NVIDIA A100) recorded in the source pipeline metadata — this is an
  observation based on the naming and near-identical performance, not a confirmed fact; the two
  weight files have not been diffed.

## Deployment Flow

The FP32 ONNX model for each artifact is auto-cast to **INT8** by the Renesas MWMX toolchain at
compile time — no separate quantization step is required.

```
resnet18_2d_of_hmdb5_32_{xavier,a100}.onnx (FP32)
        │
        └─▶  MWMX Runtime  ──▶  INT8 auto-cast  ──▶  NPX6 NPU
```

## Provided Artifacts

| Artifact | Parameters | Status | Notes |
|----------|-----------|--------|-------|
| **xavier-export (FP32 ONNX)** | ~11.7M *(approximate — based on the standard ResNet18 backbone; the optical-flow input adaptation may shift this slightly but not significantly)* | ⏳ Not yet uploaded | Checkpoint `resnet18_2d_of_hmdb5_32_xavier`; benchmark numbers below exist, the model file has not been published to this repo yet |
| **a100-export (FP32 ONNX)** | ~11.7M *(approximate — based on the standard ResNet18 backbone; the optical-flow input adaptation may shift this slightly but not significantly)* | ⏳ Not yet uploaded | Checkpoint `resnet18_2d_of_hmdb5_32_a100`; benchmark numbers below exist, the model file has not been published to this repo yet |

## Performance

Measured on **Renesas R-Car X5H** via the MWMX runtime (APM50 ship-performance CI pipeline).

> **Benchmark configuration:** Single NPU · Batch size: 1 · Input resolution: not available from source data — TBD
>
> Latencies for the two artifacts are nearly identical, consistent with them being the same
> underlying trained model (see the artifact note above).

| Artifact | Runtime | Precision | Device | Latency (ms) | Type |
|----------|---------|-----------|--------|---------------|------|
| xavier-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 1 Core · 850 MHz | 2.163819 | Measured |
| xavier-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 12 Cores · 850 MHz | 1.463519 | Measured |
| a100-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 1 Core · 850 MHz | 2.163902 | Measured |
| a100-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 12 Cores · 850 MHz | 1.452456 | Measured |

### Accuracy

TBD — not yet measured/published for this repo.

---

## Runtime Details

### MWMX Runtime

- **Engine:** Renesas MWMX (Middleware MX) native inference runtime
- **Input format:** FP32 ONNX (compiled by the MWMX toolchain)
- **NPU execution precision:** INT8 (auto-cast by MWMX toolchain)
- **Execution target:** NPX6-48K NPU on R-Car X5H

---

## Prerequisites

To run inference on Renesas R-Car X5H, you need:

1. **Renesas R-Car X5H board** with NPX6 NPU
2. **Renesas MWMX Runtime**
3. **Hugging Face CLI** to download the model (once the model files are published)

## Download

TBD — model files not yet published to this repository.

---

## Benchmark Methodology

- **HIL runs:** Hardware-in-the-loop — measured on physical R-Car X5H silicon via the MWMX
  runtime (`metawaremx_runtime` CI pipeline, "APM50" ship-performance target)
- **Precision:** FP32 ONNX input; INT8 execution (auto-cast by MWMX)
- **Slices:** results reported for both 1 AI core and 12 AI cores per NPU instance, for both artifacts