| ---
|
| license: apache-2.0
|
| base_model: []
|
| pipeline_tag: video-classification
|
| tags:
|
| - video-classification
|
| - action-recognition
|
| - optical-flow
|
| - computer-vision
|
| - renesas
|
| - x5h
|
| - onnx
|
| - resnet
|
| - resnet18
|
| ---
|
|
|
| # ResNet18-OpticalFlow (ONNX) – Renesas X5H
|
|
|
| > ⏳ **Model file not yet uploaded.** Benchmark results on this page were published ahead of the model weights — see **Provided Artifacts** below. Download/deployment steps will not work until the file is added to this repository.
|
|
|
| ## Introduction
|
|
|
| This repository hosts a **ResNet18-based 2D optical-flow action-recognition model**, targeting the
|
| **Renesas R-Car X5H** platform for inference on the NPX6 NPU.
|
|
|
| - **Model Architecture:** ResNet18 backbone operating on stacked 2D optical-flow frames for video
|
| action recognition
|
| - **Source Model:** internal export `resnet18_2d_of_hmdb5_32_{xavier,a100}`
|
| *(no HuggingFace mirror of these weights; see `model.source` in `.metadata.yaml`)*
|
| - **Task:** Video Classification / Action Recognition (2D optical-flow input stream)
|
| - **Dataset:** Likely **HMDB51** — the `hmdb5` fragment in the checkpoint names is almost certainly
|
| a truncation of "hmdb51"; confirm before publishing
|
| - **Artifacts:** Two artifacts are provided — `xavier-export` and `a100-export`. Their latencies on
|
| the X5H NPU are nearly identical (see Performance below), and the checkpoint names differ only by
|
| a reference-platform suffix (`_xavier` vs `_a100`). This strongly suggests **both artifacts are
|
| the same trained checkpoint**, exported/compiled with a different reference target platform
|
| (NVIDIA Jetson Xavier vs. NVIDIA A100) recorded in the source pipeline metadata — this is an
|
| observation based on the naming and near-identical performance, not a confirmed fact; the two
|
| weight files have not been diffed.
|
|
|
| ## Deployment Flow
|
|
|
| The FP32 ONNX model for each artifact is auto-cast to **INT8** by the Renesas MWMX toolchain at
|
| compile time — no separate quantization step is required.
|
|
|
| ```
|
| resnet18_2d_of_hmdb5_32_{xavier,a100}.onnx (FP32)
|
| │
|
| └─▶ MWMX Runtime ──▶ INT8 auto-cast ──▶ NPX6 NPU
|
| ```
|
|
|
| ## Provided Artifacts
|
|
|
| | Artifact | Parameters | Status | Notes |
|
| |----------|-----------|--------|-------|
|
| | **xavier-export (FP32 ONNX)** | ~11.7M *(approximate — based on the standard ResNet18 backbone; the optical-flow input adaptation may shift this slightly but not significantly)* | ⏳ Not yet uploaded | Checkpoint `resnet18_2d_of_hmdb5_32_xavier`; benchmark numbers below exist, the model file has not been published to this repo yet |
|
| | **a100-export (FP32 ONNX)** | ~11.7M *(approximate — based on the standard ResNet18 backbone; the optical-flow input adaptation may shift this slightly but not significantly)* | ⏳ Not yet uploaded | Checkpoint `resnet18_2d_of_hmdb5_32_a100`; benchmark numbers below exist, the model file has not been published to this repo yet |
|
|
|
| ## Performance
|
|
|
| Measured on **Renesas R-Car X5H** via the MWMX runtime (APM50 ship-performance CI pipeline).
|
|
|
| > **Benchmark configuration:** Single NPU · Batch size: 1 · Input resolution: not available from source data — TBD
|
| >
|
| > Latencies for the two artifacts are nearly identical, consistent with them being the same
|
| > underlying trained model (see the artifact note above).
|
|
|
| | Artifact | Runtime | Precision | Device | Latency (ms) | Type |
|
| |----------|---------|-----------|--------|---------------|------|
|
| | xavier-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 1 Core · 850 MHz | 2.163819 | Measured |
|
| | xavier-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 12 Cores · 850 MHz | 1.463519 | Measured |
|
| | a100-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 1 Core · 850 MHz | 2.163902 | Measured |
|
| | a100-export | MWMX Runtime | INT8 (auto) | X5H · 1× NPU · 12 Cores · 850 MHz | 1.452456 | Measured |
|
|
|
| ### Accuracy
|
|
|
| TBD — not yet measured/published for this repo.
|
|
|
| ---
|
|
|
| ## Runtime Details
|
|
|
| ### MWMX Runtime
|
|
|
| - **Engine:** Renesas MWMX (Middleware MX) native inference runtime
|
| - **Input format:** FP32 ONNX (compiled by the MWMX toolchain)
|
| - **NPU execution precision:** INT8 (auto-cast by MWMX toolchain)
|
| - **Execution target:** NPX6-48K NPU on R-Car X5H
|
|
|
| ---
|
|
|
| ## Prerequisites
|
|
|
| To run inference on Renesas R-Car X5H, you need:
|
|
|
| 1. **Renesas R-Car X5H board** with NPX6 NPU
|
| 2. **Renesas MWMX Runtime**
|
| 3. **Hugging Face CLI** to download the model (once the model files are published)
|
|
|
| ## Download
|
|
|
| TBD — model files not yet published to this repository.
|
|
|
| ---
|
|
|
| ## Benchmark Methodology
|
|
|
| - **HIL runs:** Hardware-in-the-loop — measured on physical R-Car X5H silicon via the MWMX
|
| runtime (`metawaremx_runtime` CI pipeline, "APM50" ship-performance target)
|
| - **Precision:** FP32 ONNX input; INT8 execution (auto-cast by MWMX)
|
| - **Slices:** results reported for both 1 AI core and 12 AI cores per NPU instance, for both artifacts
|
|
|
