Artem Plastinkin Claude Sonnet 5.5 commited on
Commit ·
b562ea5
1
Parent(s): 1c715bb
Add ONNX Runtime Renesas EP v2.1.0 benchmarks
Browse filesLatency, NPU/CPU split, node distribution and accuracy for 1 and 12 AI cores
from ORT_Renesas_EP_Test_v2.1.0 (MWMX 2.2 ENG4); update existing ORT results in place.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
- README.md +30 -0
- fp32/benchmarks/x5h_ort_cpu_fp32.yaml +51 -0
- int8/benchmarks/x5h_ort_npu_12core.yaml +65 -0
- int8/benchmarks/x5h_ort_npu_1core.yaml +66 -0
README.md
CHANGED
|
@@ -66,6 +66,36 @@ TBD — not yet measured/published for this repo.
|
|
| 66 |
|
| 67 |
---
|
| 68 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
## Runtime Details
|
| 70 |
|
| 71 |
### MWMX Runtime
|
|
|
|
| 66 |
|
| 67 |
---
|
| 68 |
|
| 69 |
+
<!-- ORT-V210:BEGIN -->
|
| 70 |
+
### ONNX Runtime – Renesas EP v2.1.0 (MWMX 2.2 ENG4)
|
| 71 |
+
|
| 72 |
+
Measured on **R-Car X5H** with ONNX Runtime + Renesas Execution Provider (INT8 auto-cast); nodes the NPU cannot run fall back to the CPU EP. Batch size 1, 1 NPU. Source: ORT Renesas EP test report, status 2026-09-30. This section supersedes earlier ORT figures in this card.
|
| 73 |
+
|
| 74 |
+
| Component | AI Cores | Latency (ms) | Throughput (fps) | NPU Inference (ms) | NPU % | CPU + Overhead % | Portable pkg NPU-only (ms) |
|
| 75 |
+
| :-- | :-: | :-: | :-: | :-: | :-: | :-: | :-: |
|
| 76 |
+
| Full model | 1 | 2.114 | 473 | 1.476 | 69.8 % | 30.2 % | 1.48359 |
|
| 77 |
+
| Full model | 12 | 1.729 | 578.4 | 0.799 | 46.2 % | 53.8 % | 0.692726 |
|
| 78 |
+
|
| 79 |
+
> 12-core latency is *not* always lower than 1-core: small models are dominated by CPU-side overhead.
|
| 80 |
+
|
| 81 |
+
**Graph partitioning (NPU vs CPU nodes)**
|
| 82 |
+
|
| 83 |
+
| Component | Total Nodes | NPU Nodes | CPU Nodes (Q/DQ inserted) | NPU % | CPU % |
|
| 84 |
+
| :-- | :-: | :-: | :-: | :-: | :-: |
|
| 85 |
+
| Full model | 50 | 49 | 1 (1) | 98.0 % | 2.0 % |
|
| 86 |
+
|
| 87 |
+
**Accuracy (ORT)**
|
| 88 |
+
|
| 89 |
+
| Runtime / Precision | Result |
|
| 90 |
+
| :-- | :-- |
|
| 91 |
+
| CPU EP · FP32 reference | Top-1 68.5 % · Top-5 90.1 % |
|
| 92 |
+
| Renesas EP + CPU EP · INT8 · v2.1.0 | Top-1 71.4 % · Top-5 89.7 % |
|
| 93 |
+
|
| 94 |
+
> Evaluated on a subset (1k images), not the full validation set.
|
| 95 |
+
<!-- ORT-V210:END -->
|
| 96 |
+
|
| 97 |
+
---
|
| 98 |
+
|
| 99 |
## Runtime Details
|
| 100 |
|
| 101 |
### MWMX Runtime
|
fp32/benchmarks/x5h_ort_cpu_fp32.yaml
ADDED
|
@@ -0,0 +1,51 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# FP32 reference accuracy: ONNX Runtime CPU EP (ORT_Renesas_EP_Test_v2.1.0.xlsx, 'Accuracy ( CPU fp32 )')
|
| 2 |
+
hardware:
|
| 3 |
+
vendor: renesas
|
| 4 |
+
chip: rcar-x5h
|
| 5 |
+
cpu: arm-cortex-a720
|
| 6 |
+
npu: npx6-48k
|
| 7 |
+
npu_count: 2
|
| 8 |
+
npu_cores: 12
|
| 9 |
+
npu_default_freq_mhz: 1066
|
| 10 |
+
accelerator:
|
| 11 |
+
- npu
|
| 12 |
+
|
| 13 |
+
runtime:
|
| 14 |
+
engine: onnxruntime
|
| 15 |
+
toolchain_version: "onnxruntime CPU EP"
|
| 16 |
+
format: onnx
|
| 17 |
+
execution_provider: cpu
|
| 18 |
+
execution_precision: fp32
|
| 19 |
+
|
| 20 |
+
configuration:
|
| 21 |
+
npu_instances: null
|
| 22 |
+
npu_cores_per_instance: null
|
| 23 |
+
npu_freq_mhz: null
|
| 24 |
+
|
| 25 |
+
benchmark:
|
| 26 |
+
type: reference
|
| 27 |
+
parameters:
|
| 28 |
+
batch_size: 1
|
| 29 |
+
input_resolution: [1, 3, 224, 224]
|
| 30 |
+
|
| 31 |
+
performance:
|
| 32 |
+
fps: null
|
| 33 |
+
latency: null
|
| 34 |
+
|
| 35 |
+
metrics:
|
| 36 |
+
accuracy: 0.685
|
| 37 |
+
top5_accuracy: 0.901
|
| 38 |
+
|
| 39 |
+
memory:
|
| 40 |
+
peak_mb: null
|
| 41 |
+
|
| 42 |
+
power:
|
| 43 |
+
avg_w: null
|
| 44 |
+
|
| 45 |
+
reproduce:
|
| 46 |
+
steps:
|
| 47 |
+
- title: Reproduction steps coming soon
|
| 48 |
+
command: >-
|
| 49 |
+
Step-by-step ONNX Runtime reproduction instructions are coming soon.
|
| 50 |
+
In the meantime, please contact your local Renesas representative.
|
| 51 |
+
kind: note
|
int8/benchmarks/x5h_ort_npu_12core.yaml
ADDED
|
@@ -0,0 +1,65 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ONNX Runtime + Renesas EP v2.1.0 (MWMX 2.2 ENG4); source: ORT_Renesas_EP_Test_v2.1.0.xlsx (status 2026-09-30)
|
| 2 |
+
# model/config used for the EP run: https://github.com/onnx/models/blob/main/validated/vision/classification/resnet/model/resnet18-v1-7.onnx
|
| 3 |
+
hardware:
|
| 4 |
+
vendor: renesas
|
| 5 |
+
chip: rcar-x5h
|
| 6 |
+
cpu: arm-cortex-a720
|
| 7 |
+
npu: npx6-48k
|
| 8 |
+
npu_count: 2
|
| 9 |
+
npu_cores: 12
|
| 10 |
+
npu_default_freq_mhz: 1066
|
| 11 |
+
accelerator:
|
| 12 |
+
- npu
|
| 13 |
+
|
| 14 |
+
runtime:
|
| 15 |
+
engine: onnxruntime # ONNX Runtime with Renesas Execution Provider (+ CPU EP for non-NPU nodes)
|
| 16 |
+
toolchain_version: "v2.1.0 renesas" # Renesas EP v2.1.0 (MWMX 2.2 ENG4)
|
| 17 |
+
format: onnx
|
| 18 |
+
execution_provider: npu
|
| 19 |
+
execution_precision: int8 # FP32 model is auto-cast to INT8 by the Renesas EP
|
| 20 |
+
|
| 21 |
+
configuration:
|
| 22 |
+
npu_instances: 1
|
| 23 |
+
npu_cores_per_instance: 12
|
| 24 |
+
npu_freq_mhz: 850 # not stated in the workbook; same 850 MHz as the MWMX and earlier ORT runs
|
| 25 |
+
npu_offload_pct: 98.0 # share of graph nodes executed on NPU (read by AI-Dashboard)
|
| 26 |
+
cpu_offload_pct: 2.0 # share of graph nodes falling back to CPU EP
|
| 27 |
+
|
| 28 |
+
benchmark:
|
| 29 |
+
type: hil
|
| 30 |
+
parameters:
|
| 31 |
+
batch_size: 1
|
| 32 |
+
input_resolution: [1, 3, 224, 224]
|
| 33 |
+
|
| 34 |
+
performance:
|
| 35 |
+
latency: 1.729
|
| 36 |
+
fps: 578.4
|
| 37 |
+
npu_inference_ms: 0.799
|
| 38 |
+
portable_pkg_npu_ms: 0.6927
|
| 39 |
+
|
| 40 |
+
metrics: # accuracy is a single measurement, independent of core count
|
| 41 |
+
accuracy: 0.714
|
| 42 |
+
top5_accuracy: 0.897
|
| 43 |
+
eval_subset: "1k images" # not the full validation set
|
| 44 |
+
|
| 45 |
+
node_distribution: # partitioning of the graph between NPU and CPU EP
|
| 46 |
+
total: 50
|
| 47 |
+
npu: 49
|
| 48 |
+
cpu: 1
|
| 49 |
+
qdq_inserted: 1
|
| 50 |
+
dispatched_to_cpu_compile: null
|
| 51 |
+
dispatched_to_cpu_accuracy: null
|
| 52 |
+
|
| 53 |
+
memory:
|
| 54 |
+
peak_mb: null
|
| 55 |
+
|
| 56 |
+
power:
|
| 57 |
+
avg_w: null
|
| 58 |
+
|
| 59 |
+
reproduce:
|
| 60 |
+
steps:
|
| 61 |
+
- title: Reproduction steps coming soon
|
| 62 |
+
command: >-
|
| 63 |
+
Step-by-step ONNX Runtime reproduction instructions are coming soon.
|
| 64 |
+
In the meantime, please contact your local Renesas representative.
|
| 65 |
+
kind: note
|
int8/benchmarks/x5h_ort_npu_1core.yaml
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ONNX Runtime + Renesas EP v2.1.0 (MWMX 2.2 ENG4); source: ORT_Renesas_EP_Test_v2.1.0.xlsx (status 2026-09-30)
|
| 2 |
+
# model/config used for the EP run: https://github.com/onnx/models/blob/main/validated/vision/classification/resnet/model/resnet18-v1-7.onnx
|
| 3 |
+
hardware:
|
| 4 |
+
vendor: renesas
|
| 5 |
+
chip: rcar-x5h
|
| 6 |
+
cpu: arm-cortex-a720
|
| 7 |
+
npu: npx6-48k
|
| 8 |
+
npu_count: 2
|
| 9 |
+
npu_cores: 12
|
| 10 |
+
npu_default_freq_mhz: 1066
|
| 11 |
+
accelerator:
|
| 12 |
+
- npu
|
| 13 |
+
|
| 14 |
+
runtime:
|
| 15 |
+
engine: onnxruntime # ONNX Runtime with Renesas Execution Provider (+ CPU EP for non-NPU nodes)
|
| 16 |
+
toolchain_version: "v2.1.0 renesas" # Renesas EP v2.1.0 (MWMX 2.2 ENG4)
|
| 17 |
+
format: onnx
|
| 18 |
+
execution_provider: npu
|
| 19 |
+
execution_precision: int8 # FP32 model is auto-cast to INT8 by the Renesas EP
|
| 20 |
+
|
| 21 |
+
configuration:
|
| 22 |
+
npu_instances: 1
|
| 23 |
+
npu_cores_per_instance: 1
|
| 24 |
+
npu_freq_mhz: 850 # not stated in the workbook; same 850 MHz as the MWMX and earlier ORT runs
|
| 25 |
+
npu_offload_pct: 98.0 # share of graph nodes executed on NPU (read by AI-Dashboard)
|
| 26 |
+
cpu_offload_pct: 2.0 # share of graph nodes falling back to CPU EP
|
| 27 |
+
|
| 28 |
+
benchmark:
|
| 29 |
+
type: hil
|
| 30 |
+
parameters:
|
| 31 |
+
batch_size: 1
|
| 32 |
+
input_resolution: [1, 3, 224, 224]
|
| 33 |
+
|
| 34 |
+
performance:
|
| 35 |
+
latency: 2.114
|
| 36 |
+
fps: 473.0
|
| 37 |
+
npu_inference_ms: 1.476
|
| 38 |
+
portable_pkg_npu_ms: 1.4836
|
| 39 |
+
cpu_inference_overhead_ms: 0.053
|
| 40 |
+
|
| 41 |
+
metrics: # accuracy is a single measurement, independent of core count
|
| 42 |
+
accuracy: 0.714
|
| 43 |
+
top5_accuracy: 0.897
|
| 44 |
+
eval_subset: "1k images" # not the full validation set
|
| 45 |
+
|
| 46 |
+
node_distribution: # partitioning of the graph between NPU and CPU EP
|
| 47 |
+
total: 50
|
| 48 |
+
npu: 49
|
| 49 |
+
cpu: 1
|
| 50 |
+
qdq_inserted: 1
|
| 51 |
+
dispatched_to_cpu_compile: null
|
| 52 |
+
dispatched_to_cpu_accuracy: null
|
| 53 |
+
|
| 54 |
+
memory:
|
| 55 |
+
peak_mb: null
|
| 56 |
+
|
| 57 |
+
power:
|
| 58 |
+
avg_w: null
|
| 59 |
+
|
| 60 |
+
reproduce:
|
| 61 |
+
steps:
|
| 62 |
+
- title: Reproduction steps coming soon
|
| 63 |
+
command: >-
|
| 64 |
+
Step-by-step ONNX Runtime reproduction instructions are coming soon.
|
| 65 |
+
In the meantime, please contact your local Renesas representative.
|
| 66 |
+
kind: note
|