sebastientaylor commited on
Commit
41d88cd
Β·
verified Β·
1 Parent(s): 43830aa

Update model card for yolov8-det

Browse files
Files changed (1) hide show
  1. README.md +11 -11
README.md CHANGED
@@ -101,7 +101,7 @@ Row conventions in the table below:
101
  - **Decoder variants.** EdgeFirst ships three INT8 split-decoders. The table headlines the accuracy-recovering ones: **smart** (per-tensor rescaling β€” best accuracy, extra CPU ops add some latency) and, where a smart run is absent, **logical** (the latency-optimized default β€” no CPU overhead, slightly lower accuracy). The **combined** decoder is the *standard-quantization baseline* (equivalent to typical single-scale INT8, and how the reference numbers are produced); it loses the most accuracy β€” especially on segmentation, where box/mask dynamic range collapses under one scale β€” so it is published only as a downloadable reference artifact and in the metrics export, never headlined here. Smart and logical exist precisely to recover that loss.
102
  - **Platform-label suffixes.** `(FRDM)` / `(Phytec)` name the NXP i.MX 95 development board a session ran on. `β€” latency` / `β€” throughput` mark the two pipeline configurations the NXP i.MX 95 Neutron and NXP Ara240 targets run: the **latency** pipeline runs inference serially for the lowest per-frame latency; the **throughput** pipeline runs multiple inference workers for the highest FPS, which raises per-call inference time in exchange. Rows with neither suffix run a single pipeline.
103
  - **End-to-end (ms)** is the sequential per-image latency of the compute pipeline β€” preprocess β†’ inference β†’ postprocess. Image acquisition (camera or file load + JPEG decode) overlaps these stages and is excluded from this figure.
104
- - **Realized FPS vs Theoretical FPS.** **Realized FPS** is the *measured* steady-state throughput β€” the rate at which final results are actually delivered over the full validation pipeline. It normally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames, and it is the **true, priority number**. **Theoretical FPS** (shown with a `~`) is the per-stage accelerator ceiling β€” the rate the NPU/DNPU could sustain *if it were the only bottleneck* β€” so it is a **possibly-achievable note, not a claim**. Whether a deployment approaches it depends on the surrounding pipeline, and two levers dominate: (1) **host bottlenecks** β€” these validation runs decode a JPEG per image, whereas a live **camera pipeline** skips that decode and can run closer to the ceiling; and (2) **confidence threshold** β€” validation runs at `0.001` to capture every detection for mAP, which makes NMS/decode heavy, while a deployment threshold of `0.25–0.75` produces far fewer candidate boxes and lighter postprocessing, raising realized FPS toward the ceiling.
105
 
106
  | Size | Platform | mAP@0.5 | Ξ” vs FP32 (pp) | mAP@0.5-0.95 | Inference (ms) | End-to-end (ms) | Realized FPS | Theoretical FPS | Source |
107
  |------|----------|---------|----------------|--------------|----------------|-----------------|--------------|-----------------|--------|
@@ -115,9 +115,9 @@ Row conventions in the table below:
115
  | Nano | NXP i.MX 95 + eIQ Neutron NPU β€” throughput | 48.77% | -1.72 | 34.34% | 45.47 | 73.31 | 76.5 | ~88 | [v-79e](https://edgefirst.studio/public/validation/v-79e/details?mode=charts) |
116
  | Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 47.38% | -3.11 | 32.08% | 44.44 | 59.30 | 21.5 | ~23 | [v-8b1](https://edgefirst.studio/public/validation/v-8b1/details?mode=charts) |
117
  | Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 47.38% | -3.11 | 32.08% | 62.08 | 81.32 | 61.3 | ~64 | [v-8b3](https://edgefirst.studio/public/validation/v-8b3/details?mode=charts) |
118
- | Nano | NXP Ara240 β€” throughput | 45.71% | -4.78 | 31.48% | 7.42 | 47.14 | 225.3 | ~227 | [v-6bb](https://edgefirst.studio/public/validation/v-6bb/details?mode=charts) |
119
- | Nano | NXP Ara240 (FRDM) β€” latency | 46.63% | -3.86 | 32.13% | 6.85 | 43.75 | 214.8 | ~216 | [v-898](https://edgefirst.studio/public/validation/v-898/details?mode=charts) |
120
- | Nano | Raspberry Pi 5 + Hailo-8L NPU | 49.80% | -0.69 | 35.22% | 13.20 | 24.86 | 64.9 | ~152 | [v-8df](https://edgefirst.studio/public/validation/v-8df/details?mode=charts) |
121
  | Nano | NVIDIA Jetson Orin Nano (TensorRT FP16) | 50.50% | +0.01 | 35.79% | 11.93 | 26.90 | 259.7 | ~274 | [v-91a](https://edgefirst.studio/public/validation/v-91a/details?mode=charts) |
122
  | Small | ONNX FP32 (AWS Graviton Β· 48-core) | 59.37% | -0.03 | 43.18% | 74.56 | 83.28 | 51.9 | ~54 | [v-719](https://edgefirst.studio/public/validation/v-719/details?mode=charts) |
123
  | Small | ONNX FP32 (CUDA) | 59.40% | ref | 43.22% | 15.47 | 21.25 | 203.3 | ~259 | [v-6fd](https://edgefirst.studio/public/validation/v-6fd/details?mode=charts) |
@@ -129,9 +129,9 @@ Row conventions in the table below:
129
  | Small | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 58.62% | -0.78 | 42.29% | 39.46 | 74.22 | 23.4 | ~25 | [v-902](https://edgefirst.studio/public/validation/v-902/details?mode=charts) |
130
  | Small | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 58.62% | -0.78 | 42.29% | 146.86 | 182.46 | 26.4 | ~27 | [v-903](https://edgefirst.studio/public/validation/v-903/details?mode=charts) |
131
  | Small | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β€” throughput | 58.62% | -0.78 | 42.29% | 141.60 | 174.37 | 27.4 | ~28 | [v-8f3](https://edgefirst.studio/public/validation/v-8f3/details?mode=charts) |
132
- | Small | NXP Ara240 β€” throughput | 54.79% | -4.61 | 38.18% | 11.59 | 25.06 | 118.5 | ~628 | [v-6bc](https://edgefirst.studio/public/validation/v-6bc/details?mode=charts) |
133
- | Small | NXP Ara240 (FRDM) β€” latency | 56.70% | -2.70 | 39.81% | 10.28 | 24.15 | 142.6 | ~143 | [v-8e4](https://edgefirst.studio/public/validation/v-8e4/details?mode=charts) |
134
- | Small | Raspberry Pi 5 + Hailo-8L NPU | 58.52% | -0.88 | 42.34% | 30.36 | 45.07 | 28.8 | ~66 | [v-8ec](https://edgefirst.studio/public/validation/v-8ec/details?mode=charts) |
135
  | Small | NVIDIA Jetson Orin Nano (TensorRT FP16) | 59.38% | -0.02 | 43.15% | 24.62 | 37.12 | 157.5 | ~162 | [v-922](https://edgefirst.studio/public/validation/v-922/details?mode=charts) |
136
  | Medium | ONNX FP32 (AWS Graviton Β· 48-core) | 64.70% | +0.02 | 48.46% | 181.16 | 189.63 | 21.7 | ~22 | [v-71a](https://edgefirst.studio/public/validation/v-71a/details?mode=charts) |
137
  | Medium | ONNX FP32 (CUDA) | 64.68% | ref | 48.45% | 43.58 | 49.61 | 78.4 | ~92 | [v-701](https://edgefirst.studio/public/validation/v-701/details?mode=charts) |
@@ -144,9 +144,9 @@ Row conventions in the table below:
144
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 63.51% | -1.17 | 47.11% | 101.26 | 135.73 | 9.5 | ~10 | [v-974](https://edgefirst.studio/public/validation/v-974/details?mode=charts) |
145
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 63.51% | -1.17 | 47.11% | 393.79 | 429.57 | 10.0 | ~10 | [v-976](https://edgefirst.studio/public/validation/v-976/details?mode=charts) |
146
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β€” throughput | 63.51% | -1.17 | 47.11% | 387.27 | 419.61 | 10.2 | ~10 | [v-94f](https://edgefirst.studio/public/validation/v-94f/details?mode=charts) |
147
- | Medium | NXP Ara240 β€” throughput | 59.51% | -5.17 | 42.24% | 24.56 | 40.78 | 46.9 | ~326 | [v-797](https://edgefirst.studio/public/validation/v-797/details?mode=charts) |
148
- | Medium | NXP Ara240 (FRDM) β€” latency | 59.99% | -4.69 | 42.66% | 21.49 | 34.36 | 54.3 | ~54 | [v-908](https://edgefirst.studio/public/validation/v-908/details?mode=charts) |
149
- | Medium | Raspberry Pi 5 + Hailo-8L NPU | 63.88% | -0.80 | 47.67% | 56.26 | 71.51 | 16.9 | ~36 | [v-90a](https://edgefirst.studio/public/validation/v-90a/details?mode=charts) |
150
  | Medium | NVIDIA Jetson Orin Nano (TensorRT FP16) | 64.67% | -0.01 | 48.41% | 52.44 | 64.24 | 75.3 | ~76 | [v-92b](https://edgefirst.studio/public/validation/v-92b/details?mode=charts) |
151
 
152
 
@@ -163,7 +163,7 @@ These results are produced by the EdgeFirst on-target validation pipeline:
163
 
164
  Inference latency is reported as the on-accelerator inference time. End-to-end latency is the sequential per-image latency across the compute pipeline β€” preprocessing, inference, and postprocessing; image acquisition (file or camera load and JPEG decode) overlaps these stages and is excluded from this figure.
165
 
166
- Two throughput figures are reported. **Realized FPS** is the *measured* steady-state rate at which final results are emitted, computed from the Perfetto trace as `(N-1) / (t_last - t_first)` over the warmup-trimmed steady-state region; it is the true, priority number and generally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames. **Theoretical FPS** is the per-stage accelerator ceiling (`min_s p_s / t_s` β€” the slowest stage's sustained rate), i.e. the throughput if the accelerator were the *only* bottleneck. It is a possibly-achievable ceiling, not a measured result: reaching it depends on the deployment pipeline. A validation run decodes a JPEG per image and evaluates at a `0.001` confidence threshold (to capture every detection for mAP), both of which load the host and postprocess stages; a production camera pipeline (no JPEG decode) at a deployment threshold of `0.25–0.75` (far fewer candidate boxes through NMS) moves realized throughput toward the theoretical ceiling.
167
 
168
  See [EdgeFirst Studio](https://edgefirst.studio) for the full validation pipeline.
169
 
 
101
  - **Decoder variants.** EdgeFirst ships three INT8 split-decoders. The table headlines the accuracy-recovering ones: **smart** (per-tensor rescaling β€” best accuracy, extra CPU ops add some latency) and, where a smart run is absent, **logical** (the latency-optimized default β€” no CPU overhead, slightly lower accuracy). The **combined** decoder is the *standard-quantization baseline* (equivalent to typical single-scale INT8, and how the reference numbers are produced); it loses the most accuracy β€” especially on segmentation, where box/mask dynamic range collapses under one scale β€” so it is published only as a downloadable reference artifact and in the metrics export, never headlined here. Smart and logical exist precisely to recover that loss.
102
  - **Platform-label suffixes.** `(FRDM)` / `(Phytec)` name the NXP i.MX 95 development board a session ran on. `β€” latency` / `β€” throughput` mark the two pipeline configurations the NXP i.MX 95 Neutron and NXP Ara240 targets run: the **latency** pipeline runs inference serially for the lowest per-frame latency; the **throughput** pipeline runs multiple inference workers for the highest FPS, which raises per-call inference time in exchange. Rows with neither suffix run a single pipeline.
103
  - **End-to-end (ms)** is the sequential per-image latency of the compute pipeline β€” preprocess β†’ inference β†’ postprocess. Image acquisition (camera or file load + JPEG decode) overlaps these stages and is excluded from this figure.
104
+ - **Realized FPS vs Theoretical FPS.** **Realized FPS** is the *measured* steady-state throughput β€” the rate at which final results are actually delivered over the full validation pipeline. It normally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames, and it is the **true, priority number**. **Theoretical FPS** (shown with a `~`) is the accelerator's **core** ceiling β€” `1000 / device-compute-time`, the rate the NPU/DNPU could sustain *if it were the only bottleneck* β€” so it is a **possibly-achievable note, not a claim**. It is read from the isolated device-compute stage, which (unlike the host capture/preprocess stages, whose measured time inflates when the pipeline is backpressured) is stable and load-independent. Whether a deployment approaches it depends on the surrounding pipeline, and two levers dominate: (1) **host bottlenecks** β€” these validation runs decode a JPEG per image, whereas a live **camera pipeline** skips that decode and can run closer to the ceiling; and (2) **confidence threshold** β€” validation runs at `0.001` to capture every detection for mAP, which makes NMS/decode heavy, while a deployment threshold of `0.25–0.75` produces far fewer candidate boxes and lighter postprocessing, raising realized FPS toward the ceiling.
105
 
106
  | Size | Platform | mAP@0.5 | Ξ” vs FP32 (pp) | mAP@0.5-0.95 | Inference (ms) | End-to-end (ms) | Realized FPS | Theoretical FPS | Source |
107
  |------|----------|---------|----------------|--------------|----------------|-----------------|--------------|-----------------|--------|
 
115
  | Nano | NXP i.MX 95 + eIQ Neutron NPU β€” throughput | 48.77% | -1.72 | 34.34% | 45.47 | 73.31 | 76.5 | ~88 | [v-79e](https://edgefirst.studio/public/validation/v-79e/details?mode=charts) |
116
  | Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 47.38% | -3.11 | 32.08% | 44.44 | 59.30 | 21.5 | ~23 | [v-8b1](https://edgefirst.studio/public/validation/v-8b1/details?mode=charts) |
117
  | Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 47.38% | -3.11 | 32.08% | 62.08 | 81.32 | 61.3 | ~64 | [v-8b3](https://edgefirst.studio/public/validation/v-8b3/details?mode=charts) |
118
+ | Nano | NXP Ara240 (FRDM) β€” latency | 46.63% | -3.86 | 32.13% | 6.46 | 18.54 | 127.9 | ~155 | [v-a12](https://edgefirst.studio/public/validation/v-a12/details?mode=charts) |
119
+ | Nano | NXP Ara240 (FRDM) β€” throughput | 46.63% | -3.86 | 32.13% | 6.87 | 49.38 | 210.8 | ~298 | [v-a13](https://edgefirst.studio/public/validation/v-a13/details?mode=charts) |
120
+ | Nano | Raspberry Pi 5 + Hailo-8L NPU | 49.80% | -0.69 | 35.22% | 13.20 | 24.86 | 64.9 | ~76 | [v-8df](https://edgefirst.studio/public/validation/v-8df/details?mode=charts) |
121
  | Nano | NVIDIA Jetson Orin Nano (TensorRT FP16) | 50.50% | +0.01 | 35.79% | 11.93 | 26.90 | 259.7 | ~274 | [v-91a](https://edgefirst.studio/public/validation/v-91a/details?mode=charts) |
122
  | Small | ONNX FP32 (AWS Graviton Β· 48-core) | 59.37% | -0.03 | 43.18% | 74.56 | 83.28 | 51.9 | ~54 | [v-719](https://edgefirst.studio/public/validation/v-719/details?mode=charts) |
123
  | Small | ONNX FP32 (CUDA) | 59.40% | ref | 43.22% | 15.47 | 21.25 | 203.3 | ~259 | [v-6fd](https://edgefirst.studio/public/validation/v-6fd/details?mode=charts) |
 
129
  | Small | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 58.62% | -0.78 | 42.29% | 39.46 | 74.22 | 23.4 | ~25 | [v-902](https://edgefirst.studio/public/validation/v-902/details?mode=charts) |
130
  | Small | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 58.62% | -0.78 | 42.29% | 146.86 | 182.46 | 26.4 | ~27 | [v-903](https://edgefirst.studio/public/validation/v-903/details?mode=charts) |
131
  | Small | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β€” throughput | 58.62% | -0.78 | 42.29% | 141.60 | 174.37 | 27.4 | ~28 | [v-8f3](https://edgefirst.studio/public/validation/v-8f3/details?mode=charts) |
132
+ | Small | NXP Ara240 (FRDM) β€” latency | 56.72% | -2.68 | 39.83% | 10.12 | 22.27 | 87.4 | ~99 | [v-a20](https://edgefirst.studio/public/validation/v-a20/details?mode=charts) |
133
+ | Small | NXP Ara240 (FRDM) β€” throughput | 56.70% | -2.70 | 39.81% | 10.27 | 24.17 | 142.4 | ~142 | [v-a21](https://edgefirst.studio/public/validation/v-a21/details?mode=charts) |
134
+ | Small | Raspberry Pi 5 + Hailo-8L NPU | 58.52% | -0.88 | 42.34% | 30.36 | 45.07 | 28.8 | ~33 | [v-8ec](https://edgefirst.studio/public/validation/v-8ec/details?mode=charts) |
135
  | Small | NVIDIA Jetson Orin Nano (TensorRT FP16) | 59.38% | -0.02 | 43.15% | 24.62 | 37.12 | 157.5 | ~162 | [v-922](https://edgefirst.studio/public/validation/v-922/details?mode=charts) |
136
  | Medium | ONNX FP32 (AWS Graviton Β· 48-core) | 64.70% | +0.02 | 48.46% | 181.16 | 189.63 | 21.7 | ~22 | [v-71a](https://edgefirst.studio/public/validation/v-71a/details?mode=charts) |
137
  | Medium | ONNX FP32 (CUDA) | 64.68% | ref | 48.45% | 43.58 | 49.61 | 78.4 | ~92 | [v-701](https://edgefirst.studio/public/validation/v-701/details?mode=charts) |
 
144
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” latency | 63.51% | -1.17 | 47.11% | 101.26 | 135.73 | 9.5 | ~10 | [v-974](https://edgefirst.studio/public/validation/v-974/details?mode=charts) |
145
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β€” throughput | 63.51% | -1.17 | 47.11% | 393.79 | 429.57 | 10.0 | ~10 | [v-976](https://edgefirst.studio/public/validation/v-976/details?mode=charts) |
146
  | Medium | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β€” throughput | 63.51% | -1.17 | 47.11% | 387.27 | 419.61 | 10.2 | ~10 | [v-94f](https://edgefirst.studio/public/validation/v-94f/details?mode=charts) |
147
+ | Medium | NXP Ara240 (FRDM) β€” latency | 60.01% | -4.67 | 42.67% | 21.57 | 33.60 | 43.8 | ~46 | [v-a2e](https://edgefirst.studio/public/validation/v-a2e/details?mode=charts) |
148
+ | Medium | NXP Ara240 (FRDM) β€” throughput | 59.99% | -4.69 | 42.66% | 21.56 | 34.45 | 54.1 | ~54 | [v-a2f](https://edgefirst.studio/public/validation/v-a2f/details?mode=charts) |
149
+ | Medium | Raspberry Pi 5 + Hailo-8L NPU | 63.88% | -0.80 | 47.67% | 56.26 | 71.51 | 16.9 | ~18 | [v-90a](https://edgefirst.studio/public/validation/v-90a/details?mode=charts) |
150
  | Medium | NVIDIA Jetson Orin Nano (TensorRT FP16) | 64.67% | -0.01 | 48.41% | 52.44 | 64.24 | 75.3 | ~76 | [v-92b](https://edgefirst.studio/public/validation/v-92b/details?mode=charts) |
151
 
152
 
 
163
 
164
  Inference latency is reported as the on-accelerator inference time. End-to-end latency is the sequential per-image latency across the compute pipeline β€” preprocessing, inference, and postprocessing; image acquisition (file or camera load and JPEG decode) overlaps these stages and is excluded from this figure.
165
 
166
+ Two throughput figures are reported. **Realized FPS** is the *measured* steady-state rate at which final results are emitted, computed from the Perfetto trace as `(N-1) / (t_last - t_first)` over the warmup-trimmed steady-state region; it is the true, priority number and generally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames. **Theoretical FPS** is the accelerator's **core** ceiling β€” `1000 / device-compute-time`, i.e. the throughput if the accelerator were the *only* bottleneck. It is taken from the isolated device-compute stage (on transfer-split runtimes the trace separates host↔device transfers from device compute), which is load-independent β€” unlike the host capture/preprocess service times, whose measured cost inflates under pipeline backpressure (the same 5000 JPEGs cost ~7.8 ms/frame serialized but far more under throughput backpressure), so the slowest-stage figure would *understate* a fast accelerator. It is a possibly-achievable ceiling, not a measured result: reaching it depends on the deployment pipeline. A validation run decodes a JPEG per image and evaluates at a `0.001` confidence threshold (to capture every detection for mAP), both of which load the host and postprocess stages; a production camera pipeline (no JPEG decode) at a deployment threshold of `0.25–0.75` (far fewer candidate boxes through NMS) moves realized throughput toward the theoretical ceiling.
167
 
168
  See [EdgeFirst Studio](https://edgefirst.studio) for the full validation pipeline.
169