Update model card for yolo11-det
Browse files
README.md
CHANGED
|
@@ -102,7 +102,7 @@ Row conventions in the table below:
|
|
| 102 |
- **Decoder variants.** EdgeFirst ships three INT8 split-decoders. The table headlines the accuracy-recovering ones: **smart** (per-tensor rescaling β best accuracy, extra CPU ops add some latency) and, where a smart run is absent, **logical** (the latency-optimized default β no CPU overhead, slightly lower accuracy). The **combined** decoder is the *standard-quantization baseline* (equivalent to typical single-scale INT8, and how the reference numbers are produced); it loses the most accuracy β especially on segmentation, where box/mask dynamic range collapses under one scale β so it is published only as a downloadable reference artifact and in the metrics export, never headlined here. Smart and logical exist precisely to recover that loss.
|
| 103 |
- **Platform-label suffixes.** `(FRDM)` / `(Phytec)` name the NXP i.MX 95 development board a session ran on. `β latency` / `β throughput` mark the two pipeline configurations the NXP i.MX 95 Neutron and NXP Ara240 targets run: the **latency** pipeline runs inference serially for the lowest per-frame latency; the **throughput** pipeline runs multiple inference workers for the highest FPS, which raises per-call inference time in exchange. Rows with neither suffix run a single pipeline.
|
| 104 |
- **End-to-end (ms)** is the sequential per-image latency of the compute pipeline β preprocess β inference β postprocess. Image acquisition (camera or file load + JPEG decode) overlaps these stages and is excluded from this figure.
|
| 105 |
-
- **Realized FPS vs Theoretical FPS.** **Realized FPS** is the *measured* steady-state throughput β the rate at which final results are actually delivered over the full validation pipeline. It normally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames, and it is the **true, priority number**. **Theoretical FPS** (shown with a `~`) is the
|
| 106 |
|
| 107 |
| Size | Platform | mAP@0.5 | Ξ vs FP32 (pp) | mAP@0.5-0.95 | Inference (ms) | End-to-end (ms) | Realized FPS | Theoretical FPS | Source |
|
| 108 |
|------|----------|---------|----------------|--------------|----------------|-----------------|--------------|-----------------|--------|
|
|
@@ -116,8 +116,9 @@ Row conventions in the table below:
|
|
| 116 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β latency | Not supported | β | β | β | β | β | β | [v-8a4](https://edgefirst.studio/public/validation/v-8a4/details?mode=charts) |
|
| 117 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β throughput | Not supported | β | β | β | β | β | β | [v-8a5](https://edgefirst.studio/public/validation/v-8a5/details?mode=charts) |
|
| 118 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β throughput | Not supported | β | β | β | β | β | β | [v-887](https://edgefirst.studio/public/validation/v-887/details?mode=charts) |
|
| 119 |
-
| Nano | NXP Ara240 (FRDM) β latency | 50.
|
| 120 |
-
| Nano |
|
|
|
|
| 121 |
| Nano | NVIDIA Jetson Orin Nano (TensorRT FP16) | 53.05% | -0.05 | 37.80% | 11.46 | 26.68 | 244.9 | ~262 | [v-917](https://edgefirst.studio/public/validation/v-917/details?mode=charts) |
|
| 122 |
| Small | ONNX FP32 (AWS Graviton Β· 48-core) | 61.16% | -0.03 | 44.88% | 63.52 | 72.25 | 60.3 | ~63 | [v-71c](https://edgefirst.studio/public/validation/v-71c/details?mode=charts) |
|
| 123 |
| Small | ONNX FP32 (CUDA) | 61.19% | ref | 44.88% | 15.36 | 20.95 | 205.5 | ~260 | [v-705](https://edgefirst.studio/public/validation/v-705/details?mode=charts) |
|
|
@@ -126,8 +127,9 @@ Row conventions in the table below:
|
|
| 126 |
| Small | Apple M2 Max β CoreML Metal GPU (FP16) | 60.36% | -0.83 | 44.18% | 14.68 | 18.53 | 192.9 | ~204 | [v-9d1](https://edgefirst.studio/public/validation/v-9d1/details?mode=charts) |
|
| 127 |
| Small | Apple M2 Max β CoreML CPU (FP16) | 60.32% | -0.87 | 44.15% | 30.78 | 35.00 | 62.7 | ~65 | [v-9f5](https://edgefirst.studio/public/validation/v-9f5/details?mode=charts) |
|
| 128 |
| Small | NXP i.MX 8M Plus + VeriSilicon NPU (FRDM) | 3.48% | -57.71 β | 2.34% | 168.47 | 220.09 | 5.6 | ~6 | [v-9bc](https://edgefirst.studio/public/validation/v-9bc/details?mode=charts) |
|
| 129 |
-
| Small | NXP Ara240 (FRDM) β latency | 57.
|
| 130 |
-
| Small |
|
|
|
|
| 131 |
| Small | NVIDIA Jetson Orin Nano (TensorRT FP16) | 61.13% | -0.06 | 44.85% | 24.64 | 36.52 | 156.9 | ~162 | [v-91f](https://edgefirst.studio/public/validation/v-91f/details?mode=charts) |
|
| 132 |
| Medium | ONNX FP32 (AWS Graviton Β· 48-core) | 65.94% | +0.03 | 49.60% | 169.65 | 178.45 | 23.0 | ~24 | [v-71e](https://edgefirst.studio/public/validation/v-71e/details?mode=charts) |
|
| 133 |
| Medium | ONNX FP32 (CUDA) | 65.91% | ref | 49.59% | 39.57 | 45.08 | 88.4 | ~101 | [v-707](https://edgefirst.studio/public/validation/v-707/details?mode=charts) |
|
|
@@ -136,8 +138,9 @@ Row conventions in the table below:
|
|
| 136 |
| Medium | Apple M2 Max β CoreML Metal GPU (FP16) | 64.75% | -1.16 | 48.59% | 49.69 | 54.31 | 58.8 | ~60 | [v-9da](https://edgefirst.studio/public/validation/v-9da/details?mode=charts) |
|
| 137 |
| Medium | Apple M2 Max β CoreML CPU (FP16) | 64.71% | -1.20 | 48.55% | 65.01 | 69.09 | 30.2 | ~31 | [v-9db](https://edgefirst.studio/public/validation/v-9db/details?mode=charts) |
|
| 138 |
| Medium | NXP i.MX 8M Plus + VeriSilicon NPU (FRDM) | 54.45% | -11.46 β | 40.02% | 328.40 | 380.36 | 2.9 | ~3 | [v-98d](https://edgefirst.studio/public/validation/v-98d/details?mode=charts) |
|
| 139 |
-
| Medium | NXP Ara240 (FRDM) β latency | 62.
|
| 140 |
-
| Medium |
|
|
|
|
| 141 |
| Medium | NVIDIA Jetson Orin Nano (TensorRT FP16) | 65.89% | -0.02 | 49.56% | 50.85 | 63.05 | 77.5 | ~79 | [v-927](https://edgefirst.studio/public/validation/v-927/details?mode=charts) |
|
| 142 |
|
| 143 |
> **β Below expectations β under investigation.** The rows marked β above measure more than 10 percentage points below the same training session's float reference: the model accuracy on that platform is below our expectations. We publish the measured numbers rather than hiding them, and we are investigating the results to make improvements β the next snapshot of this card will reflect any recovered accuracy.
|
|
@@ -155,7 +158,7 @@ These results are produced by the EdgeFirst on-target validation pipeline:
|
|
| 155 |
|
| 156 |
Inference latency is reported as the on-accelerator inference time. End-to-end latency is the sequential per-image latency across the compute pipeline β preprocessing, inference, and postprocessing; image acquisition (file or camera load and JPEG decode) overlaps these stages and is excluded from this figure.
|
| 157 |
|
| 158 |
-
Two throughput figures are reported. **Realized FPS** is the *measured* steady-state rate at which final results are emitted, computed from the Perfetto trace as `(N-1) / (t_last - t_first)` over the warmup-trimmed steady-state region; it is the true, priority number and generally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames. **Theoretical FPS** is the
|
| 159 |
|
| 160 |
See [EdgeFirst Studio](https://edgefirst.studio) for the full validation pipeline.
|
| 161 |
|
|
|
|
| 102 |
- **Decoder variants.** EdgeFirst ships three INT8 split-decoders. The table headlines the accuracy-recovering ones: **smart** (per-tensor rescaling β best accuracy, extra CPU ops add some latency) and, where a smart run is absent, **logical** (the latency-optimized default β no CPU overhead, slightly lower accuracy). The **combined** decoder is the *standard-quantization baseline* (equivalent to typical single-scale INT8, and how the reference numbers are produced); it loses the most accuracy β especially on segmentation, where box/mask dynamic range collapses under one scale β so it is published only as a downloadable reference artifact and in the metrics export, never headlined here. Smart and logical exist precisely to recover that loss.
|
| 103 |
- **Platform-label suffixes.** `(FRDM)` / `(Phytec)` name the NXP i.MX 95 development board a session ran on. `β latency` / `β throughput` mark the two pipeline configurations the NXP i.MX 95 Neutron and NXP Ara240 targets run: the **latency** pipeline runs inference serially for the lowest per-frame latency; the **throughput** pipeline runs multiple inference workers for the highest FPS, which raises per-call inference time in exchange. Rows with neither suffix run a single pipeline.
|
| 104 |
- **End-to-end (ms)** is the sequential per-image latency of the compute pipeline β preprocess β inference β postprocess. Image acquisition (camera or file load + JPEG decode) overlaps these stages and is excluded from this figure.
|
| 105 |
+
- **Realized FPS vs Theoretical FPS.** **Realized FPS** is the *measured* steady-state throughput β the rate at which final results are actually delivered over the full validation pipeline. It normally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames, and it is the **true, priority number**. **Theoretical FPS** (shown with a `~`) is the accelerator's **core** ceiling β `1000 / device-compute-time`, the rate the NPU/DNPU could sustain *if it were the only bottleneck* β so it is a **possibly-achievable note, not a claim**. It is read from the isolated device-compute stage, which (unlike the host capture/preprocess stages, whose measured time inflates when the pipeline is backpressured) is stable and load-independent. Whether a deployment approaches it depends on the surrounding pipeline, and two levers dominate: (1) **host bottlenecks** β these validation runs decode a JPEG per image, whereas a live **camera pipeline** skips that decode and can run closer to the ceiling; and (2) **confidence threshold** β validation runs at `0.001` to capture every detection for mAP, which makes NMS/decode heavy, while a deployment threshold of `0.25β0.75` produces far fewer candidate boxes and lighter postprocessing, raising realized FPS toward the ceiling.
|
| 106 |
|
| 107 |
| Size | Platform | mAP@0.5 | Ξ vs FP32 (pp) | mAP@0.5-0.95 | Inference (ms) | End-to-end (ms) | Realized FPS | Theoretical FPS | Source |
|
| 108 |
|------|----------|---------|----------------|--------------|----------------|-----------------|--------------|-----------------|--------|
|
|
|
|
| 116 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β latency | Not supported | β | β | β | β | β | β | [v-8a4](https://edgefirst.studio/public/validation/v-8a4/details?mode=charts) |
|
| 117 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (FRDM) β throughput | Not supported | β | β | β | β | β | β | [v-8a5](https://edgefirst.studio/public/validation/v-8a5/details?mode=charts) |
|
| 118 |
| Nano | NXP i.MX 95 + eIQ Neutron NPU (Phytec) β throughput | Not supported | β | β | β | β | β | β | [v-887](https://edgefirst.studio/public/validation/v-887/details?mode=charts) |
|
| 119 |
+
| Nano | NXP Ara240 (FRDM) β latency | 50.05% | -3.05 | 34.47% | 6.36 | 18.58 | 127.9 | ~157 | [v-a14](https://edgefirst.studio/public/validation/v-a14/details?mode=charts) |
|
| 120 |
+
| Nano | NXP Ara240 (FRDM) β throughput | 50.03% | -3.07 | 34.45% | 6.68 | 27.53 | 196.0 | ~308 | [v-a15](https://edgefirst.studio/public/validation/v-a15/details?mode=charts) |
|
| 121 |
+
| Nano | Raspberry Pi 5 + Hailo-8L NPU | 51.77% | -1.33 | 36.73% | 17.86 | 36.29 | 52.6 | ~56 | [v-8dc](https://edgefirst.studio/public/validation/v-8dc/details?mode=charts) |
|
| 122 |
| Nano | NVIDIA Jetson Orin Nano (TensorRT FP16) | 53.05% | -0.05 | 37.80% | 11.46 | 26.68 | 244.9 | ~262 | [v-917](https://edgefirst.studio/public/validation/v-917/details?mode=charts) |
|
| 123 |
| Small | ONNX FP32 (AWS Graviton Β· 48-core) | 61.16% | -0.03 | 44.88% | 63.52 | 72.25 | 60.3 | ~63 | [v-71c](https://edgefirst.studio/public/validation/v-71c/details?mode=charts) |
|
| 124 |
| Small | ONNX FP32 (CUDA) | 61.19% | ref | 44.88% | 15.36 | 20.95 | 205.5 | ~260 | [v-705](https://edgefirst.studio/public/validation/v-705/details?mode=charts) |
|
|
|
|
| 127 |
| Small | Apple M2 Max β CoreML Metal GPU (FP16) | 60.36% | -0.83 | 44.18% | 14.68 | 18.53 | 192.9 | ~204 | [v-9d1](https://edgefirst.studio/public/validation/v-9d1/details?mode=charts) |
|
| 128 |
| Small | Apple M2 Max β CoreML CPU (FP16) | 60.32% | -0.87 | 44.15% | 30.78 | 35.00 | 62.7 | ~65 | [v-9f5](https://edgefirst.studio/public/validation/v-9f5/details?mode=charts) |
|
| 129 |
| Small | NXP i.MX 8M Plus + VeriSilicon NPU (FRDM) | 3.48% | -57.71 β | 2.34% | 168.47 | 220.09 | 5.6 | ~6 | [v-9bc](https://edgefirst.studio/public/validation/v-9bc/details?mode=charts) |
|
| 130 |
+
| Small | NXP Ara240 (FRDM) β latency | 57.16% | -4.03 | 40.13% | 10.17 | 22.06 | 87.2 | ~98 | [v-a22](https://edgefirst.studio/public/validation/v-a22/details?mode=charts) |
|
| 131 |
+
| Small | NXP Ara240 (FRDM) β throughput | 57.15% | -4.04 | 40.11% | 10.31 | 24.04 | 141.3 | ~141 | [v-a23](https://edgefirst.studio/public/validation/v-a23/details?mode=charts) |
|
| 132 |
+
| Small | Raspberry Pi 5 + Hailo-8L NPU | 59.45% | -1.74 | 43.25% | 36.19 | 51.00 | 25.4 | ~28 | [v-8e7](https://edgefirst.studio/public/validation/v-8e7/details?mode=charts) |
|
| 133 |
| Small | NVIDIA Jetson Orin Nano (TensorRT FP16) | 61.13% | -0.06 | 44.85% | 24.64 | 36.52 | 156.9 | ~162 | [v-91f](https://edgefirst.studio/public/validation/v-91f/details?mode=charts) |
|
| 134 |
| Medium | ONNX FP32 (AWS Graviton Β· 48-core) | 65.94% | +0.03 | 49.60% | 169.65 | 178.45 | 23.0 | ~24 | [v-71e](https://edgefirst.studio/public/validation/v-71e/details?mode=charts) |
|
| 135 |
| Medium | ONNX FP32 (CUDA) | 65.91% | ref | 49.59% | 39.57 | 45.08 | 88.4 | ~101 | [v-707](https://edgefirst.studio/public/validation/v-707/details?mode=charts) |
|
|
|
|
| 138 |
| Medium | Apple M2 Max β CoreML Metal GPU (FP16) | 64.75% | -1.16 | 48.59% | 49.69 | 54.31 | 58.8 | ~60 | [v-9da](https://edgefirst.studio/public/validation/v-9da/details?mode=charts) |
|
| 139 |
| Medium | Apple M2 Max β CoreML CPU (FP16) | 64.71% | -1.20 | 48.55% | 65.01 | 69.09 | 30.2 | ~31 | [v-9db](https://edgefirst.studio/public/validation/v-9db/details?mode=charts) |
|
| 140 |
| Medium | NXP i.MX 8M Plus + VeriSilicon NPU (FRDM) | 54.45% | -11.46 β | 40.02% | 328.40 | 380.36 | 2.9 | ~3 | [v-98d](https://edgefirst.studio/public/validation/v-98d/details?mode=charts) |
|
| 141 |
+
| Medium | NXP Ara240 (FRDM) β latency | 62.58% | -3.33 | 45.13% | 24.25 | 36.17 | 39.2 | ~41 | [v-a30](https://edgefirst.studio/public/validation/v-a30/details?mode=charts) |
|
| 142 |
+
| Medium | NXP Ara240 (FRDM) β throughput | 62.56% | -3.35 | 45.12% | 24.29 | 37.03 | 47.1 | ~47 | [v-a31](https://edgefirst.studio/public/validation/v-a31/details?mode=charts) |
|
| 143 |
+
| Medium | Raspberry Pi 5 + Hailo-8L NPU | 63.78% | -2.13 | 47.69% | 70.71 | 86.10 | 13.6 | ~14 | [v-90f](https://edgefirst.studio/public/validation/v-90f/details?mode=charts) |
|
| 144 |
| Medium | NVIDIA Jetson Orin Nano (TensorRT FP16) | 65.89% | -0.02 | 49.56% | 50.85 | 63.05 | 77.5 | ~79 | [v-927](https://edgefirst.studio/public/validation/v-927/details?mode=charts) |
|
| 145 |
|
| 146 |
> **β Below expectations β under investigation.** The rows marked β above measure more than 10 percentage points below the same training session's float reference: the model accuracy on that platform is below our expectations. We publish the measured numbers rather than hiding them, and we are investigating the results to make improvements β the next snapshot of this card will reflect any recovered accuracy.
|
|
|
|
| 158 |
|
| 159 |
Inference latency is reported as the on-accelerator inference time. End-to-end latency is the sequential per-image latency across the compute pipeline β preprocessing, inference, and postprocessing; image acquisition (file or camera load and JPEG decode) overlaps these stages and is excluded from this figure.
|
| 160 |
|
| 161 |
+
Two throughput figures are reported. **Realized FPS** is the *measured* steady-state rate at which final results are emitted, computed from the Perfetto trace as `(N-1) / (t_last - t_first)` over the warmup-trimmed steady-state region; it is the true, priority number and generally exceeds `1000 / end-to-end` because the runtime overlaps stages across frames. **Theoretical FPS** is the accelerator's **core** ceiling β `1000 / device-compute-time`, i.e. the throughput if the accelerator were the *only* bottleneck. It is taken from the isolated device-compute stage (on transfer-split runtimes the trace separates hostβdevice transfers from device compute), which is load-independent β unlike the host capture/preprocess service times, whose measured cost inflates under pipeline backpressure (the same 5000 JPEGs cost ~7.8 ms/frame serialized but far more under throughput backpressure), so the slowest-stage figure would *understate* a fast accelerator. It is a possibly-achievable ceiling, not a measured result: reaching it depends on the deployment pipeline. A validation run decodes a JPEG per image and evaluates at a `0.001` confidence threshold (to capture every detection for mAP), both of which load the host and postprocess stages; a production camera pipeline (no JPEG decode) at a deployment threshold of `0.25β0.75` (far fewer candidate boxes through NMS) moves realized throughput toward the theoretical ceiling.
|
| 162 |
|
| 163 |
See [EdgeFirst Studio](https://edgefirst.studio) for the full validation pipeline.
|
| 164 |
|