mlboydaisuke commited on
Commit
33c6087
·
verified ·
1 Parent(s): 0aa5d10

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +34 -14
README.md CHANGED
@@ -21,7 +21,6 @@ All variants take and return fp32 tensors — swap the `.pte` file, keep your ap
21
  | build | file | size (MB) | parity vs fp32 eager (worst corr) | Mac median (ms)* |
22
  |-----------|------|-----------|------------------------------------|------------------|
23
  | fp32 | `ssdlite320_mobilenetv3_xnnpack_fp32.pte` | 13.8 | 1.000000 | 5.1 |
24
- | int8 | `ssdlite320_mobilenetv3_xnnpack_int8.pte` | 3.9 | 0.968820 | 7.1 |
25
  | Core ML (fp16, iOS) | `ssdlite320_mobilenetv3_coreml_all.pte` | 7.5 | 0.999658 | 0.9 |
26
 
27
 
@@ -37,13 +36,44 @@ only, not a device number (torch eager fp32 on the same machine: 115.3 ms).
37
 
38
  ### Checked in the task's own units
39
 
40
- Correlation is a first filter. These are the numbers that decide:
 
 
41
 
42
- - **int8** — measured in the units that matter for this model — fraction of firing detections agreeing: 0.999 of the fp32 build's detections are matched (26988 of 27004 across 10 images), worst single image 0.998.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
43
 
44
  ### Builds that did not earn a slot
45
 
46
- - **fp16 is not shipped**: it comes out at 100% of the fp32 file (13.8 MB vs 13.8 MB), so it buys nothing. XNNPACK serializes convolution weights as fp32 no matter what dtype the graph carries, so on a conv-heavy model fp16 saves no disk and only adds cast operations. Reach for int8 here, not fp16.
47
 
48
  ## Verification (executorch 1.4.0, torch 2.13.0)
49
 
@@ -71,13 +101,3 @@ XNNPACK delegate coverage (fp32): 94.8% (289/305 ops); ops left on the portable
71
 
72
  torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
73
  (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))
74
-
75
- <!-- funnel:v1 -->
76
-
77
- ---
78
-
79
- **More models in this format:** [ExecuTorch Model Zoo](https://huggingface.co/collections/mlboydaisuke/executorch-model-zoo-6a7ff328390b63075ffeae5e) — 31 models, each with the recipe that produced it.
80
-
81
- **Want a different model on-device?** [Open a request](https://github.com/john-rocky/on-device-requests) — free, open weights only; the export and its measured numbers get published publicly.
82
-
83
- <!-- /funnel:v1 -->
 
21
  | build | file | size (MB) | parity vs fp32 eager (worst corr) | Mac median (ms)* |
22
  |-----------|------|-----------|------------------------------------|------------------|
23
  | fp32 | `ssdlite320_mobilenetv3_xnnpack_fp32.pte` | 13.8 | 1.000000 | 5.1 |
 
24
  | Core ML (fp16, iOS) | `ssdlite320_mobilenetv3_coreml_all.pte` | 7.5 | 0.999658 | 0.9 |
25
 
26
 
 
36
 
37
  ### Checked in the task's own units
38
 
39
+ The graph stops at the raw heads, so the numbers that decide have to be taken *after* the
40
+ anchor decode and NMS this card tells callers to run — using torchvision's own
41
+ `anchor_generator` and `postprocess_detections`, not a hand-rolled substitute.
42
 
43
+ An earlier version of this section counted "firing detections", 26,988 of 27,004 across
44
+ 10 images. Those were raw anchors above a floor, not objects: 27,004 of them for ten
45
+ photographs that contain perhaps twenty things. Counting anchors measures the background
46
+ agreeing with itself.
47
+
48
+ Measured properly — **71 detections over 32 photographs** of people and animals, held out
49
+ from the `street` images the int8 build was calibrated on, at a score threshold of 0.30:
50
+
51
+ | build | added | dropped | relabelled | worst matched IoU | worst score shift |
52
+ |---|---|---|---|---|---|
53
+ | fp32 | 0 | 0 | 0 | 1.000 | 0.0000 |
54
+ | Core ML | 1 | 0 | 0 | 0.984 | 0.0732 |
55
+
56
+ Core ML's single disagreement sits within its own score error of the threshold — a
57
+ detection the cut is ambivalent about, which a photograph shifted by a pixel moves in
58
+ eager too. Every object it keeps is in the same place: worst matched IoU 0.984.
59
+
60
+ ## Withdrawn: int8 (2026-08-27)
61
+
62
+ `ssdlite320_mobilenetv3_xnnpack_int8.pte` was published and has been **withdrawn**.
63
+ Measured the same way it **adds 19 detections, drops 5 and relabels 1** — 25 changes to a
64
+ 71-detection answer, better than a third of it — and the boxes it does keep fall to
65
+ **IoU 0.673**. Its worst score shift is **0.4132**, larger than the 0.30 threshold itself.
66
+
67
+ Its correlation was already the warning: **0.968820**, the lowest on this shelf's detection
68
+ models, and it shipped anyway because the number that was supposed to catch it counted
69
+ anchors instead of objects.
70
+
71
+ At 3.9 MB against fp32's 13.8 it was much the smallest build. The Core ML build is 7.5 MB
72
+ and, on this host, the fastest of the three.
73
 
74
  ### Builds that did not earn a slot
75
 
76
+ - **fp16 is not shipped**: it comes out at 100% of the fp32 file (13.8 MB vs 13.8 MB), so it buys nothing. XNNPACK serializes convolution weights as fp32 no matter what dtype the graph carries, so on a conv-heavy model fp16 saves no disk and only adds cast operations. Quantization, not fp16, is the reduction that can shrink a graph like this.
77
 
78
  ## Verification (executorch 1.4.0, torch 2.13.0)
79
 
 
101
 
102
  torch.export -> to_edge_transform_and_lower(partitioner) -> .pte
103
  (conversion scripts: [executorch-models](https://github.com/john-rocky/executorch-models))