Add XNNPACK int8 export

README.md

@@ -17,7 +17,17 @@ Upstream model: [RFDetr](https://github.com/roboflow/rf-detr)
 | Path | Backend | Precision |
 | --- | --- | --- |
 | `coreml/rfdetr_keypoint_preview_coreml_fp16.pte` | coreml | fp16 |
+| `vulkan/rfdetr_keypoint_preview_vulkan_fp16.pte` | vulkan | fp16 |
 | `xnnpack/rfdetr_keypoint_preview_xnnpack_fp32.pte` | xnnpack | fp32 |
+| `xnnpack/rfdetr_keypoint_preview_xnnpack_int8.pte` | xnnpack | int8 |
+
+## XNNPACK int8
+
+`xnnpack/rfdetr_keypoint_preview_xnnpack_int8.pte` is 8da8w (int8 dynamic
+activations, int8 per-channel weights) on the Linear layers, except the
+deformable-attention offsets and the class, box and keypoint heads; convolutions stay
+fp32. COCO keypoint (OKS) mAP50-95 on the first 500 val2017 images: fp32
+70.37, int8 70.00.
 
 ## Unshipped backends
 
@@ -33,8 +43,11 @@ config.json                                       37 B
 coreml/config.json                                1.1 kB
 coreml/rfdetr_keypoint_preview_coreml_fp16.pte    71.5 MB
 mlx/NOTES.md                                      1.4 kB
-xnnpack/config.json                               1.1 kB
+vulkan/config.json                                1.1 kB
+vulkan/rfdetr_keypoint_preview_vulkan_fp16.pte    70.0 MB
+xnnpack/config.json                               1.9 kB
 xnnpack/rfdetr_keypoint_preview_xnnpack_fp32.pte  139 MB
+xnnpack/rfdetr_keypoint_preview_xnnpack_int8.pte  56.6 MB
 ```
 
 ## Compatibility

xnnpack/config.json

@@ -49,6 +49,48 @@
           ]
         }
       }
+    },
+    {
+      "file": "rfdetr_keypoint_preview_xnnpack_int8.pte",
+      "precision": "int8",
+      "methods": {
+        "forward": {
+          "inputs": [
+            {
+              "shape": [
+                1,
+                3,
+                576,
+                576
+              ],
+              "dtype": "float32"
+            }
+          ],
+          "outputs": [
+            {
+              "shape": [
+                100,
+                4
+              ],
+              "dtype": "float32"
+            },
+            {
+              "shape": [
+                100
+              ],
+              "dtype": "float32"
+            },
+            {
+              "shape": [
+                100,
+                17,
+                3
+              ],
+              "dtype": "float32"
+            }
+          ]
+        }
+      }
     }
   ]
 }

xnnpack/rfdetr_keypoint_preview_xnnpack_int8.pte

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d66d6d57709904eb52ed088689fc932ba55e268a9f2e0fe24829c9a989a36b71
+size 59381136