Add XNNPACK int8 (8da8w) variant

.gitattributes

@@ -39,3 +39,4 @@ xnnpack/all_mpnet_base_v2_xnnpack_fp32.pte filter=lfs diff=lfs merge=lfs -text
 vulkan/all_mpnet_base_v2_vulkan_fp16.pte filter=lfs diff=lfs merge=lfs -text
 vulkan/all_mpnet_base_v2_vulkan_int8.pte filter=lfs diff=lfs merge=lfs -text
 xnnpack/all_mpnet_base_v2_xnnpack_8da4w.pte filter=lfs diff=lfs merge=lfs -text
+xnnpack/all_mpnet_base_v2_xnnpack_int8.pte filter=lfs diff=lfs merge=lfs -text

README.md

@@ -20,16 +20,19 @@ Upstream model: [all-mpnet-base-v2](https://huggingface.co/sentence-transformers
 | `vulkan/all_mpnet_base_v2_vulkan_int8.pte` | vulkan | int8 |
 | `xnnpack/all_mpnet_base_v2_xnnpack_8da4w.pte` | xnnpack | 8da4w |
 | `xnnpack/all_mpnet_base_v2_xnnpack_fp32.pte` | xnnpack | fp32 |
+| `xnnpack/all_mpnet_base_v2_xnnpack_int8.pte` | xnnpack | int8 |
 
 ## XNNPACK int8
 
-`xnnpack/all_mpnet_base_v2_xnnpack_8da4w.pte` is 8da4w (int8 dynamic activations, int4 weights,
-group 32) on the Linear layers, with an int8 word-embedding table. SQuAD v1.1 dev
-passage retrieval (2000 questions against their 315 paragraphs), dot product:
+`xnnpack/all_mpnet_base_v2_xnnpack_int8.pte` is 8da8w (int8 dynamic activations, int8 per-channel
+weights) and `xnnpack/all_mpnet_base_v2_xnnpack_8da4w.pte` is 8da4w (int4 weights, group 32), both on the
+Linear layers with an int8 word-embedding table. SQuAD v1.1 dev passage retrieval (2000
+questions against their 315 paragraphs), dot product:
 
 | | top-1 | MRR |
 | --- | --- | --- |
 | fp32 | 65.05 | 0.7631 |
+| int8 | 65.00 | 0.7633 |
 | 8da4w | 64.60 | 0.7606 |
 
 ## Repository structure
@@ -43,7 +46,8 @@ vulkan/all_mpnet_base_v2_vulkan_int8.pte     127 MB
 vulkan/config.json                           1.5 kB
 xnnpack/all_mpnet_base_v2_xnnpack_8da4w.pte  69.1 MB
 xnnpack/all_mpnet_base_v2_xnnpack_fp32.pte   416 MB
-xnnpack/config.json                          1.5 kB
+xnnpack/all_mpnet_base_v2_xnnpack_int8.pte   105 MB
+xnnpack/config.json                          2.2 kB
 ```
 
 ## Compatibility

xnnpack/all_mpnet_base_v2_xnnpack_int8.pte

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:61befa934901b544c665ec90d178ce4b1bb2ac1a772bb627df97face7323c2b9
+size 109965568

xnnpack/config.json

@@ -41,6 +41,39 @@
         }
       }
     },
+    {
+      "file": "all_mpnet_base_v2_xnnpack_int8.pte",
+      "precision": "int8",
+      "methods": {
+        "forward": {
+          "inputs": [
+            {
+              "shape": [
+                1,
+                382
+              ],
+              "dtype": "int64"
+            },
+            {
+              "shape": [
+                1,
+                382
+              ],
+              "dtype": "int64"
+            }
+          ],
+          "outputs": [
+            {
+              "shape": [
+                1,
+                768
+              ],
+              "dtype": "float32"
+            }
+          ]
+        }
+      }
+    },
     {
       "file": "all_mpnet_base_v2_xnnpack_8da4w.pte",
       "precision": "8da4w",