Add XNNPACK int8 (8da8w) variant

.gitattributes

@@ -39,3 +39,4 @@ xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_fp32.pte filter=lfs diff=lfs merge=lf
 vulkan/multi_qa_mpnet_base_dot_v1_vulkan_fp16.pte filter=lfs diff=lfs merge=lfs -text
 vulkan/multi_qa_mpnet_base_dot_v1_vulkan_int8.pte filter=lfs diff=lfs merge=lfs -text
 xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte filter=lfs diff=lfs merge=lfs -text
+xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte filter=lfs diff=lfs merge=lfs -text

README.md

@@ -20,16 +20,19 @@ Upstream model: [multi-qa-mpnet-base-dot-v1](https://huggingface.co/sentence-tra
 | `vulkan/multi_qa_mpnet_base_dot_v1_vulkan_int8.pte` | vulkan | int8 |
 | `xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte` | xnnpack | 8da4w |
 | `xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_fp32.pte` | xnnpack | fp32 |
+| `xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte` | xnnpack | int8 |
 
 ## XNNPACK int8
 
-`xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte` is 8da4w (int8 dynamic activations, int4 weights,
-group 32) on the Linear layers, with an int8 word-embedding table. SQuAD v1.1 dev
-passage retrieval (2000 questions against their 315 paragraphs), dot product:
+`xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte` is 8da8w (int8 dynamic activations, int8 per-channel
+weights) and `xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte` is 8da4w (int4 weights, group 32), both on the
+Linear layers with an int8 word-embedding table. SQuAD v1.1 dev passage retrieval (2000
+questions against their 315 paragraphs), dot product:
 
 | | top-1 | MRR |
 | --- | --- | --- |
 | fp32 | 64.60 | 0.7645 |
+| int8 | 65.00 | 0.7671 |
 | 8da4w | 63.80 | 0.7578 |
 
 ## Repository structure
@@ -41,9 +44,10 @@ tokenizer_config.json                                 363 B
 vulkan/config.json                                    1.6 kB
 vulkan/multi_qa_mpnet_base_dot_v1_vulkan_fp16.pte     208 MB
 vulkan/multi_qa_mpnet_base_dot_v1_vulkan_int8.pte     127 MB
-xnnpack/config.json                                   1.6 kB
+xnnpack/config.json                                   2.2 kB
 xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte  69.1 MB
 xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_fp32.pte   416 MB
+xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte   105 MB
 ```
 
 ## Compatibility

xnnpack/config.json

@@ -41,6 +41,39 @@
         }
       }
     },
+    {
+      "file": "multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte",
+      "precision": "int8",
+      "methods": {
+        "forward": {
+          "inputs": [
+            {
+              "shape": [
+                1,
+                510
+              ],
+              "dtype": "int64"
+            },
+            {
+              "shape": [
+                1,
+                510
+              ],
+              "dtype": "int64"
+            }
+          ],
+          "outputs": [
+            {
+              "shape": [
+                1,
+                768
+              ],
+              "dtype": "float32"
+            }
+          ]
+        }
+      }
+    },
     {
       "file": "multi_qa_mpnet_base_dot_v1_xnnpack_8da4w.pte",
       "precision": "8da4w",

xnnpack/multi_qa_mpnet_base_dot_v1_xnnpack_int8.pte

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:77fc6dfe6490d401dc0e5d3556251560f3c2c2d3095ba8ac795ead81c69d3a5c
+size 109965568