Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -37,6 +37,26 @@ portable option and are what runs on Android.
|
|
| 37 |
\*Mac arm64, single process, median of 10 — a reference point for relative cost
|
| 38 |
only, not a device number (torch eager fp32 on the same machine: 34.4 ms).
|
| 39 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 40 |
## Verification (executorch 1.4.0, torch 2.13.0)
|
| 41 |
|
| 42 |
Parity is measured against the fp32 eager model on real image input; `corr` is
|
|
|
|
| 37 |
\*Mac arm64, single process, median of 10 — a reference point for relative cost
|
| 38 |
only, not a device number (torch eager fp32 on the same machine: 34.4 ms).
|
| 39 |
|
| 40 |
+
### Checked in the task's own units
|
| 41 |
+
|
| 42 |
+
Correlation is a first filter, and on an embedding it is a weak one: a build can read
|
| 43 |
+
0.999 against eager and still move an answer, because what decides is whether the error is
|
| 44 |
+
smaller than the gap between the answer a query gets and the runner-up.
|
| 45 |
+
|
| 46 |
+
The test is **near-duplicate retrieval** over 52 photographs, each also present centre-cropped to 90% and re-encoded as JPEG. Every item has one obviously
|
| 47 |
+
correct nearest neighbour, and each build has to find it.
|
| 48 |
+
|
| 49 |
+
| build | correct neighbour kept | worst score shift | budget spent |
|
| 50 |
+
|---|---|---|---|
|
| 51 |
+
| fp32 | 104 of 104 | 0.0015 | 5% |
|
| 52 |
+
| fp16 | 104 of 104 | 0.0015 | 5% |
|
| 53 |
+
| Core ML (fp16, iOS) | 104 of 104 | 0.0083 | 28% |
|
| 54 |
+
|
| 55 |
+
The closest decision eager makes on this set is **0.0298**, and the bar is half of
|
| 56 |
+
it. An earlier version of this card cleared int8 on "cosine similarity of the embeddings",
|
| 57 |
+
which is correlation wearing a task metric's clothes — it never asked whether a retrieval
|
| 58 |
+
answer moved.
|
| 59 |
+
|
| 60 |
## Verification (executorch 1.4.0, torch 2.13.0)
|
| 61 |
|
| 62 |
Parity is measured against the fp32 eager model on real image input; `corr` is
|