The published build rejected seq=1 with "Size (1) of dimension (1) is not in
allowed range (2..126)", which surfaced in React Native ExecuTorch as
Error::DelegateInvalidHandle for any single-word input. See
software-mansion/react-native-executorch#1164.
torch.export keeps the declared min of 1; to_edge_transform_and_lower is what
widened the token dim to 2..126. Exporting under
torch.fx.experimental._config.backed_size_oblivious keeps 1 in the range.
schema.json already declared min 1, so the file now matches the contract it
publishes rather than the other way round.
Verified on macOS (ExecuTorch 1.4.1) against the eager model, cosine at every
length from 1 to the maximum:
seq=1 0.99999
seq=2 0.99999
seq=16 0.99999
seq=126 0.99999
and on an iPhone 16 in the demo app, where a single word now embeds instead of
failing.
The file name is unchanged, so nothing moves in the model registry.
quantized duplicated precisions.json, which is authoritative for the
quantized/non-quantized partition. default expressed a per-repository
recommendation nothing consumed: the library resolves its own per-platform
default. Both are gone from the schema.
Adds library_name to the model card, and pipeline_tag where it was missing.
Brings the config and card licenses into agreement, and repairs the variant
declarations that did not match the files actually published here.
Lowered with eager attention and rank-3 attention (the singleton batch folded
into the head dim), mean pooling written with keepdim so the reduction stays on
the GPU, and the token embedding kept off texture storage, without which fp16
returns silently wrong values (pytorch/executorch#22333). That last point
matters most here: at 250037 rows this is by far the largest vocabulary in the
family.
1.40x XNNPACK on an Adreno 840 and 1.50x on a Mali-G76, at half the download.
Verified against the CPU reference at cosine >= 0.99997 on both.
Requires the ExecuTorch 1.4.1 Vulkan runtime from the v0.10.0-libs-1.4.1
artifacts.
xnnpack: 8da4w is the default; fp32 stays as the accuracy reference (fp32 re-exported)
These graphs fail the Apple Neural Engine path (coremltools#2825 masked mean-pool over a dynamic sequence dim; clip-text additionally ANECCompile FAILED). Core ML still returned correct output because it silently fell back to CPU, but the failed ANE compile was paid on every call. Pinning compute_unit=CPU_ONLY skips the attempt.
Measured on an iPhone 16, fp16, 3 warmup + 15 timed runs, median:
all-MiniLM 13.7 ms -> 2.5 ms 5.5x
multi-qa 14.1 ms -> 2.5 ms 5.6x
paraphrase-L12 26.9 ms -> 4.2 ms 6.4x
distiluse 33.0 ms -> 8.0 ms 4.1x
clip-text 18.1 ms -> 4.9 ms 3.7x
The fp32 variant is removed. Once fp16 is pinned it is dominated on both axes: slower (all-MiniLM 4.5 vs 2.5 ms, clip-text 9.5 vs 4.9 ms) and twice the size. Output fidelity against fp32 was gated first: relative L2 4e-6 (all-MiniLM, multi-qa), 0.0010 (paraphrase), 0.0046 (distiluse).
v0.9.0 and below are untouched.
Fix Core ML artifacts: the published builds failed to load
The published Core ML .pte files for this model did not load at all, at either fp16 or fp32, while coreml was declared as a supported backend:
[Core ML] Model init failed `MLModelConfiguration`'s `.functionName` property must be
`nil` unless the model type is ML Program.
CoreMLBackend: Failed to init the model.
Init failed for backend CoreMLBackend: 0x23
Re-exporting with the current exporter fixes it; the graph and recipe are unchanged and the artifact sizes are within 0.4 MB of the previous ones, so these were stale builds carrying the wrong Core ML model type.
Verified against the XNNPACK fp32 build on identical input: the Core ML fp32 embedding matches exactly and fp16 matches to four decimals.
v0.9.0 and below are untouched.
