experimentalmachines/Phi-4-mini-instruct-3.8B-ExecuTorch

Phi-4-mini-instruct-3.8B for ExecuTorch

ExecuTorch exports of microsoft/Phi-4-mini-instruct (revision cfbefacb9925) for on-device inference with the openweights Android app or any ExecuTorch 1.5.1 runtime.

Files

Windows in this repo: XNNPACK (CPU) at 2k to 32k; Vulkan (GPU) at 2k to 32k. The window is fixed inside the file: the runtime allocates the whole KV cache at load, so pick the largest window the device can hold (fits_phone_budget in each folder's config.json is the estimate against a 5 GB budget).

Backend	Target	File	Window	Size	Smoke test
XNNPACK (CPU)	any arm64	xnnpack/Phi-4-mini-instruct-3.8B-8da4w-gptq-2k.pte	2,048 tokens	3.05 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Phi-4-mini-instruct-3.8B-8da4w-gptq-4k.pte	4,096 tokens	3.05 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Phi-4-mini-instruct-3.8B-8da4w-gptq-8k.pte	8,192 tokens	3.05 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Phi-4-mini-instruct-3.8B-8da4w-gptq-16k.pte	16,384 tokens	3.06 GB	passed ("Paris")
XNNPACK (CPU)	any arm64	xnnpack/Phi-4-mini-instruct-3.8B-8da4w-gptq-32k.pte	32,768 tokens	3.07 GB	passed ("Paris")
Vulkan (GPU)	any arm64	vulkan/Phi-4-mini-instruct-3.8B-vulkan-8da4w-gptq-2k.pte	2,048 tokens	3.66 GB	structure checked (no host NPU runtime)
Vulkan (GPU)	any arm64	vulkan/Phi-4-mini-instruct-3.8B-vulkan-8da4w-gptq-4k.pte	4,096 tokens	3.67 GB	structure checked (no host NPU runtime)
Vulkan (GPU)	any arm64	vulkan/Phi-4-mini-instruct-3.8B-vulkan-8da4w-gptq-8k.pte	8,192 tokens	3.69 GB	structure checked (no host NPU runtime)
Vulkan (GPU)	any arm64	vulkan/Phi-4-mini-instruct-3.8B-vulkan-8da4w-gptq-16k.pte	16,384 tokens	3.73 GB	structure checked (no host NPU runtime)
Vulkan (GPU)	any arm64	vulkan/Phi-4-mini-instruct-3.8B-vulkan-8da4w-gptq-32k.pte	32,768 tokens	3.81 GB	structure checked (no host NPU runtime)

Tokenizer: tokenizer.json, copied unchanged from the source repo. Each backend folder has a config.json listing every window as a variant with the metadata the .pte reports, and an export-report-<window>.json per file with the full export record.

Memory

• XNNPACK (CPU) at 2,048 tokens: the KV cache costs 262,144 bytes per token (fp32), 536,870,912 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 4,096 tokens: the KV cache costs 262,144 bytes per token (fp32), 1,073,741,824 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 8,192 tokens: the KV cache costs 262,144 bytes per token (fp32), 2,147,483,648 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 16,384 tokens: the KV cache costs 262,144 bytes per token (fp32), 4,294,967,296 bytes for the whole window, allocated in full when the model loads.
• XNNPACK (CPU) at 32,768 tokens: the KV cache costs 262,144 bytes per token (fp32), 8,589,934,592 bytes for the whole window, allocated in full when the model loads.
• Vulkan (GPU) at 2,048 tokens: the KV cache costs 262,144 bytes per token (fp32), 536,870,912 bytes for the whole window, allocated in full when the model loads.
• Vulkan (GPU) at 4,096 tokens: the KV cache costs 262,144 bytes per token (fp32), 1,073,741,824 bytes for the whole window, allocated in full when the model loads.
• Vulkan (GPU) at 8,192 tokens: the KV cache costs 262,144 bytes per token (fp32), 2,147,483,648 bytes for the whole window, allocated in full when the model loads.
• Vulkan (GPU) at 16,384 tokens: the KV cache costs 262,144 bytes per token (fp32), 4,294,967,296 bytes for the whole window, allocated in full when the model loads.
• Vulkan (GPU) at 32,768 tokens: the KV cache costs 262,144 bytes per token (fp32), 8,589,934,592 bytes for the whole window, allocated in full when the model loads.

How it was made

• XNNPACK (CPU) any arm64: ExecuTorch 1.5.1 export_llm: 8-bit dynamic activations and 4-bit weights in groups of 32 (the output head 8-bit per channel), int8 per-channel embeddings, XNNPACK with extended ops, prefill chunk 2048, fp32 KV cache. Published past its decision gate, which asks that 90% of fp32's tool calls be kept: 2k kept 7 of 8, 4k kept 7 of 8, 8k kept 7 of 8, 16k kept 7 of 8, 32k kept 7 of 8. Accepted because the unquantised model is itself near a tie on several tool-call rows (fp32 0.40 to 0.78), the 4-bit recipes measured keep 7 of 8, and the operator chose this file over an all-int8 one 1.4 times its size (2026-11-09).
• Vulkan (GPU) any arm64: ExecuTorch 1.5.1 export_llm: 8-bit dynamic activations and 4-bit weights in groups of 32 (the output head 8-bit per channel), int8 per-channel embeddings, the Vulkan delegate, prefill chunk 2048, fp32 KV cache. Published past its decision gate, which asks that 90% of fp32's tool calls be kept: 2k kept 7 of 8, 4k kept 7 of 8, 8k kept 7 of 8, 16k kept 7 of 8, 32k kept 7 of 8. Accepted because the unquantised model is itself near a tie on several tool-call rows (fp32 0.40 to 0.78), the 4-bit recipes measured keep 7 of 8, and the operator chose this file over an all-int8 one 1.4 times its size (2026-11-09).

License

A quantized derivative of microsoft/Phi-4-mini-instruct, distributed under the same terms (mit). The upstream license files are included unchanged: LICENSE, NOTICE.md.