Add measured on-device speed table; note wi8 runtime requirement
#11
by mlboydaisuke - opened
Thanks for the MiniCPM5 bundles — the new gpu_opt build shows a real GPU win on this machine (+18% decode over the non-opt int4 file). This PR adds a measured speed table for all three files and documents one thing that cost me a debugging pass: the dynamic_wi8 file needs litert-lm >= 0.16 — on 0.15 it fails to load with 'Unsupported model type', which is hard to trace back to a runtime version without trying both. Measured with the litert-lm CLI (benchmark -p 256 -d 256 --runs 3 --cache no) on an idle Apple M4 Max, generation-gated first (the backend produced correct text before any number was recorded). Happy to adjust the format if you'd like these to read differently.
Yoursmiling changed pull request status to merged