msluszniak's picture
Unpublish variants that lose on device
6ef6540 verified
|
Raw History Blame Contribute Delete
261 Bytes

Exported and not published. On a Galaxy S26 Ultra it decodes at 93 ms/token against 28.5 ms for XNNPACK 8da8w, and its output is wrong: RMSNorm is decomposed in fp16 and the residual stream's outliers (|x| up to 1749) overflow x*x. Use xnnpack/ on Android.