8-bit quantization

by amarosnithe - opened Apr 8

Apr 8

Hi, could you guys (or anyone) quantize this to fp8? The ones that've been done so far are only for MLX (so I can't run them as a Windows user).

Aug 23

Looks like people did, thanks guys!

amarosnithe changed discussion status to closed Aug 23

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment