Post
513
I just built my first space using Midnight-Miqu-103B (4-bit Quantization) and it's super slow. Can anyone tell me what's possibly wrong?
Join the community of Machine Learners and AI enthusiasts.
Sign UpThe model is quite large, so it may be slow because it lacks the VRAM for the space and is offloaded to the normal RAM.
Incidentally, the free Spaces has 0 VRAM.
I'm using 4xL40S which should provide plenty of VRAM for the model.