Converted to mlx

#2
by bukaosiji - opened

Hi everyone! For Apple Silicon / MLX users, I've converted RadixArk/DaisyRose-27B to standard 4-bit MLX format and uploaded it to mlx-community:

๐Ÿ‘‰ mlx-community/DaisyRose-27B-4bit

Conversion Details

  • Converter: mlx-vlm v0.8.0 (full multimodal vision_tower preserved โ€” works with both mlx-vlm for vision+text and mlx-lm for text-only / local server)
  • Quantization: 4-bit affine, group_size=64
  • Size: ~15 GB (14.95 GiB across 3 safetensors shards) โ€” fits comfortably on 24 GB+ Apple Silicon Macs
  • Sanity checks: Verified zero NaN/Inf across all tensors and smoke-tested both text-only and image+text generation.

Quick Start

With mlx-vlm (text + image):

pip install -U mlx-vlm
python -m mlx_vlm generate \
  --model mlx-community/DaisyRose-27B-4bit \
  --max-tokens 256 \
  --prompt "Describe this image." \
  --image <path_to_image>

With mlx-lm (text-only):

pip install -U mlx-lm
mlx_lm.generate \
  --model mlx-community/DaisyRose-27B-4bit \
  --max-tokens 256 \
  --prompt "Write a quick bash script to check open ports."

Feedback or requests for other quantization levels (e.g., 6-bit / 8-bit) are welcome!

Sign up or log in to comment