v6: reusable MLX conversion pipeline (venv, in-script token, prompts, cleanup menu) + full project docs
5545d68 |
Download CONVERSATION.md from jesusoctavioas/mlxtorepo: direct link, hf CLI and curl.
- Browser
- Download file 3.3 kB
-
https://huggingface.co/jesusoctavioas/mlxtorepo/resolve/main/CONVERSATION.md
- Command line
-
hf download hf://jesusoctavioas/mlxtorepo/CONVERSATION.md
-
curl -L -o CONVERSATION.md https://huggingface.co/jesusoctavioas/mlxtorepo/resolve/main/CONVERSATION.md
3.3 kB
Conversation: mlx-my-repo β local MLX conversion
Context
Working dir: /Users/jesusoctavioacostasantos/git
1. What is mlx-my-repo?
Explored /Users/jesusoctavioacostasantos/git/mlx-my-repo β it's the code for
the Hugging Face Space mlx-community/mlx-my-repo.
- A Gradio web app that converts HF models to MLX format and uploads the result
to the user's own HF namespace (e.g.
youruser/llama-mlx-Q4Bit). app.py: login button βHuggingfaceHubSearchmodel picker β dropdown (FP16 / Q2βQ8) βmlx_lm.convert()βHfApi.upload_file()per file.- Writes a README card with
base_model,mlxtag, and usage snippet. - Notable: runs
mlx[cpu]on Linux (HF Spaces) β slow, no Apple silicon. - Background APScheduler job reboots the Space every 6 hours (21600 s) to free memory.
clear_hf_cache_space()wipes the HF cache after each run (Space disk quota).demo.queue(default_concurrency_limit=1)β one conversion at a time.
2. How to implement it on your own HF repo
Two options discussed:
- Local conversion (recommended):
mlx_lm.convert+huggingface-cliβ much faster on a Mac since it uses Metal. - Fork the Space:
huggingface-cli repo create ... --type space --sdk gradio, copyapp.py+requirements.txt+ pngs, change the Space repo id inrestart_space(), push.
3. Can it convert Qwen3.8-Flash-Next?
Yes. Verified via HF API:
- Base model:
Qwen/Qwen3.8-Flash-Nextβ MoE (99B-A5B), multimodal180 GB** safetensors (131 shards).image-text-to-text, ** - Existing MLX conversions already on the Hub:
mlx-community/Qwen3.8-Flash-Next-oQ5e-mtp(7,950 dl)mlx-community/Qwen3.8-Flash-Next-oQ6e-mtp,oQ8e-mtpTensorFold/Qwen3.8-Flash-Next-MLX-4bit, and others
- Recommendation: avoid FP16 (~200 GB+ weights); Q4 is the sweet spot.
4. Step-by-step local conversion plan
- Free up ~230 GB (180 GB download + ~50 GB Q4 output). β οΈ At the time of conversation the Mac had only 45 GB free.
pip install -U huggingface_hub hf-transfer "mlx-lm>=0.29.1"+hf auth login- Repo:
youruser/Qwen3.8-Flash-Next-mlx-q4bit hf download Qwen/Qwen3.8-Flash-Next --local-dir <workdir>/Qwen3.8-Flash-Nextmlx_lm.convert --model <local-dir> --q-bits 4 --mlx-path <workdir>/...-4bit- Optional sanity check:
mlx_lm.generate --model ... --prompt "Say hello" - Write README card (
base_model, tags, usage snippet) hf upload youruser/Qwen3.8-Flash-Next-mlx-q4bit <converted-dir> --repo-type model- Clean up the 180 GB original
- Verify on the Hub
5. Final decisions (this repo)
- Folder:
mlxtorepo(this directory) - 230 GB lives on the internal disk (
~/mlxtorepo/work) - Repo is public; if the repo already exists β just upload the files
- Runs inside a venv (
~/mlxtorepo/.venv); HF token prompted inside the script - v5 β REUSABLE: prompts for the model repo and quantization (2-8 or fp16)
(also via
--model/--quantflags). Fixed bug:mlx_lm.convertneeds-qin addition to--q-bitsto actually quantize (v4 saved full precision). Note: mlx-lm affine supports q2βq8 (no q1); fp16 available. Version banner (convert_and_upload.sh β version: 5) printed on every run.