SolvrenAI/hipaa-fda-rag-eval
Viewer • Updated • 513 • 138
Nitya Gupta PRO
nitya-gupta
·
AI & ML interests
* NLP
* Fine Tuning
* RAG
* Agentic AI
* Physical AI
Recent Activity
updated a dataset 14 days ago
SolvrenAI/hipaa-fda-rag-eval published a dataset 26 days ago
SolvrenAI/hipaa-fda-rag-eval updated a Space 26 days ago
SolvrenAI/READMEOrganizations
updated a
dataset 14 days ago
published a
dataset 26 days ago
updated a
Space 26 days ago
published a
Space 26 days ago
published 2
datasets 26 days ago
published 2
models 26 days ago
updated a
dataset 26 days ago
updated a
model 26 days ago
updated a
dataset 26 days ago
updated a
model 26 days ago
reacted to harshitkgupta's post with 🚀 about 2 months ago
Post
8,763
Fine-tuned Qwen 2.5 (0.5B → 3B) on real coding-agent traces, 10 controlled runs, one 16GB Mac. Compared PyTorch MPS vs. Apple MLX for local LoRA SFT — and the honest answer is "it depends on what you're optimizing for":
• PyTorch MPS: 2.2x–5.7x faster raw throughput, but hits a hard memory wall — can't load a 3B model in FP16 on 16GB.
• Apple MLX: 4-bit QLoRA fits 3B+ models with almost flat memory scaling as context grows (+109 MB going from 1k→4k tokens).
• 4-bit quantization doesn't cost you convergence — eval loss tracks closely across backends.
• The bigger surprise: most of MLX's slowdown isn't the 4-bit dequant tax. Two of the 10 runs went unquantized to isolate it — dequant only explains 1.07x–1.4x of the gap. A ~4.1–4.6x framework-level gap remains either way.
All 10 LoRA adapters + Trackio logs are public so the numbers are checkable, not just claimed.
Full writeup: https://huggingface.co/blog/harshitkgupta/fine-tuning-coding-agents-on-mac-pytorch-mps-mlx
• PyTorch MPS: 2.2x–5.7x faster raw throughput, but hits a hard memory wall — can't load a 3B model in FP16 on 16GB.
• Apple MLX: 4-bit QLoRA fits 3B+ models with almost flat memory scaling as context grows (+109 MB going from 1k→4k tokens).
• 4-bit quantization doesn't cost you convergence — eval loss tracks closely across backends.
• The bigger surprise: most of MLX's slowdown isn't the 4-bit dequant tax. Two of the 10 runs went unquantized to isolate it — dequant only explains 1.07x–1.4x of the gap. A ~4.1–4.6x framework-level gap remains either way.
All 10 LoRA adapters + Trackio logs are public so the numbers are checkable, not just claimed.
Full writeup: https://huggingface.co/blog/harshitkgupta/fine-tuning-coding-agents-on-mac-pytorch-mps-mlx
- 7 replies
upvoted an article about 2 months ago
Article
Fine-Tuning Coding Agents on a 16GB Mac: PyTorch MPS vs. Apple MLX
harshitkgupta
• • 3