Experimental global target bits‑per‑weight quantization of openbmb/MiniCPM5-1B and openbmb/MiniCPM5-2B.
Unlike standard llama.cpp quantization that rely on fixed type heuristics (e.g., Q4_K_M), the Target BPW approach automatically optimizes per-tensor precision where it matters the most, and produces high quality models that meet a precise global size target.
Key Advantages: - VRAM Maximization: Can generate high quality models sized exactly to fit hardware constraints (e.g., fitting the model into exactly 24GB VRAM). - Data-Driven Precision: Quantization mix is determined by actual weight error sensitivity rather than hardcoded rules, often yielding better PPL/KLD size trade-offs.
Full benchmarks (PPL, KLD, ARC, GPQA, MMLU, etc.) and methodology in the model's card.
176 models, 54 orgs, 5 benchmarks, and a whole community of support!
Thanks to everyone who’s contributed models, reported issues, suggested benchmark improvements, or used the leaderboard to compare and evaluate small language models.
It’s been awesome watching the leaderboard grow into a broader community resource for transparent and reproducible SLM evaluation.
Published a guide to TurboQuant quantization: how the algorithm works and what Qdrant adds on top of it.
It also includes a benchmark comparing float32, scalar, binary and TurboQuant across BEIR's SciFact, ArguAna and NFCorpus, measured with recall@10, precision@10 and nDCG@10.