Distillation-Aware Weight Quant of models. The RTN is further trained using data generated from BF16 model or 8bit RTN quant if BF16 is not feasible.
Hermit Dave
hermitdave
AI & ML interests
None yet
Recent Activity
updated a dataset about 4 hours ago
hermitdave/Inkling-2-Mini-imatrix published a dataset about 4 hours ago
hermitdave/Inkling-2-Mini-imatrix updated a model about 4 hours ago
hermitdave/Inkling-2-Mini-MLX-RCO-3.5bpwOrganizations
Statistically lossless quantisation
MLX quants done using a port of IST-DASLab SLQ. The mechanism used is Heuristics based greedy allocation. Not the one paper suggests
-
hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw
Text Generation • 3B • Updated • 538 -
hermitdave/LFM2.5-2.6B-MLX-SLQ-4.49bpw
Text Generation • 0.6B • Updated • 381 -
hermitdave/Qwen3-1.7B-MLX-slq-4.49bpw
Text Generation • 0.5B • Updated • 276 -
hermitdave/Spark-X2.6-4B-MLX-SLQ-4.50bpw
Text Generation • 0.9B • Updated • 844
Distillation-Aware Weight Quantization
Distillation-Aware Weight Quant of models. The RTN is further trained using data generated from BF16 model or 8bit RTN quant if BF16 is not feasible.
Statistically lossless quantisation
MLX quants done using a port of IST-DASLab SLQ. The mechanism used is Heuristics based greedy allocation. Not the one paper suggests
-
hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw
Text Generation • 3B • Updated • 538 -
hermitdave/LFM2.5-2.6B-MLX-SLQ-4.49bpw
Text Generation • 0.6B • Updated • 381 -
hermitdave/Qwen3-1.7B-MLX-slq-4.49bpw
Text Generation • 0.5B • Updated • 276 -
hermitdave/Spark-X2.6-4B-MLX-SLQ-4.50bpw
Text Generation • 0.9B • Updated • 844
models 77
hermitdave/Inkling-2-Mini-MLX-RCO-3.5bpw
26B • Updated
hermitdave/Inkling-2-Mini-MLX-RCO-4.5bpw
26B • Updated
hermitdave/Agnes-3.0-Qwen-RCO-2.27bpw
Image-Text-to-Text • 180B • Updated
hermitdave/Agnes-3.0-Qwen-RCO-2.5bpw
Image-Text-to-Text • 180B • Updated
hermitdave/Qwen3.8-27B-hermes-MLX-RCO-4.5bpw
Image-Text-to-Text • 28B • Updated • 45
hermitdave/LFM2.5-24B-A2B-MLX-RCO-4.5bpw
24B • Updated • 134
hermitdave/Nemotron-3.5-Bolt-74B-A9B-MLX-RCO-3.0bpw
Text Generation • 74B • Updated • 102
hermitdave/Nemotron-3.5-Bolt-74B-A9B-MLX-RCO-4.0bpw
Text Generation • 74B • Updated • 179
hermitdave/SenseNova-Loop-9B-MLX-RCO-4.5bpw-imatrix-ours-4x6
Text Generation • 10B • Updated • 271
hermitdave/Recursive-Flint-39.8B-A2.5B-MLX-RCO-3.0bpw
40B • Updated • 47
datasets 8
hermitdave/Inkling-2-Mini-imatrix
Updated
hermitdave/SenseNova-Loop-9B-sensitivity-results
Updated • 38
hermitdave/flint-imatrix
Updated • 53
hermitdave/Qwen3.6-35B-A3B-imatrix
Viewer • Updated • 1 • 49
hermitdave/LFM2.5-24B-A2B-imatrix
Updated • 38
hermitdave/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-imatrix
Updated • 54
hermitdave/Nemotron-3.5-Bolt-74B-A9B-sensitivity-results
Viewer • Updated • 6 • 183
hermitdave/qwen3.8-27b-sensitivity
Preview • Updated • 39