Distillation-Aware Weight Quant of models. The RTN is further trained using data generated from BF16 model or 8bit RTN quant if BF16 is not feasible.
Hermit Dave
hermitdave
·
AI & ML interests
None yet
Recent Activity
updated a dataset about 4 hours ago
hermitdave/Inkling-2-Mini-imatrix published a dataset about 4 hours ago
hermitdave/Inkling-2-Mini-imatrix updated a model about 5 hours ago
hermitdave/Inkling-2-Mini-MLX-RCO-3.5bpwOrganizations
oQ Quants
oQ qunats built with oMLX
NeoHorse 1 9B Dense
NeoHorse 1 9B Dense quants
Qwen 3.6 35B A3B
MLX mixed quant experiments
Statistically lossless quantisation
MLX quants done using a port of IST-DASLab SLQ. The mechanism used is Heuristics based greedy allocation. Not the one paper suggests
-
hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw
Text Generation • 3B • Updated • 538 -
hermitdave/LFM2.5-2.6B-MLX-SLQ-4.49bpw
Text Generation • 0.6B • Updated • 381 -
hermitdave/Qwen3-1.7B-MLX-slq-4.49bpw
Text Generation • 0.5B • Updated • 276 -
hermitdave/Spark-X2.6-4B-MLX-SLQ-4.50bpw
Text Generation • 0.9B • Updated • 844
BigBang v1
BigBang v1 35B A3B quants
K2 Horizon 36B A4B
K2 Horizon MoE 36B A4B
K2 Horizon 7B Dense
K2 Horizon MLX quants
Distillation-Aware Weight Quantization
Distillation-Aware Weight Quant of models. The RTN is further trained using data generated from BF16 model or 8bit RTN quant if BF16 is not feasible.
Statistically lossless quantisation
MLX quants done using a port of IST-DASLab SLQ. The mechanism used is Heuristics based greedy allocation. Not the one paper suggests
-
hermitdave/K2-Horizon-7B-Uno-MLX-slq-4.5bpw
Text Generation • 3B • Updated • 538 -
hermitdave/LFM2.5-2.6B-MLX-SLQ-4.49bpw
Text Generation • 0.6B • Updated • 381 -
hermitdave/Qwen3-1.7B-MLX-slq-4.49bpw
Text Generation • 0.5B • Updated • 276 -
hermitdave/Spark-X2.6-4B-MLX-SLQ-4.50bpw
Text Generation • 0.9B • Updated • 844
oQ Quants
oQ qunats built with oMLX
BigBang v1
BigBang v1 35B A3B quants
NeoHorse 1 9B Dense
NeoHorse 1 9B Dense quants
K2 Horizon 36B A4B
K2 Horizon MoE 36B A4B
Qwen 3.6 35B A3B
MLX mixed quant experiments
K2 Horizon 7B Dense
K2 Horizon MLX quants