Inkling-2-Mini-oQ2e

This model was quantized using oQ (oMLX v0.8.2) mixed-precision quantization.

The effective average is 3.215 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

Storage format Logical weights Share of total Tensor storage Effective bits/weight
Affine 2-bit, group 64 24.461B 91.592% 7.119 GiB 2.50
Affine 8-bit, group 64 1.572B 5.887% 1.556 GiB 8.50
BF16 0.637B 2.387% 1.187 GiB 16.00
FP32 0.036B 0.133% 0.133 GiB 32.00
Total 26.707B 100% 9.995 GiB 3.215

Storage Breakdown

Component Logical weights Storage
Language model 26.388B 10.096 GB / 9.402 GiB
Vision 0.318B 0.637 GB / 0.593 GiB
Total 26.707B 10.732 GB / 9.995 GiB
Downloads last month
-
Safetensors
Model size
26B params
Tensor type
U32
路
BF16
路
F32
路
MLX
Hardware compatibility
Log In to add your hardware

2-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for g-oQ/Inkling-2-Mini-oQ2e

Quantized
(94)
this model