Inkling-2-Mini-oQ6e

This model was quantized using oQ (oMLX v0.8.2) mixed-precision quantization.

The effective average is 5.352 bits/weight for the complete checkpoint.

Quantization and Bit Distribution

Storage format Logical weights Share of total Tensor storage Effective bits/weight
NVFP4, group 16 20.384B 76.327% 10.679 GiB 4.50
Affine 6-bit, group 64 4.077B 15.265% 3.085 GiB 6.50
Affine 8-bit, group 64 1.572B 5.887% 1.556 GiB 8.50
BF16 0.637B 2.387% 1.187 GiB 16.00
FP32 0.036B 0.133% 0.133 GiB 32.00
Total 26.707B 100% 16.640 GiB 5.352

Storage Breakdown

Component Logical weights Storage
Language model 26.388B 17.230 GB / 16.047 GiB
Vision 0.318B 0.637 GB / 0.593 GiB
Total 26.707B 17.867 GB / 16.640 GiB
Downloads last month
-
Safetensors
Model size
26B params
Tensor type
U32
路
BF16
路
F32
路
MLX
Hardware compatibility
Log In to add your hardware

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 馃檵 Ask for provider support

Model tree for g-oQ/Inkling-2-Mini-oQ6e

Quantized
(94)
this model