gte-modernbert-base-mlx-8bit

8-bit integer affine quantization of the embedder's linear weights (group size 64), produced with MLX. CLS pooling as the source's 1_Pooling/config.json. Measured against the float16 source on the Premisa retrieval study's chunk set (180 questions, 50-candidate pools): 98 percent top-8 neighbour overlap; the full study arm at 2000-character chunks and pool 50 matched the source's delivered-hit figures. English-only, as the source.

Source: Alibaba-NLP/gte-modernbert-base at revision e7f32e3c00f91d699e8c43b53106206bcc72bb22 (source weights SHA-256 3e85899d5728cb7de79781c0c3acfb91ccef9f875f1f7e0b3c9f3dd4b6a724ba), quantized on 2026-10-21 with Premisa's model_quant_spike.py quantize --bits 8 (mode affine, group size 64). Serving: the mlx-vlm fork's task server (python -m mlx_vlm.server --embedding-model or --reranker-model with this directory); the pooling configuration is read from 1_Pooling/config.json where present.

License: the source model's license applies to these derived weights (apache-2.0); see the source card for its terms. Quantization and measurements by Premisa (ai.premisa.desktop); figures are pre-adjudication study measurements, not quality claims.

Downloads last month
61
Safetensors
Model size
0.1B params
Tensor type
F16
ยท
U32
ยท
MLX
Hardware compatibility
Log In to add your hardware

Quantized

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for caslca/gte-modernbert-base-mlx-8bit

Finetuned
(43)
this model