gbert-base-mlx / README.md
pkcpkc's picture
deepset/gbert-base prepared for oMLX (float16, mean pooling)
013877f verified
|
Raw History Blame Contribute Delete
1.85 kB
---
license: mit
base_model: deepset/gbert-base
base_model_relation: quantized
library_name: mlx
pipeline_tag: feature-extraction
language:
- de
tags:
- mlx
- bert
- omlx
---
# gbert-base-mlx
[deepset/gbert-base](https://huggingface.co/deepset/gbert-base) (German BERT,
110M parameters, 512 tokens) prepared for [oMLX](https://github.com/jundot/omlx):
float16 safetensors and mean pooling.
## How it was made
oMLX loads BERT natively from safetensors, but the original checkpoint doesn't
fit its loader. The changes:
- **Weights:** `bert.` prefix removed from the names, pretraining heads
(`cls.*`) dropped, cast to float16.
- **`config.json`:** `model_type: bert`, `architectures: [BertModel]`,
`layer_norm_eps: 1e-12`, `pad_token_id: 0`. Without these, oMLX falls back
to XLM-RoBERTa defaults and silently numbers positions wrongly.
- **`tokenizer_config.json`:** `tokenizer_class: BertTokenizer`,
`model_max_length: 512`, special tokens.
- **Pooling:** `modules.json` and `1_Pooling/config.json` select the mean over
real tokens (padding excluded), normalized to length 1, instead of BERT's
pretraining pooler output.
Tested on oMLX 0.7.1: vectors of a comment are identical alone or batched with
longer inputs (max difference 2e-4, float16 rounding).
## Use
Copy the folder into oMLX's model directory. oMLX serves it as an embedding
model on `POST /v1/embeddings` (model id `gbert-base-mlx`), 768-d vectors.
This is the pretrained masked-language model, not a sentence-embedding
model. For classification or search, train a head or fine-tune it first.
## Licence
MIT, see `license.txt` (Copyright 2022 deepset GmbH). Credit for the model
goes to deepset; see the [original model card](https://huggingface.co/deepset/gbert-base)
for the paper and citation. This conversion is not affiliated with or endorsed
by deepset.