gbert-base-mlx / README.md
pkcpkc's picture
deepset/gbert-base prepared for oMLX (float16, mean pooling)
013877f verified
|
Raw History Blame Contribute Delete
1.85 kB
metadata
license: mit
base_model: deepset/gbert-base
base_model_relation: quantized
library_name: mlx
pipeline_tag: feature-extraction
language:
  - de
tags:
  - mlx
  - bert
  - omlx

gbert-base-mlx

deepset/gbert-base (German BERT, 110M parameters, 512 tokens) prepared for oMLX: float16 safetensors and mean pooling.

How it was made

oMLX loads BERT natively from safetensors, but the original checkpoint doesn't fit its loader. The changes:

  • Weights: bert. prefix removed from the names, pretraining heads (cls.*) dropped, cast to float16.
  • config.json: model_type: bert, architectures: [BertModel], layer_norm_eps: 1e-12, pad_token_id: 0. Without these, oMLX falls back to XLM-RoBERTa defaults and silently numbers positions wrongly.
  • tokenizer_config.json: tokenizer_class: BertTokenizer, model_max_length: 512, special tokens.
  • Pooling: modules.json and 1_Pooling/config.json select the mean over real tokens (padding excluded), normalized to length 1, instead of BERT's pretraining pooler output.

Tested on oMLX 0.7.1: vectors of a comment are identical alone or batched with longer inputs (max difference 2e-4, float16 rounding).

Use

Copy the folder into oMLX's model directory. oMLX serves it as an embedding model on POST /v1/embeddings (model id gbert-base-mlx), 768-d vectors.

This is the pretrained masked-language model, not a sentence-embedding model. For classification or search, train a head or fine-tune it first.

Licence

MIT, see license.txt (Copyright 2022 deepset GmbH). Credit for the model goes to deepset; see the original model card for the paper and citation. This conversion is not affiliated with or endorsed by deepset.