Update README.md with new model card content

README.md

@@ -1,22 +1,191 @@
 ---
 library_name: keras-hub
 ---
-This is a [`ModernBert` model](https://keras.io/api/keras_hub/models/modern_bert) uploaded using the KerasHub library and can be used with JAX, TensorFlow, and PyTorch backends.
-This model is related to a `MaskedLM` task.
-
-Model config:
-* **name:** modern_bert_backbone
-* **trainable:** True
-* **dtype:** {'module': 'keras', 'class_name': 'DTypePolicy', 'config': {'name': 'float32'}, 'registered_name': None}
-* **vocabulary_size:** 50368
-* **hidden_dim:** 1024
-* **intermediate_dim:** 2624
-* **num_layers:** 28
-* **num_heads:** 16
-* **local_attention_window:** 128
-* **global_attn_every_n_layers:** 3
-* **rotary_max_wavelength:** 160000.0
-* **local_rotary_max_wavelength:** 10000.0
-* **layer_norm_epsilon:** 1e-05
-
-This model card has been generated automatically and should be completed by the model author. See [Model Cards documentation](https://huggingface.co/docs/hub/model-cards) for more information.
+### Model Overview
+### Model Overview
+ModernBERT is a family of encoder-only Transformer models published by [Answer.AI](https://www.answer.ai/) and [LightOn](https://www.lighton.ai/). It is a modernized drop-in replacement for BERT: pre-trained on 2 trillion tokens of English and code with a native 8,192-token context window, it uses Rotary Position Embeddings (RoPE), alternating local and global attention, and GeGLU feed-forward layers. ModernBERT is intended for classification, retrieval, and embedding of text, not for text-generation. See the model card below for benchmarks, data sources, and intended use cases.
+
+Weights are released under the [Apache 2 License](https://github.com/AnswerDotAI/ModernBERT/blob/main/LICENSE). Keras model code is released under the [Apache 2 License](https://github.com/keras-team/keras-hub/blob/master/LICENSE).
+
+## Links
+
+* [ModernBERT Quickstart Notebook](https://www.kaggle.com/code/ruchithaxwf/modernbert-quickstart)
+* [ModernBERT Model Card](https://huggingface.co/answerdotai/ModernBERT-base)
+* [ModernBERT Model Paper](https://arxiv.org/abs/2412.13663)
+* [KerasHub Beginner Guide](https://keras.io/guides/keras_hub/getting_started/)
+* [KerasHub Model Publishing Guide](https://keras.io/guides/keras_hub/upload/)
+
+## Installation
+
+Keras and KerasHub can be installed with:
+
+```
+pip install -U -q keras-hub
+pip install -U -q keras
+```
+
+Jax, TensorFlow, and Torch come preinstalled in Kaggle Notebooks. For instructions on installing them in another environment see the [Keras Getting Started](https://keras.io/getting_started/) page.
+
+## Presets
+
+The following model checkpoints are provided by the Keras team. Full code examples for each are available below.
+
+| Preset name            | Parameters | Description                                                                                                                                            |
+|------------------------|------------|--------------------------------------------------------------------------------------------------------------------------------------------------------|
+| `modern_bert_base_en`  | 149.01M    | 22-layer ModernBERT Base encoder model pretrained on English for masked language modeling. Uses RoPE, alternating local and global attention, and GeGLU feedforward layers.  |
+| `modern_bert_large_en` | 394.78M    | 28-layer ModernBERT Large encoder model pretrained on English for masked language modeling. Uses RoPE, alternating local and global attention, and GeGLU feedforward layers. |
+
+## Example Usage
+```python
+import keras
+import keras_hub
+import numpy as np
+```
+
+Raw string data.
+
+```python
+features = ["The quick brown fox jumped.", "I forgot my homework."]
+labels = [0, 3]
+
+# Pretrained classifier.
+classifier = keras_hub.models.ModernBertTextClassifier.from_preset(
+    "modern_bert_large_en",
+    num_classes=4,
+)
+classifier.fit(x=features, y=labels, batch_size=2)
+classifier.predict(x=features, batch_size=2)
+
+# Re-compile (e.g., with a new learning rate).
+classifier.compile(
+    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+    optimizer=keras.optimizers.Adam(5e-5),
+    jit_compile=True,
+)
+# Access backbone programmatically (e.g., to change `trainable`).
+classifier.backbone.trainable = False
+# Fit again.
+classifier.fit(x=features, y=labels, batch_size=2)
+```
+
+Preprocessed integer data.
+
+```python
+features = {
+    "token_ids": np.ones(shape=(2, 12), dtype="int32"),
+    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
+}
+labels = [0, 3]
+
+# Pretrained classifier without preprocessing.
+classifier = keras_hub.models.ModernBertTextClassifier.from_preset(
+    "modernbert_base_en",
+    num_classes=4,
+    preprocessor=None,
+)
+classifier.fit(x=features, y=labels, batch_size=2)
+```
+
+Masked language modeling.
+
+```python
+masked_lm = keras_hub.models.ModernBertMaskedLM.from_preset("modern_bert_large_en")
+tokenizer = masked_lm.preprocessor.tokenizer
+
+text = "The capital of France is [MASK]."
+token_ids = (
+    [tokenizer.cls_token_id]
+    + list(np.asarray(tokenizer(text)))
+    + [tokenizer.sep_token_id]
+)
+mask_positions = [
+    i for i, t in enumerate(token_ids) if t == tokenizer.mask_token_id
+]
+logits = masked_lm(
+    {
+        "token_ids": np.array([token_ids], dtype="int32"),
+        "padding_mask": np.ones((1, len(token_ids)), dtype="bool"),
+        "mask_positions": np.array([mask_positions], dtype="int32"),
+    }
+)
+top5 = np.argsort(-np.asarray(logits)[0, 0])[:5]
+print([tokenizer.detokenize([int(t)]).strip() for t in top5])
+```
+
+## Example Usage with Hugging Face URI
+
+```python
+import keras
+import keras_hub
+import numpy as np
+```
+
+Raw string data.
+
+```python
+features = ["The quick brown fox jumped.", "I forgot my homework."]
+labels = [0, 3]
+
+# Pretrained classifier.
+classifier = keras_hub.models.ModernBertTextClassifier.from_preset(
+    "hf://keras/modern_bert_large_en",
+    num_classes=4,
+)
+classifier.fit(x=features, y=labels, batch_size=2)
+classifier.predict(x=features, batch_size=2)
+
+# Re-compile (e.g., with a new learning rate).
+classifier.compile(
+    loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
+    optimizer=keras.optimizers.Adam(5e-5),
+    jit_compile=True,
+)
+# Access backbone programmatically (e.g., to change `trainable`).
+classifier.backbone.trainable = False
+# Fit again.
+classifier.fit(x=features, y=labels, batch_size=2)
+```
+
+Preprocessed integer data.
+
+```python
+features = {
+    "token_ids": np.ones(shape=(2, 12), dtype="int32"),
+    "padding_mask": np.array([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0]] * 2),
+}
+labels = [0, 3]
+
+# Pretrained classifier without preprocessing.
+classifier = keras_hub.models.ModernBertTextClassifier.from_preset(
+    "modernbert_base_en",
+    num_classes=4,
+    preprocessor=None,
+)
+classifier.fit(x=features, y=labels, batch_size=2)
+```
+
+Masked language modeling.
+
+```python
+masked_lm = keras_hub.models.ModernBertMaskedLM.from_preset("hf://keras/modern_bert_large_en")
+tokenizer = masked_lm.preprocessor.tokenizer
+
+text = "The capital of France is [MASK]."
+token_ids = (
+    [tokenizer.cls_token_id]
+    + list(np.asarray(tokenizer(text)))
+    + [tokenizer.sep_token_id]
+)
+mask_positions = [
+    i for i, t in enumerate(token_ids) if t == tokenizer.mask_token_id
+]
+logits = masked_lm(
+    {
+        "token_ids": np.array([token_ids], dtype="int32"),
+        "padding_mask": np.ones((1, len(token_ids)), dtype="bool"),
+        "mask_positions": np.array([mask_positions], dtype="int32"),
+    }
+)
+top5 = np.argsort(-np.asarray(logits)[0, 0])[:5]
+print([tokenizer.detokenize([int(t)]).strip() for t in top5])
+```