Instructions to use Sefaria/BERURIA with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Sefaria/BERURIA with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="Sefaria/BERURIA")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForMaskedLM tokenizer = AutoTokenizer.from_pretrained("Sefaria/BERURIA") model = AutoModelForMaskedLM.from_pretrained("Sefaria/BERURIA", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Rename to BERTZALEL
Browse files
README.md
CHANGED
|
@@ -14,11 +14,11 @@ tags:
|
|
| 14 |
- sefaria
|
| 15 |
---
|
| 16 |
|
| 17 |
-
#
|
| 18 |
|
| 19 |
**An English-native encoder that understands Torah vocabulary.**
|
| 20 |
|
| 21 |
-
|
| 22 |
|
| 23 |
General English encoders see this material rarely, so they split terms like *Mishneh Torah*, *Tosafot* or *Ketubot* into meaningless fragments and know little about how they are used. This model is meant as a better starting point for fine-tuning on English-language Jewish text, for example NER (people, citations), classification, retrieval and linking.
|
| 24 |
|
|
@@ -40,7 +40,7 @@ The Hebrew tokens let the model read Hebrew quoted inline in English text withou
|
|
| 40 |
```python
|
| 41 |
from transformers import pipeline
|
| 42 |
|
| 43 |
-
fill = pipeline("fill-mask", model="Sefaria/
|
| 44 |
fill("Rashi was a medieval [MASK] who wrote a commentary on the Torah.")
|
| 45 |
# commentator, scholar, sage, ...
|
| 46 |
```
|
|
@@ -50,8 +50,8 @@ For fine-tuning, load it like any ModernBERT checkpoint:
|
|
| 50 |
```python
|
| 51 |
from transformers import AutoModelForTokenClassification, AutoTokenizer
|
| 52 |
|
| 53 |
-
tok = AutoTokenizer.from_pretrained("Sefaria/
|
| 54 |
-
model = AutoModelForTokenClassification.from_pretrained("Sefaria/
|
| 55 |
```
|
| 56 |
|
| 57 |
Requires `transformers>=4.48`.
|
|
|
|
| 14 |
- sefaria
|
| 15 |
---
|
| 16 |
|
| 17 |
+
# BERTZALEL
|
| 18 |
|
| 19 |
**An English-native encoder that understands Torah vocabulary.**
|
| 20 |
|
| 21 |
+
BERTZALEL stands for **B**idirectional **E**ncoder **R**epresentations from **T**ransformers, **Z**eroed-in on **A**ggadah, **L**aw, **E**xegesis & **L**iturgy. It is [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) with its masked-language-model pretraining continued on Sefaria's English corpus. The goal is a general-purpose English encoder that knows the vocabulary of Jewish texts: names of sages and books, Hebrew and Aramaic terms transliterated in English prose, and the citation and commentary conventions of the Jewish library.
|
| 22 |
|
| 23 |
General English encoders see this material rarely, so they split terms like *Mishneh Torah*, *Tosafot* or *Ketubot* into meaningless fragments and know little about how they are used. This model is meant as a better starting point for fine-tuning on English-language Jewish text, for example NER (people, citations), classification, retrieval and linking.
|
| 24 |
|
|
|
|
| 40 |
```python
|
| 41 |
from transformers import pipeline
|
| 42 |
|
| 43 |
+
fill = pipeline("fill-mask", model="Sefaria/BERTZALEL")
|
| 44 |
fill("Rashi was a medieval [MASK] who wrote a commentary on the Torah.")
|
| 45 |
# commentator, scholar, sage, ...
|
| 46 |
```
|
|
|
|
| 50 |
```python
|
| 51 |
from transformers import AutoModelForTokenClassification, AutoTokenizer
|
| 52 |
|
| 53 |
+
tok = AutoTokenizer.from_pretrained("Sefaria/BERTZALEL")
|
| 54 |
+
model = AutoModelForTokenClassification.from_pretrained("Sefaria/BERTZALEL", num_labels=5)
|
| 55 |
```
|
| 56 |
|
| 57 |
Requires `transformers>=4.48`.
|