noahsantacruz commited on
Commit
f876f1e
·
verified ·
1 Parent(s): fcdcc29

Rename to BERTZALEL

Browse files
Files changed (1) hide show
  1. README.md +5 -5
README.md CHANGED
@@ -14,11 +14,11 @@ tags:
14
  - sefaria
15
  ---
16
 
17
- # ModernBERTorah
18
 
19
  **An English-native encoder that understands Torah vocabulary.**
20
 
21
- ModernBERTorah is [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) with its masked-language-model pretraining continued on Sefaria's English corpus. The goal is a general-purpose English encoder that knows the vocabulary of Jewish texts: names of sages and books, Hebrew and Aramaic terms transliterated in English prose, and the citation and commentary conventions of the Jewish library.
22
 
23
  General English encoders see this material rarely, so they split terms like *Mishneh Torah*, *Tosafot* or *Ketubot* into meaningless fragments and know little about how they are used. This model is meant as a better starting point for fine-tuning on English-language Jewish text, for example NER (people, citations), classification, retrieval and linking.
24
 
@@ -40,7 +40,7 @@ The Hebrew tokens let the model read Hebrew quoted inline in English text withou
40
  ```python
41
  from transformers import pipeline
42
 
43
- fill = pipeline("fill-mask", model="Sefaria/ModernBERTorah")
44
  fill("Rashi was a medieval [MASK] who wrote a commentary on the Torah.")
45
  # commentator, scholar, sage, ...
46
  ```
@@ -50,8 +50,8 @@ For fine-tuning, load it like any ModernBERT checkpoint:
50
  ```python
51
  from transformers import AutoModelForTokenClassification, AutoTokenizer
52
 
53
- tok = AutoTokenizer.from_pretrained("Sefaria/ModernBERTorah")
54
- model = AutoModelForTokenClassification.from_pretrained("Sefaria/ModernBERTorah", num_labels=5)
55
  ```
56
 
57
  Requires `transformers>=4.48`.
 
14
  - sefaria
15
  ---
16
 
17
+ # BERTZALEL
18
 
19
  **An English-native encoder that understands Torah vocabulary.**
20
 
21
+ BERTZALEL stands for **B**idirectional **E**ncoder **R**epresentations from **T**ransformers, **Z**eroed-in on **A**ggadah, **L**aw, **E**xegesis & **L**iturgy. It is [ModernBERT-base](https://huggingface.co/answerdotai/ModernBERT-base) with its masked-language-model pretraining continued on Sefaria's English corpus. The goal is a general-purpose English encoder that knows the vocabulary of Jewish texts: names of sages and books, Hebrew and Aramaic terms transliterated in English prose, and the citation and commentary conventions of the Jewish library.
22
 
23
  General English encoders see this material rarely, so they split terms like *Mishneh Torah*, *Tosafot* or *Ketubot* into meaningless fragments and know little about how they are used. This model is meant as a better starting point for fine-tuning on English-language Jewish text, for example NER (people, citations), classification, retrieval and linking.
24
 
 
40
  ```python
41
  from transformers import pipeline
42
 
43
+ fill = pipeline("fill-mask", model="Sefaria/BERTZALEL")
44
  fill("Rashi was a medieval [MASK] who wrote a commentary on the Torah.")
45
  # commentator, scholar, sage, ...
46
  ```
 
50
  ```python
51
  from transformers import AutoModelForTokenClassification, AutoTokenizer
52
 
53
+ tok = AutoTokenizer.from_pretrained("Sefaria/BERTZALEL")
54
+ model = AutoModelForTokenClassification.from_pretrained("Sefaria/BERTZALEL", num_labels=5)
55
  ```
56
 
57
  Requires `transformers>=4.48`.