Token Classification
Transformers
Safetensors
English
Hebrew
xlm-roberta
code-switching
hebrew
transliteration
romanization
text-to-speech
Eval Results (legacy)
Instructions to use danielrosehill/hebrew-latin-token-classifier with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use danielrosehill/hebrew-latin-token-classifier with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("token-classification", model="danielrosehill/hebrew-latin-token-classifier")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForTokenClassification tokenizer = AutoTokenizer.from_pretrained("danielrosehill/hebrew-latin-token-classifier") model = AutoModelForTokenClassification.from_pretrained("danielrosehill/hebrew-latin-token-classifier", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Train FacebookAI/xlm-roberta-base on danielrosehill/hebrew-latin-code-switching: test F1 0.941
e6cfd24 verified Download test_errors.jsonl from danielrosehill/hebrew-latin-token-classifier: direct link, hf CLI and curl.
- Browser
- Download file 5.44 kB
-
https://huggingface.co/danielrosehill/hebrew-latin-token-classifier/resolve/main/test_errors.jsonl
- Command line
-
hf download hf://danielrosehill/hebrew-latin-token-classifier/test_errors.jsonl
-
curl -L -o test_errors.jsonl https://huggingface.co/danielrosehill/hebrew-latin-token-classifier/resolve/main/test_errors.jsonl
5.44 kB
| {"id": "g00057", "text": "I picked up some msabbaha from the shuk on Friday — the guy there makes it with just the right amount of lemon.", "gold": [], "pred": ["msabbaha"]} | |
| {"id": "g00059", "text": "Msabbaha again? You had that yesterday.", "gold": [], "pred": ["Msabbaha"]} | |
| {"id": "g00159", "text": "We're already late, so yalla, get in the car.", "gold": [], "pred": ["yalla"]} | |
| {"id": "g00259", "text": "Yeah, we're olim, just landed last week.", "gold": ["olim"], "pred": []} | |
| {"id": "g00260", "text": "The absorption center helps new olim navigate the bureaucracy of health insurance and ulpan registration.", "gold": ["olim"], "pred": ["ulpan"]} | |
| {"id": "g00432", "text": "Have you ever grabbed a coffee on dizengoff on a Friday morning?", "gold": [], "pred": ["dizengoff"]} | |
| {"id": "g00433", "text": "Yeah, I'll meet you at dizengoff, no problem.", "gold": [], "pred": ["dizengoff"]} | |
| {"id": "g00434", "text": "We spent the whole afternoon wandering around dizengoff, popping into shops and arguing about where to eat.", "gold": [], "pred": ["dizengoff"]} | |
| {"id": "g00545", "text": "We celebrated sukkot in Tel Aviv last year and the weather was perfect.", "gold": ["sukkot"], "pred": []} | |
| {"id": "g00576", "text": "I grabbed some peppers and eggs at the shuk because I'm making shakshuka tonight.", "gold": ["shakshuka"], "pred": []} | |
| {"id": "g00577", "text": "Do you know if that new deli sells shakshuka by the container?", "gold": ["shakshuka"], "pred": []} | |
| {"id": "g00578", "text": "Shakshuka? I thought we were buying falafel.", "gold": ["Shakshuka"], "pred": []} | |
| {"id": "g00705", "text": "I finally got my mas hachnasa hafkaa approved after three months of paperwork.", "gold": [], "pred": ["hafkaa"]} | |
| {"id": "g00708", "text": "I need the heshbonit mas kabala from the contractor before I can pay him.", "gold": ["heshbonit", "mas", "kabala"], "pred": ["heshbonit"]} | |
| {"id": "g00709", "text": "Did you remember to ask for a heshbonit mas kabala at the supermarket?", "gold": ["heshbonit", "mas", "kabala"], "pred": ["heshbonit", "mas"]} | |
| {"id": "g00807", "text": "I used to grab lunch in ramat gan every Thursday when I worked nearby.", "gold": [], "pred": ["ramat", "gan"]} | |
| {"id": "g00808", "text": "Have you ever tried the little Yemenite place in ramat gan?", "gold": [], "pred": ["gan"]} | |
| {"id": "g00999", "text": "I bought lechem at the shuk this morning.", "gold": [], "pred": ["lechem"]} | |
| {"id": "g01035", "text": "Do we need more babaganoush for the party, or is one tub enough?", "gold": [], "pred": ["babaganoush"]} | |
| {"id": "g01052", "text": "Just toss the dirty laundry in the cheder.", "gold": ["cheder"], "pred": []} | |
| {"id": "g01069", "text": "How do you file mas hachnasa shelo mitzvot if you're a freelancer?", "gold": [], "pred": ["mas", "hachnasa"]} | |
| {"id": "g01070", "text": "mas hachnasa shelo mitzvot, again? I thought I was done with that.", "gold": [], "pred": ["mas", "hachnasa"]} | |
| {"id": "g01134", "text": "Did you remember to buy botz?", "gold": ["botz"], "pred": []} | |
| {"id": "g01135", "text": "No botz, no party.", "gold": ["botz"], "pred": []} | |
| {"id": "g01365", "text": "Do we have any besamim for Havdalah tonight, or did we finish the last box?", "gold": ["besamim", "Havdalah"], "pred": ["besamim"]} | |
| {"id": "g01626", "text": "Od paam?! I told you to lock the door before you left.", "gold": [], "pred": ["Od", "paam"]} | |
| {"id": "g01627", "text": "Can you play that song od paam, but slower this time?", "gold": [], "pred": ["od", "paam"]} | |
| {"id": "g01628", "text": "Every time we argue about the dishes it's the same od paam conversation, and honestly I'm tired of it.", "gold": [], "pred": ["od"]} | |
| {"id": "g01869", "text": "Did you remember to take your kapsulot this morning?", "gold": [], "pred": ["kapsulot"]} | |
| {"id": "g02040", "text": "Why are you being so chamutz today?", "gold": ["chamutz"], "pred": []} | |
| {"id": "g02042", "text": "I bought a bag of those little green apples, but they were so chamutz that even the kids wouldn't eat them.", "gold": ["chamutz"], "pred": []} | |
| {"id": "g02114", "text": "I keep the sakin in the drawer next to the stove.", "gold": [], "pred": ["sakin"]} | |
| {"id": "g02247", "text": "Which mishnah are we up to this week?", "gold": [], "pred": ["mishnah"]} | |
| {"id": "g02248", "text": "mishnah? I haven't even opened the book.", "gold": [], "pred": ["mishnah"]} | |
| {"id": "g02249", "text": "Last night I fell asleep halfway through the mishnah.", "gold": [], "pred": ["mishnah"]} | |
| {"id": "g03034", "text": "yeri again, right outside the mall.", "gold": ["yeri"], "pred": []} | |
| {"id": "g03271", "text": "Mechiron on everything—don't miss it!", "gold": ["Mechiron"], "pred": []} | |
| {"id": "g03992", "text": "Tzarchania on the corner, ten minutes before Shabbat — total chaos.", "gold": ["Tzarchania"], "pred": []} | |
| {"id": "d_20261014T200315Z_kcg6w6", "text": "I'm circling around Talpiot but I can't find chanaya anywhere, so I'll be about fifteen minutes late.", "gold": ["chanaya"], "pred": ["Talpiot", "chanaya"]} | |
| {"id": "d_20261015T065549Z_d79is9", "text": "Don't forget to pick up chatzilim on the way home, the baba ganoush is no good without them.", "gold": ["chatzilim"], "pred": ["chatzilim", "baba", "ganoush"]} | |
| {"id": "d_20261015T064357Z_txiltm", "text": "Hey, I won't be around much during Chol Hamoed, so please recreate the venv on the staging server before then and check that the requirements install cleanly.", "gold": ["Chol", "Hamoed"], "pred": ["Chol", "Hamoed", "venv"]} | |