Horizon-Labs commited on
Commit
b34472d
·
verified ·
1 Parent(s): 8a263ba

Add multilingual educational-quality classifier

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -21,6 +21,7 @@ reports results against other open models, including where ours are worse.
21
  | **Phishing & Scam Detector** | flags phishing, scams and spam in emails, SMS and chats in many languages | [base](https://huggingface.co/Horizon-Labs/phishing-scam-detector-base) |
22
  | **Multilingual Toxicity** | Detoxify-style toxic-comment labels (toxicity, insult, threat, ...) in 34 languages | [small](https://huggingface.co/Horizon-Labs/multilingual-toxicity-small) · [base](https://huggingface.co/Horizon-Labs/multilingual-toxicity-base) |
23
  | **Multilingual Hate Speech** | hate speech and the targeted group (race, religion, origin, gender, sexuality, age, disability) in ~50 languages | [base](https://huggingface.co/Horizon-Labs/multilingual-hate-speech-base) |
 
24
  | **Hallucination Guard** | checks whether an answer is supported by its source (RAG, summaries) | [small](https://huggingface.co/Horizon-Labs/hallucination-guard-small) · [base](https://huggingface.co/Horizon-Labs/hallucination-guard-base) |
25
  | **Language Detection** | identifies the language of text among 183 languages, including short strings | [small](https://huggingface.co/Horizon-Labs/language-detection-small) |
26
  | **Punctuation Restoration** | restores . , ? : - in unpunctuated text such as ASR transcripts, in 88 languages | [small](https://huggingface.co/Horizon-Labs/punctuation-restoration-small) · [base](https://huggingface.co/Horizon-Labs/punctuation-restoration-base) |
@@ -42,7 +43,8 @@ Small = 141M and base = 308M parameters (mmBERT backbones, 8k context); large =
42
  [reranker](https://huggingface.co/spaces/Horizon-Labs/multilingual-reranker) ·
43
  [embedding search](https://huggingface.co/spaces/Horizon-Labs/multilingual-embedding) ·
44
  [toxicity](https://huggingface.co/spaces/Horizon-Labs/multilingual-toxicity) ·
45
- [hate speech](https://huggingface.co/spaces/Horizon-Labs/multilingual-hate-speech)
 
46
 
47
  **Data and evaluation**: [prompt-injection eval suite](https://huggingface.co/datasets/Horizon-Labs/prompt-injection-eval-suite) ·
48
  [injection detector leaderboard](https://huggingface.co/spaces/Horizon-Labs/prompt-injection-leaderboard) ·
 
21
  | **Phishing & Scam Detector** | flags phishing, scams and spam in emails, SMS and chats in many languages | [base](https://huggingface.co/Horizon-Labs/phishing-scam-detector-base) |
22
  | **Multilingual Toxicity** | Detoxify-style toxic-comment labels (toxicity, insult, threat, ...) in 34 languages | [small](https://huggingface.co/Horizon-Labs/multilingual-toxicity-small) · [base](https://huggingface.co/Horizon-Labs/multilingual-toxicity-base) |
23
  | **Multilingual Hate Speech** | hate speech and the targeted group (race, religion, origin, gender, sexuality, age, disability) in ~50 languages | [base](https://huggingface.co/Horizon-Labs/multilingual-hate-speech-base) |
24
+ | **Educational-Quality Classifier** | FineWeb-Edu style 0-5 educational score for web pages in 66 languages, for pretraining-data filtering | [small](https://huggingface.co/Horizon-Labs/multilingual-edu-classifier-small) · [base](https://huggingface.co/Horizon-Labs/multilingual-edu-classifier-base) |
25
  | **Hallucination Guard** | checks whether an answer is supported by its source (RAG, summaries) | [small](https://huggingface.co/Horizon-Labs/hallucination-guard-small) · [base](https://huggingface.co/Horizon-Labs/hallucination-guard-base) |
26
  | **Language Detection** | identifies the language of text among 183 languages, including short strings | [small](https://huggingface.co/Horizon-Labs/language-detection-small) |
27
  | **Punctuation Restoration** | restores . , ? : - in unpunctuated text such as ASR transcripts, in 88 languages | [small](https://huggingface.co/Horizon-Labs/punctuation-restoration-small) · [base](https://huggingface.co/Horizon-Labs/punctuation-restoration-base) |
 
43
  [reranker](https://huggingface.co/spaces/Horizon-Labs/multilingual-reranker) ·
44
  [embedding search](https://huggingface.co/spaces/Horizon-Labs/multilingual-embedding) ·
45
  [toxicity](https://huggingface.co/spaces/Horizon-Labs/multilingual-toxicity) ·
46
+ [hate speech](https://huggingface.co/spaces/Horizon-Labs/multilingual-hate-speech) ·
47
+ [educational quality](https://huggingface.co/spaces/Horizon-Labs/multilingual-edu-classifier)
48
 
49
  **Data and evaluation**: [prompt-injection eval suite](https://huggingface.co/datasets/Horizon-Labs/prompt-injection-eval-suite) ·
50
  [injection detector leaderboard](https://huggingface.co/spaces/Horizon-Labs/prompt-injection-leaderboard) ·