| --- |
| license: apache-2.0 |
| language: |
| - ar |
| base_model: oddadmix/50M-2048-Emhotob |
| library_name: transformers |
| tags: [arabic, zero-shot-classification, prompt-routing, router] |
| --- |
| |
| # Nawah-Router-v3 — موجّه عربي صفري |
|
|
| **52M parameters.** Give it a text and **any categories in plain Arabic**; it scores all of them in |
| **one forward pass**. Categories are chosen at inference — no fixed taxonomy. |
|
|
| > **بالعربية:** نموذج عربي يوجّه أي نص إلى فئة من فئات تكتبها أنت بلغة طبيعية، في مسار واحد. |
|
|
| ## Results |
|
|
| Trained on [`oddadmix/arabic-prompt-routing`](https://huggingface.co/datasets/oddadmix/arabic-prompt-routing) |
| (233,720 rows, 12 routing axes). |
|
|
| | eval | v3 | v2 (51K corpus) | random | |
| |---|---:|---:|---:| |
| | unseen category sets | **0.9305** | 0.9199 | 0.2137 | |
| | unseen domains | **0.6976** | 0.6665 | 0.2521 | |
| | **unseen axes** | **0.6130** | *(n/a)* | 0.2358 | |
| | deliberately adjacent categories | 0.9008 | 0.9149 | 0.2109 | |
|
|
| `unseen_axis` is the strongest claim here: `tools` and `retrieval` appear **nowhere** in training, |
| and the model still routes along them at 0.61 against a 0.24 baseline. |
|
|
| ## Usage |
|
|
| ```python |
| from transformers import AutoTokenizer |
| from routing_model import RouterModel, route # ships in this repo |
| |
| M = "oddadmix/Nawah-Router-v3" |
| tok = AutoTokenizer.from_pretrained(M) |
| model = RouterModel.from_pretrained(M) |
| |
| route(model, tok, "كم صار سعر صرف الدولار اليوم؟", |
| ["بحث في الويب", "حاسبة", "تقويم ومواعيد", "لا يحتاج أداة"]) |
| ``` |
|
|
| ## Choosing a checkpoint |
|
|
| No single configuration wins everything, and the trade-off is real: |
|
|
| | epochs | unseen lanes | unseen domains | unseen axes | |
| |---:|---:|---:|---:| |
| | 1 | **0.9327** | 0.6968 | 0.5950 | |
| | **2** (this) | 0.9305 | **0.6976** | 0.6130 | |
| | 3 | 0.9181 | 0.6496 | **0.6376** | |
|
|
| Longer training helps the hardest transfer (unseen axes) and costs the everyday cases. 2 epochs is |
| shipped as the balance. Higher learning rates are simply worse — 6e-4 and 1e-3 both degrade. |
|
|
| ## How the head works |
|
|
| Text and categories share one sequence, **text first**. Each category's span is mean-pooled into |
| its own vector and a **shared** scorer turns each into one logit; the softmax runs over the |
| categories supplied. Because the scorer is shared it reads category *content*, not slot index — |
| which is what makes the label set free text. |
|
|
| A fixed-slot head (`num_labels = max_lanes`) scored **exactly 1/n at every lane count**: its |
| weights were positional, and the corpus randomises category order, so there was nothing to learn. |
|
|
| ## Limitations |
|
|
| Unseen domains (0.70) and unseen axes (0.61) trail unseen category sets (0.93) — it generalises |
| best inside verticals and dimensions it has seen. Confidence is **not calibrated**: clear cases |
| saturate near 1.0, so use the ranking, not the number. Arabic, 1–3 line messages. |
|
|
| 3 epochs at LR 3e-4 cosine, batch 32, bf16, `max_length` 320. |
|
|
