| --- |
| license: cc-by-4.0 |
| tags: |
| - multi-label-classification |
| - text-classification |
| - onnx |
| - web-classification |
| - firefox-ai |
| - preview |
| language: |
| - multilingual |
| datasets: |
| - tshasan/multi-label-web-classification |
| base_model: Alibaba-NLP/gte-modernbert-base |
| pipeline_tag: text-classification |
| library_name: transformers |
| --- |
| |
| # URL-TITLE-classifier-preview |
|
|
| ## Model Overview |
|
|
| This is a **preview version** of a multi-label web classification model fine-tuned from [`Alibaba-NLP/gte-modernbert-base`](https://huggingface.co/Alibaba-NLP/gte-modernbert-base). It classifies websites into multiple categories based on their URLs and titles. |
|
|
| The model supports **11 labels**: |
| `Uncategorized`, `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, and `Travel`. |
|
|
| - **Developed by**: Taimur Hasan |
| - **Model Type**: Multi-label Text Classification |
| - **Status**: Preview (under active development) |
|
|
| ### Architecture |
|
|
| - **Fine-tuning Strategy**: Unfroze the last 4 encoder layers and the pooler |
| - **Problem Type**: Multi-label classification |
| - **Output Labels**: |
| - `News`, `Entertainment`, `Shop`, `Chat`, `Education`, `Government`, `Health`, `Technology`, `Work`, `Travel`, `Uncategorized` |
| - **Input Format**: Concatenated string: |
| `"{url}:{title}"` |
|
|
| --- |
|
|
| ## Evaluation Metrics (Validation Data) |
|
|
| | Metric | Value | |
| |-----------------------|--------| |
| | **Loss** | 0.207 | |
| | **Hamming Loss** | 0.083 | |
| | **Exact Match** | 0.445 | |
| | **Precision (Micro)** | 0.917 | |
| | **Recall (Micro)** | 0.917 | |
| | **F1 Score (Micro)** | 0.917 | |
| | **Precision (Macro)** | 0.795 | |
| | **Recall (Macro)** | 0.598 | |
| | **F1 Score (Macro)** | 0.677 | |
| | **Precision (Weighted)** | 0.798 | |
| | **Recall (Weighted)** | 0.647 | |
| | **F1 Score (Weighted)** | 0.711 | |
| | **ROC AUC (Micro)** | 0.941 | |
| | **ROC AUC (Macro)** | 0.928 | |
| | **PR AUC (Micro)** | 0.815 | |
| | **PR AUC (Macro)** | 0.765 | |
| | **Jaccard (Micro)** | 0.848 | |
| | **Jaccard (Macro)** | 0.520 | |
|
|
| ### Per-Label F1 Scores |
|
|
| | Label | F1 Score | |
| |----------------|----------| |
| | News | 0.605 | |
| | Entertainment | 0.764 | |
| | Shop | 0.704 | |
| | Chat | 0.875 | |
| | Education | 0.763 | |
| | Government | 0.667 | |
| | Health | 0.574 | |
| | Technology | 0.738 | |
| | Work | 0.527 | |
| | Travel | 0.571 | |
| | Uncategorized | 0.657 | |
|
|
| --- |
|
|
| > **Note:** This model is in preview and may not generalize well outside of its training dataset. Feedback and contributions are welcome. |
