| --- |
| language: |
| - en |
| license: mit |
| library_name: transformers |
| tags: |
| - cybersecurity |
| - phishing-detection |
| - url-classification |
| - distilbert |
| - network-security |
| datasets: |
| - tranco-top-1m |
| - phishtank |
| - urlhaus |
| metrics: |
| - accuracy |
| - f1 |
| pipeline_tag: text-classification |
| widget: |
| - text: "https://secure-login.appleid-support.com/auth" |
| example_title: "Phishing Threat" |
| - text: "https://www.wikipedia.org/wiki/Computer_security" |
| example_title: "Benign Domain" |
| - text: "http://192.168.1.100/download/update.bin" |
| example_title: "Malicious Distribution" |
| --- |
| |
| # Ph5 URL Classifier: Multi-Class Phishing & Malware Detector (DistilBERT) |
|
|
| <p align="center"> |
| <img src="https://img.shields.io/badge/Model-DistilBERT-blue.svg" alt="Model"> |
| <img src="https://img.shields.io/badge/Accuracy-99.50%25-brightgreen.svg" alt="Accuracy"> |
| <img src="https://img.shields.io/badge/Macro%20F1-99.49%25-success.svg" alt="Macro F1"> |
| <img src="https://img.shields.io/badge/Inference%20Latency-11.2ms-orange.svg" alt="Latency"> |
| <img src="https://img.shields.io/badge/License-MIT-purple.svg" alt="License"> |
| </p> |
|
|
| ## Model Summary |
|
|
| The **Ph5 URL Classifier** is a fine-tuned `distilbert-base-uncased` transformer engineered to detect adversarial phishing and malware distribution URLs with sub-12ms latency. It serves as the primary URL inference branch in the **Ph5 Multimodal Cybersecurity Framework**. |
|
|
| Rather than relying on static blacklist lookups or hand-engineered lexical heuristics, this model learns tokenized sub-word positional representations of fully qualified domain names (FQDNs), obfuscated paths, homoglyphs, and typosquatting patterns. |
|
|
| --- |
|
|
| ## Performance & Evaluation Benchmark |
|
|
| Evaluated on an independent, strictly isolated held-out test split of **22,182 deduplicated URLs** (7,500 Benign, 7,182 Phishing, 7,500 Malware): |
|
|
| | Metric | Measured Score | Standard Deviation (5-Fold CV) | |
| | :--- | :--- | :--- | |
| | **Accuracy** | **99.50%** | ± 0.03% | |
| | **Macro Precision** | **99.50%** | ± 0.02% | |
| | **Macro Recall** | **99.49%** | ± 0.03% | |
| | **Macro F1-Score** | **99.49%** | ± 0.03% | |
| | **False Positive Rate** | **0.51%** | — | |
| | **Mean Inference Time** | **11.2 ms** | Batch Size = 1 (RTX 3050 Laptop GPU) | |
|
|
| ### Baseline Comparison |
|
|
| | Architecture | Accuracy | Macro F1 | Latency | |
| | :--- | :--- | :--- | :--- | |
| | Handcrafted Lexical + Random Forest (100 Trees) | 96.44% | 96.44% | 0.002 ms | |
| | WebPhish Character-CNN | 96.30% | 95.95% | 8.5 ms | |
| | **Ph5 DistilBERT (This Model)** | **99.50%** | **99.49%** | **11.2 ms** | |
|
|
| --- |
|
|
| ## Confusion Matrix (Test Split: N = 22,182) |
|
|
| ```text |
| Predicted |
| Benign Phishing Malware |
| Benign 7462 23 15 |
| Phishing 38 7118 26 |
| Malware 11 21 7468 |
| ``` |
|
|
| --- |
|
|
| ## Intended Use & Integration |
|
|
| ### Python (Transformers & PyTorch) |
|
|
| ```python |
| import torch |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification |
| |
| # Load model and tokenizer from Hugging Face Hub |
| repo_id = "xynaxdev/ph5-url-distilbert" |
| tokenizer = AutoTokenizer.from_pretrained(repo_id) |
| model = AutoModelForSequenceClassification.from_pretrained(repo_id) |
| model.eval() |
| |
| # Sample URL |
| url = "https://secure-login.appleid-support.com/auth" |
| |
| # Tokenize and run forward pass |
| inputs = tokenizer(url, return_tensors="pt", max_length=64, truncation=True) |
| with torch.no_grad(): |
| logits = model(**inputs).logits |
| probabilities = torch.softmax(logits, dim=-1).squeeze().tolist() |
| |
| labels = ["Benign", "Phishing", "Malware"] |
| results = {label: round(prob, 4) for label, prob in zip(labels, probabilities)} |
| |
| print(f"URL: {url}") |
| print(f"Predictions: {results}") |
| # Predicted: {'Benign': 0.0001, 'Phishing': 0.9984, 'Malware': 0.0015} |
| ``` |
|
|
| --- |
|
|
| ## Training Configuration & Data Provenance |
|
|
| - **Pretrained Backbone:** `distilbert-base-uncased` (6 layers, 768 hidden dimensions, 12 attention heads) |
| - **Classification Head:** Linear projection (768 -> 3) with Softmax output |
| - **Training Samples:** 147,876 URLs total (103,514 train / 22,180 val / 22,182 test) |
| - **Benign Source:** Tranco Top-1M Research List (NDSS 2019) |
| - **Phishing Source:** OpenDNS / Cisco PhishTank verified threat feed |
| - **Malware Source:** abuse.ch URLhaus live malware exchange |
| - **Optimizer:** AdamW (beta1=0.9, beta2=0.999, weight decay = 0.01) |
| - **Learning Rate:** 1e-5 with linear learning rate warmup |
| - **Batch Size:** 16 (Trained with PyTorch Automatic Mixed Precision) |
| - **Token Length Limit:** 64 tokens (truncated to prevent trailing dynamic victim tracking parameter leakage) |
|
|
| --- |
|
|
| ## Citation & Framework |
|
|
| ```bibtex |
| @article{devi2026ph5, |
| title={Ph5: A Multimodal Framework for Cross-Modality Evasion-Resilient Phishing Detection in Multi-Vector Environments}, |
| author={Ritu Devi, Deepti Singh and Akash Sah}, |
| journal={Journal of Computer & Mathematical Sciences / JCMM}, |
| year={2026} |
| } |
| ``` |
