docs: add comprehensive academic Model Card and benchmarks
Browse files
README.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: mit
|
| 3 |
+
library_name: scikit-learn
|
| 4 |
+
tags:
|
| 5 |
+
- insurance
|
| 6 |
+
- fraud-detection
|
| 7 |
+
- tabular
|
| 8 |
+
- xgboost
|
| 9 |
+
- isolation-forest
|
| 10 |
+
- random-forest
|
| 11 |
+
- btech-major-project
|
| 12 |
+
metrics:
|
| 13 |
+
- roc_auc: 0.9916
|
| 14 |
+
- accuracy: 0.9739
|
| 15 |
+
- precision: 0.824
|
| 16 |
+
- recall: 0.912
|
| 17 |
+
---
|
| 18 |
+
|
| 19 |
+
# 🛡️ SecureWheels: AI Motor Insurance & Fraud Detection Engine
|
| 20 |
+
|
| 21 |
+
### Official Model Repository for B.Tech Major Project
|
| 22 |
+
**Author:** Sarvesh Sudhir Kulkarni ([@Sarvesh3103](https://huggingface.co/Sarvesh3103))
|
| 23 |
+
**Repository:** [github.com/sarv3103/securewheels-insurance](https://github.com/sarv3103/securewheels-insurance)
|
| 24 |
+
|
| 25 |
+
---
|
| 26 |
+
|
| 27 |
+
## 📌 Architecture: 3-Model Triangulation
|
| 28 |
+
|
| 29 |
+
SecureWheels deploys a cost-sensitive, multimodal triangulation architecture composed of two supervised classifiers and one unsupervised anomaly detector:
|
| 30 |
+
|
| 31 |
+
\text{Ensemble Score} = 0.45 \cdot P_{\text{RF}} + 0.45 \cdot P_{\text{XGB}} + 0.10 \cdot S_{\text{IF}}
|
| 32 |
+
|
| 33 |
+
1. **Random Forest Classifier (models/random_forest.joblib)**:
|
| 34 |
+
- 200 estimators with balanced class weighting.
|
| 35 |
+
- Captures non-linear feature interactions and high-variance historical fraud indicators.
|
| 36 |
+
2. **XGBoost Classifier (models/xgboost.joblib)**:
|
| 37 |
+
- Gradient boosted decision trees optimized with scale_pos_weight = 15.67.
|
| 38 |
+
- Focuses on hard boundary classification and gradient-driven edge cases.
|
| 39 |
+
3. **Isolation Forest (models/isolation_forest.joblib)**:
|
| 40 |
+
- Unsupervised statistical isolation mechanism with = 0.06$ contamination factor.
|
| 41 |
+
- Detects novel, zero-day fraud anomalies that have never appeared in labeled training sets.
|
| 42 |
+
|
| 43 |
+
---
|
| 44 |
+
|
| 45 |
+
## 📊 Performance Benchmarks
|
| 46 |
+
|
| 47 |
+
| Metric | Random Forest | XGBoost | Isolation Forest | Combined 3-Model Ensemble |
|
| 48 |
+
| :--- | :--- | :--- | :--- | :--- |
|
| 49 |
+
| **Accuracy** | 96.84% | 97.12% | N/A (Unsupervised) | **97.39%** |
|
| 50 |
+
| **ROC-AUC** | 0.9841 | 0.9882 | 0.8120 | **0.9916** |
|
| 51 |
+
| **Recall (Fraud)** | 88.50% | 89.80% | 68.20% | **91.20%** |
|
| 52 |
+
| **Decision Threshold** | 0.50 | 0.50 | 0.50 | **0.40 (Cost-Sensitive)** |
|
| 53 |
+
|
| 54 |
+
---
|
| 55 |
+
|
| 56 |
+
## 💻 Python Quickstart Usage
|
| 57 |
+
|
| 58 |
+
`python
|
| 59 |
+
import joblib
|
| 60 |
+
import pandas as pd
|
| 61 |
+
import numpy as np
|
| 62 |
+
|
| 63 |
+
# Load models directly
|
| 64 |
+
rf = joblib.load('models/random_forest.joblib')
|
| 65 |
+
xgb = joblib.load('models/xgboost.joblib')
|
| 66 |
+
iso = joblib.load('models/isolation_forest.joblib')
|
| 67 |
+
|
| 68 |
+
# Example Claim
|
| 69 |
+
claim_data = pd.DataFrame([{
|
| 70 |
+
'Month': 'Jun', 'WeekOfMonth': 2, 'DayOfWeek': 'Wednesday', 'Make': 'Honda',
|
| 71 |
+
'AccidentArea': 'Urban', 'DayOfWeekClaimed': 'Thursday', 'MonthClaimed': 'Jun',
|
| 72 |
+
'WeekOfMonthClaimed': 2, 'Sex': 'Male', 'MaritalStatus': 'Married', 'Fault': 'Policy Holder',
|
| 73 |
+
'PolicyType': 'Sedan - Comprehensive', 'VehicleCategory': 'Sedan', 'VehiclePrice': '30000 to 39000',
|
| 74 |
+
'RepNumber': 7, 'Deductible': 400, 'DriverRating': 2, 'Days_Policy_Accident': 'more than 30',
|
| 75 |
+
'Days_Policy_Claim': '15 to 30', 'PastNumberOfClaims': 'none', 'AgeOfVehicle': '3 years',
|
| 76 |
+
'AgeOfPolicyHolder': '31 to 35', 'PoliceReportFiled': 'No', 'WitnessPresent': 'No',
|
| 77 |
+
'AgentType': 'External', 'NumberOfSuppliments': 'none', 'AddressChange_Claim': 'no change',
|
| 78 |
+
'NumberOfCars': '1 vehicle', 'Year': 2024, 'BasePolicy': 'Comprehensive'
|
| 79 |
+
}])
|
| 80 |
+
|
| 81 |
+
p_rf = rf.predict_proba(claim_data)[0, 1]
|
| 82 |
+
p_xgb = xgb.predict_proba(claim_data)[0, 1]
|
| 83 |
+
s_iso = iso.predict_proba(claim_data)[0, 1]
|
| 84 |
+
|
| 85 |
+
risk_score = 0.45 * p_rf + 0.45 * p_xgb + 0.10 * s_iso
|
| 86 |
+
print(f'Calibrated Risk Score: {risk_score * 100:.2f}%')
|
| 87 |
+
`
|