amaedaqureshi/sia-churn-model

SIA Retention Engine — Churn Risk Model (v2.0.0)

Calibrated XGBoost classifier trained on the public IBM/Kaggle Telco Customer Churn dataset, mapped to SIA's Subscriber schema. Every input feature is real — see the dataset card for the column-by-column provenance.

Headline metrics (held-out test split, n=1056)

	ROC-AUC	PR-AUC	Brier
This model (calibrated)	0.8476	0.6511	0.1348
Legacy 2-feature model	0.795	—	—
Logistic regression	0.8519	—	—

The legacy row is an identical XGBoost trained on only the two real features the previous schema had (avg_monthly_spend, active_plan), scored on the same rows. The gap is what rebuilding the feature set bought.

Calibration

Probabilities are calibrated with isotonic regression fitted on the validation split. Brier score improved from 0.1495 (raw) to 0.1348 (calibrated), so a reported risk of 0.30 corresponds to roughly a 30% observed churn rate.

Top features (mean |SHAP|, test split)

• tenure_months: 0.5664
• active_plan_Flexi: 0.5278
• active_plan_Premium: 0.3944
• avg_monthly_spend: 0.2664
• lifetime_spend: 0.2546
• spend_per_tenure_month: 0.2333
• addon_security_No: 0.2276
• spend_trend_ratio: 0.1991
• internet_tier_Turbo: 0.1898
• paperless_billing_No: 0.1374

Intended use

Churn risk scoring for SIA's Monitor agent. Pair with backend/economics.py to convert a probability into a contact decision — a flat threshold ignores that a high-value customer is worth contacting at far lower risk than a low-value one.

Files

• model.joblib — trained XGBoost classifier
• calibrator.joblib — isotonic calibrator (apply to predict_proba output)
• feature_columns.json — exact column order/names the model expects
• metrics.json — full evaluation