docs/METRICS.md

Unified Scientific Metric Definition Standard & Evaluation Glossary

Document Reference: CFI-SPEC-METRICS-2026-V1
Applicable Governance Frameworks: Federal Reserve SR 11-7 / OCC 2011-12 (Model Risk Management), EU Artificial Intelligence Act (Regulation (EU) 2024/1689) Article 15 (Accuracy, Robustness and Cybersecurity), Basel Committee on Banking Supervision (BCBS) Working Paper 32.

---

1. Overview & Epistemological Purpose

In enterprise federated machine learning and regulatory model risk validation, evaluation metrics must be mathematically unambiguous, strictly bounded, and suited to the underlying financial data distribution.

Financial crime and Anti-Money Laundering (AML) transaction datasets exhibit extreme class imbalance (typically $\le 0.15$% fraud prevalence). Under extreme imbalance, standard machine learning metrics such as Accuracy or uncalibrated ROC-AUC produce dangerously misleading assessments:

• An uncalibrated trivial classifier predicting the majority class achieves 99.85% accuracy while detecting zero fraud.
• Receiver Operating Characteristic (ROC-AUC) can remain superficially high ($> 0.95$) even when thousands of false alerts overwhelm human investigator queues, because the vast true negative denominator artificially suppresses the False Positive Rate ($\mathrm{FPR} = \frac{\mathrm{FP}}{\mathrm{FP} + \mathrm{TN}}$).

This specification establishes the authoritative mathematical formulations, boundary constraints, operational thresholds, and regulatory mappings for all quantitative metrics reported across CF-Intelligence.

---

2. Core Classification & Detection Performance Metrics

2.1 Precision-Recall Area Under Curve (PR-AUC / Average Precision)

Mathematical Definition

Precision-Recall Area Under the Curve evaluates the trade-off between positive predictive value (Precision) and sensitivity (Recall) across all classification thresholds $\theta \in [0, 1]$:

$$\mathrm{P}\mathrm{R}\text{-}\mathrm{A}\mathrm{U}\mathrm{C}={\int }_0^{1}p(r)dr\approx \sum _{k=1}^K(r_k-r_{k-1})\cdot p(r_k)\backslash mathrm\{PR\backslash text\{-\}AUC\}\; =\; \backslash int\_\{0\}^\{1\}\; p(r)\; \backslash ,\; dr\; \backslash approx\; \backslash sum\_\{k=1\}^\{K\}\; (r\_k\; -\; r\_\{k-1\})\; \backslash cdot\; p(r\_k)$$

where Precision $p(r)$ and Recall $r$ are parameterized by threshold $\theta$:

$$p(\theta )=\frac{\mathrm{T}\mathrm{P}(\theta )}{\mathrm{T}\mathrm{P}(\theta )+\mathrm{F}\mathrm{P}(\theta )},r(\theta )=\frac{\mathrm{T}\mathrm{P}(\theta )}{\mathrm{T}\mathrm{P}(\theta )+\mathrm{F}\mathrm{N}(\theta )}p(\backslash theta)\; =\; \backslash frac\{\backslash mathrm\{TP\}(\backslash theta)\}\{\backslash mathrm\{TP\}(\backslash theta)\; +\; \backslash mathrm\{FP\}(\backslash theta)\},\; \backslash quad\; r(\backslash theta)\; =\; \backslash frac\{\backslash mathrm\{TP\}(\backslash theta)\}\{\backslash mathrm\{TP\}(\backslash theta)\; +\; \backslash mathrm\{FN\}(\backslash theta)\}$$

In finite sample implementations, CF-Intelligence uses the non-interpolated Average Precision ($\mathrm{AP}$) formulation:

$$\mathrm{A}\mathrm{P}=\sum _{n=1}^N(R_n-R_{n-1})\cdot P_n\backslash mathrm\{AP\}\; =\; \backslash sum\_\{n=1\}^\{N\}\; (R\_n\; -\; R\_\{n-1\})\; \backslash cdot\; P\_n$$

where $P_n$ and $R_n$ are the precision and recall at the $n$-th operational threshold.

Operational Interpretation & Target Bounds

• Range: $\mathrm{PR\text{-}AUC} \in [\pi, 1.0]$, where $\pi = \frac{P}{P + N}$ is the baseline fraud prevalence (random guessing baseline).
• Target Performance:
  • $\mathrm{PR\text{-}AUC} \ge 0.75$: High-efficacy production champion candidate.
  • $\mathrm{PR\text{-}AUC} \in [0.40, 0.74]$: Acceptable multi-bank federated detector under non-IID skew ($\alpha \le 0.50$).
  • $\mathrm{PR\text{-}AUC} < 0.20$: Sub-baseline / non-converged local silo.

---

2.2 Receiver Operating Characteristic Area Under Curve (ROC-AUC)

Mathematical Definition

The ROC curve traces the True Positive Rate ($\mathrm{TPR}$) against the False Positive Rate ($\mathrm{FPR}$) across all continuous classification thresholds:

$$\mathrm{R}\mathrm{O}\mathrm{C}\text{-}\mathrm{A}\mathrm{U}\mathrm{C}={\int }_0^1\mathrm{T}\mathrm{P}\mathrm{R}({\mathrm{F}\mathrm{P}\mathrm{R}}^{-1}(t))dt\backslash mathrm\{ROC\backslash text\{-\}AUC\}\; =\; \backslash int\_\{0\}^\{1\}\; \backslash mathrm\{TPR\}(\backslash mathrm\{FPR\}^\{-1\}(t))\; \backslash ,\; dt$$

Probabilistically, $\mathrm{ROC\text{-}AUC}$ equals the Wilcoxon-Mann-Whitney concordance probability that a randomly chosen fraudulent transaction $x^+$ receives a strictly higher predicted risk score than a randomly chosen legitimate transaction $x^-$:

$$\mathrm{R}\mathrm{O}\mathrm{C}\text{-}\mathrm{A}\mathrm{U}\mathrm{C}=\mathbb{P}(\widehat{s}(x^{+})>\widehat{s}(x^{-})\mid y(x^{+})=1,y(x^{-})=0)\backslash mathrm\{ROC\backslash text\{-\}AUC\}\; =\; \backslash mathbb\{P\}\backslash left(\backslash hat\{s\}(x^+)\; >\; \backslash hat\{s\}(x^-)\; \backslash mid\; y(x^+)\; =\; 1,\; y(x^-)\; =\; 0\backslash right)$$

In discrete finite samples:

$$\mathrm{R}\mathrm{O}\mathrm{C}\text{-}\mathrm{A}\mathrm{U}\mathrm{C}=\frac{1}{N^{+}\cdot N^{-}}\sum _{i:y_i=1}\sum _{j:y_j=0}(\mathbb{I}({\widehat{s}}_i>{\widehat{s}}_j)+\frac{1}{2}\mathbb{I}({\widehat{s}}_i={\widehat{s}}_j))\backslash mathrm\{ROC\backslash text\{-\}AUC\}\; =\; \backslash frac\{1\}\{N^+\; \backslash cdot\; N^-\}\; \backslash sum\_\{i:\; y\_i\; =\; 1\}\; \backslash sum\_\{j:\; y\_j\; =\; 0\}\; \backslash left(\; \backslash mathbb\{I\}(\backslash hat\{s\}\_i\; >\; \backslash hat\{s\}\_j)\; +\; \backslash frac\{1\}\{2\}\; \backslash mathbb\{I\}(\backslash hat\{s\}\_i\; =\; \backslash hat\{s\}\_j)\; \backslash right)$$

Operational Interpretation & Target Bounds

• Range: $\mathrm{ROC\text{-}AUC} \in [0.0, 1.0]$ (random baseline: $0.50$).
• Regulatory Caveat (SR 11-7): Must never be reported as the sole validation metric on imbalanced financial data. Must always be accompanied by $\mathrm{PR\text{-}AUC}$ and $\mathrm{Recall@FPR}$.

---

2.3 Recall at Fixed False Positive Rate (Recall@FPR)

Mathematical Definition

In production anti-fraud operations, human compliance investigator capacity is strictly bounded. Models cannot be evaluated at arbitrary probability thresholds (e.g. $\theta = 0.50$). Instead, the operational threshold $\theta^{\ast}$ is calibrated to guarantee that false alarms do not exceed a statutory ceiling $\alpha_{\mathrm{target}}$:

$${\theta }^{\ast }=\inf \{\theta \in [0,1]:\mathrm{F}\mathrm{P}\mathrm{R}(\theta )\le {\alpha }_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}}\}\backslash theta^\{\backslash ast\}\; =\; \backslash inf\; \backslash left\backslash lbrace\; \backslash theta\; \backslash in\; [0,\; 1]\; :\; \backslash mathrm\{FPR\}(\backslash theta)\; \backslash le\; \backslash alpha\_\{\backslash mathrm\{target\}\}\; \backslash right\backslash rbrace$$

where:

$$\mathrm{F}\mathrm{P}\mathrm{R}(\theta )=\frac{\sum _{j:y_j=0}\mathbb{I}({\widehat{s}}_j\ge \theta )}{\sum _j\mathbb{I}(y_j=0)}\backslash mathrm\{FPR\}(\backslash theta)\; =\; \backslash frac\{\backslash sum\_\{j:\; y\_j\; =\; 0\}\; \backslash mathbb\{I\}(\backslash hat\{s\}\_j\; \backslash ge\; \backslash theta)\}\{\backslash sum\_\{j\}\; \backslash mathbb\{I\}(y\_j\; =\; 0)\}$$

$\mathrm{Recall@FPR}$ is the sensitivity achieved at this operational threshold:

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}({\alpha }_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}})=\mathrm{T}\mathrm{P}\mathrm{R}({\theta }^{\ast })=\frac{\sum _{i:y_i=1}\mathbb{I}({\widehat{s}}_i\ge {\theta }^{\ast })}{\sum _i\mathbb{I}(y_i=1)}\backslash mathrm\{Recall\}(\backslash alpha\_\{\backslash mathrm\{target\}\})\; =\; \backslash mathrm\{TPR\}(\backslash theta^\{\backslash ast\})\; =\; \backslash frac\{\backslash sum\_\{i:\; y\_i\; =\; 1\}\; \backslash mathbb\{I\}(\backslash hat\{s\}\_i\; \backslash ge\; \backslash theta^\{\backslash ast\})\}\{\backslash sum\_\{i\}\; \backslash mathbb\{I\}(y\_i\; =\; 1)\}$$

Standard Banking Operating Points

• Recall @ 0.01% FPR ($\alpha = 0.0001$): Ultra-strict tier. Permits at most 1 false alarm per $10{,}000$ legitimate transactions. Target: $\ge 0.50$ (50% fraud caught before friction).
• Recall @ 0.1% FPR ($\alpha = 0.0010$): Standard automated review tier. Permits at most 1 false alarm per $1{,}000$ legitimate transactions. Target: $\ge 0.60$.

---

3. Probability Calibration & Proper Scoring Rules

3.1 Expected Calibration Error (ECE)

Mathematical Definition

Expected Calibration Error quantifies the discrepancy between model predicted probabilities and empirical ground-truth event frequencies. Predictions $\hat{p}_i \in [0, 1]$ are partitioned into $M$ equal-width bins $B_1, B_2, \dots, B_M \subset [0, 1]$ (default $M=10$):

$$B_m=(\frac{m-1}{M},\frac{m}{M}]B\_m\; =\; \backslash left(\; \backslash frac\{m-1\}\{M\},\; \backslash frac\{m\}\{M\}\; \backslash right]$$

$\mathrm{ECE}$ is the weighted average absolute difference between empirical accuracy and average confidence across all bins:

$$\mathrm{E}\mathrm{C}\mathrm{E}=\sum _{m=1}^M\frac{\mathrm{\mid }{B}_m\mathrm{\mid }}{N}\mid \mathrm{a}\mathrm{c}\mathrm{c}(B_m)-\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}(B_m)\mid \backslash mathrm\{ECE\}\; =\; \backslash sum\_\{m=1\}^\{M\}\; \backslash frac\{|B\_m|\}\{N\}\; \backslash left|\; \backslash mathrm\{acc\}(B\_m)\; -\; \backslash mathrm\{conf\}(B\_m)\; \backslash right|$$

where:

$$\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}(B_m)=\frac{1}{\mathrm{\mid }{B}_m\mathrm{\mid }}\sum _{i\in B_m}{\widehat{p}}_i,\mathrm{a}\mathrm{c}\mathrm{c}(B_m)=\frac{1}{\mathrm{\mid }{B}_m\mathrm{\mid }}\sum _{i\in B_m}{y}_i\backslash mathrm\{conf\}(B\_m)\; =\; \backslash frac\{1\}\{|B\_m|\}\; \backslash sum\_\{i\; \backslash in\; B\_m\}\; \backslash hat\{p\}\_i,\; \backslash quad\; \backslash mathrm\{acc\}(B\_m)\; =\; \backslash frac\{1\}\{|B\_m|\}\; \backslash sum\_\{i\; \backslash in\; B\_m\}\; y\_i$$

Operational Interpretation

• Range: $\mathrm{ECE} \in [0.0, 1.0]$.
• Target: $\mathrm{ECE} \le 0.030$ (3.0%).
• Implication: Prevents overconfident deep learning models from distorting downstream Bayesian risk aggregators or automated payment blocking rules.

---

3.2 Brier Score (BS)

Mathematical Definition

The Brier Score is a strictly proper scoring rule quantifying the mean squared error of probabilistic forecasts:

$$\mathrm{B}\mathrm{S}=\frac{1}{N}\sum _{i=1}^N({\widehat{p}}_i-y_i{)}^2\backslash mathrm\{BS\}\; =\; \backslash frac\{1\}\{N\}\; \backslash sum\_\{i=1\}^\{N\}\; (\backslash hat\{p\}\_i\; -\; y\_i)^2$$

Under Murphy's decomposition (Murphy, 1973), the Brier Score decomposes into three orthogonal components:

$$\mathrm{B}\mathrm{S}=\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}-\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{U}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{y}\backslash mathrm\{BS\}\; =\; \backslash mathrm\{Reliability\}\; -\; \backslash mathrm\{Resolution\}\; +\; \backslash mathrm\{Uncertainty\}$$

where:

$$\mathrm{R}\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}=\sum _{m=1}^M\frac{\mathrm{\mid }{B}_m\mathrm{\mid }}{N}{(\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{f}(B_m)-\mathrm{a}\mathrm{c}\mathrm{c}(B_m))}^2\backslash mathrm\{Reliability\}\; =\; \backslash sum\_\{m=1\}^\{M\}\; \backslash frac\{|B\_m|\}\{N\}\; \backslash left(\; \backslash mathrm\{conf\}(B\_m)\; -\; \backslash mathrm\{acc\}(B\_m)\; \backslash right)^2$$

$$\mathrm{R}\mathrm{e}\mathrm{s}\mathrm{o}\mathrm{l}\mathrm{u}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}=\sum _{m=1}^M\frac{\mathrm{\mid }{B}_m\mathrm{\mid }}{N}{(\mathrm{a}\mathrm{c}\mathrm{c}(B_m)-\bar{y})}^2\backslash mathrm\{Resolution\}\; =\; \backslash sum\_\{m=1\}^\{M\}\; \backslash frac\{|B\_m|\}\{N\}\; \backslash left(\; \backslash mathrm\{acc\}(B\_m)\; -\; \backslash bar\{y\}\; \backslash right)^2$$

$$\mathrm{U}\mathrm{n}\mathrm{c}\mathrm{e}\mathrm{r}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{n}\mathrm{t}\mathrm{y}=\bar{y}(1-\bar{y}),\bar{y}=\frac{1}{N}\sum _{i=1}^N{y}_i\backslash mathrm\{Uncertainty\}\; =\; \backslash bar\{y\}(1\; -\; \backslash bar\{y\}),\; \backslash quad\; \backslash bar\{y\}\; =\; \backslash frac\{1\}\{N\}\backslash sum\_\{i=1\}^N\; y\_i$$

Operational Interpretation

• Range: $\mathrm{BS} \in [0.0, 1.0]$ (lower is strictly superior).
• Target: $\mathrm{BS} \le 0.020$ on low-prevalence financial streams.

---

4. Distribution Drift & Population Stability Metrics

4.1 Population Stability Index (PSI)

Mathematical Definition

The Population Stability Index measures the shift between a baseline reference distribution $P$ (e.g. historical baseline or training partition) and an actual production distribution $Q$ partitioned across $B$ bins (default deciles $B=10$):

$$\mathrm{P}\mathrm{S}\mathrm{I}=\sum _{b=1}^B(q_b-p_b)\cdot \ln \left(\frac{q_b+{ϵ}_{\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}}{p_b+{ϵ}_{\mathrm{s}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}}}\right)\backslash mathrm\{PSI\}\; =\; \backslash sum\_\{b=1\}^\{B\}\; (q\_b\; -\; p\_b)\; \backslash cdot\; \backslash ln\backslash left(\; \backslash frac\{q\_b\; +\; \backslash epsilon\_\{\backslash mathrm\{smooth\}\}\}\{p\_b\; +\; \backslash epsilon\_\{\backslash mathrm\{smooth\}\}\}\; \backslash right)$$

where $p_b$ is the proportion of reference samples in bin $b$, $q_b$ is the proportion of actual samples in bin $b$, and $\epsilon_{\mathrm{smooth}} = 10^{-7}$ prevents numerical division by zero.

Regulatory Traffic-Light Action Matrix (Federal Reserve SR 11-7 / OCC)

PSI Value	Drift Classification	System Action & Governance Response
$\mathrm{PSI} < 0.10$	Stable / No Drift	Model operates within validated baseline. No action required.
$0.10 \le \mathrm{PSI} \le 0.25$	Moderate Drift	Warning issued. Retraining queue enters MONITORING_ALERT status.
$\mathrm{PSI} > 0.25$	Significant Drift	Automated canary trigger: platform dispatches retrain task to coordinator.

---

4.2 Jensen-Shannon Divergence (JSD)

Mathematical Definition

The Jensen-Shannon Divergence is a symmetric, smoothed, and bounded version of the Kullback-Leibler ($\mathrm{KL}$) divergence between two probability distributions $P$ and $Q$:

$$\mathrm{J}\mathrm{S}\mathrm{D}(P\parallel Q)=\frac{1}{2}{D}_{\mathrm{K}\mathrm{L}}(P\parallel M)+\frac{1}{2}{D}_{\mathrm{K}\mathrm{L}}(Q\parallel M)\backslash mathrm\{JSD\}(P\; \backslash parallel\; Q)\; =\; \backslash frac\{1\}\{2\}\; D\_\{\backslash mathrm\{KL\}\}(P\; \backslash parallel\; M)\; +\; \backslash frac\{1\}\{2\}\; D\_\{\backslash mathrm\{KL\}\}(Q\; \backslash parallel\; M)$$

where $M = \frac{1}{2}(P + Q)$ is the equal mixture distribution, and $D_{\mathrm{KL}}$ is the discrete Kullback-Leibler divergence:

$$D_{\mathrm{K}\mathrm{L}}(P\parallel M)=\sum _{x}P(x)\ln \left(\frac{P(x)}{M(x)}\right)D\_\{\backslash mathrm\{KL\}\}(P\; \backslash parallel\; M)\; =\; \backslash sum\_\{x\}\; P(x)\; \backslash ln\backslash left(\; \backslash frac\{P(x)\}\{M(x)\}\; \backslash right)$$

Key Mathematical Properties

1. Symmetry: $\mathrm{JSD}(P \parallel Q) = \mathrm{JSD}(Q \parallel P)$.
2. Bounded: When using base-2 logarithm ($\log_2$), $\mathrm{JSD} \in [0.0, 1.0]$. When using natural logarithm ($\ln$), $\mathrm{JSD} \in [0.0, \ln 2 \approx 0.6931]$.
3. Metric Property: The square root $\sqrt{\mathrm{JSD}(P \parallel Q)}$ satisfies the triangle inequality and qualifies as a true mathematical metric (Endres & Schindelin, 2003).

---

5. Financial Cost-Utility & Fairness Metrics

5.1 Financial Cost-Utility Function

Mathematical Definition

To ground statistical scores in bank profit-and-loss (P&L) economics, the platform defines the expected financial loss function $\mathcal{L}_{\mathrm{financial}}$ parameterized by empirical chargeback and review costs:

$${\mathcal{L}}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{a}\mathrm{l}}(\theta )=C_{\mathrm{F}\mathrm{N}}\cdot \mathrm{F}\mathrm{N}(\theta )+C_{\mathrm{F}\mathrm{P}}\cdot \mathrm{F}\mathrm{P}(\theta )+C_{\mathrm{a}\mathrm{d}\mathrm{m}\mathrm{i}\mathrm{n}}\cdot (\mathrm{T}\mathrm{P}(\theta )+\mathrm{F}\mathrm{P}(\theta ))\backslash mathcal\{L\}\_\{\backslash mathrm\{financial\}\}(\backslash theta)\; =\; C\_\{\backslash mathrm\{FN\}\}\; \backslash cdot\; \backslash mathrm\{FN\}(\backslash theta)\; +\; C\_\{\backslash mathrm\{FP\}\}\; \backslash cdot\; \backslash mathrm\{FP\}(\backslash theta)\; +\; C\_\{\backslash mathrm\{admin\}\}\; \backslash cdot\; (\backslash mathrm\{TP\}(\backslash theta)\; +\; \backslash mathrm\{FP\}(\backslash theta))$$

where empirical industry loss parameters are:

• $C_{\mathrm{FN}} = 850\text{ USD}$ (direct fraud chargeback, card scheme penalty, customer dispute settlement).
• $C_{\mathrm{FP}} = 25\text{ USD}$ (investigator triage labor, SMS two-factor step-up challenge, friction cost).
• $C_{\mathrm{admin}} = 5\text{ USD}$ (automated routing, database ingestion, compliance ledger overhead).

The optimal economic operating threshold $\theta^{\ast}_{\mathrm{cost}}$ minimizes total loss:

$${\theta }_{\mathrm{c}\mathrm{o}\mathrm{s}\mathrm{t}}^{\ast }=\arg \underset{\theta \in [0,1]}{\min }{\mathcal{L}}_{\mathrm{f}\mathrm{i}\mathrm{n}\mathrm{a}\mathrm{n}\mathrm{c}\mathrm{i}\mathrm{a}\mathrm{l}}(\theta )\backslash theta^\{\backslash ast\}\_\{\backslash mathrm\{cost\}\}\; =\; \backslash arg\backslash min\_\{\backslash theta\; \backslash in\; [0,\; 1]\}\; \backslash mathcal\{L\}\_\{\backslash mathrm\{financial\}\}(\backslash theta)$$

---

5.2 Algorithmic Fairness Metrics (EEOC Four-Fifths & ECOA Regulation B)

Pursuant to the Equal Credit Opportunity Act (ECOA, 12 CFR Part 1002) and European Non-Discrimination mandates, proxy attributes ($A \in {0, 1}$) are audited for disparate impact:

1. Disparate Impact Ratio (DIR):

   $$\mathrm{DIR} = \frac{\mathbb{P}(\hat{Y}=1 \mid A=0)}{\mathbb{P}(\hat{Y}=1 \mid A=1)}$$

   EEOC 80% Rule Compliance Boundary: $0.80 \le \mathrm{DIR} \le 1.25$.

2. Equal Opportunity Difference (EOD):

   $$\mathrm{EOD} = \mathrm{TPR}{A=0} - \mathrm{TPR}{A=1} = \mathbb{P}(\hat{Y}=1 \mid Y=1, A=0) - \mathbb{P}(\hat{Y}=1 \mid Y=1, A=1)$$

   Target Bound: $|\mathrm{EOD}| \le 0.05$.

3. Demographic Parity Difference (DPD):

   $$\mathrm{DPD} = \mathbb{P}(\hat{Y}=1 \mid A=0) - \mathbb{P}(\hat{Y}=1 \mid A=1)$$

   Target Bound: $|\mathrm{DPD}| \le 0.05$.

---

6. Master Summary of Unified Metric Definitions

Metric Symbol	Full Metric Name	Domain & Purpose	Mathematical Bounds	Production Target	Regulatory Mapping	Implementation Module
PR-AUC	Precision-Recall AUC	Extreme Imbalance Detection	$[\pi, 1.0]$	$\ge 0.75$	SR 11-7 Outcomes Analysis	metrics_service.py
ROC-AUC	Receiver Operating Characteristic	General Discrimination Ranking	$[0.0, 1.0]$	$\ge 0.95$	SR 11-7 Model Discrimination	metrics_service.py
Recall @ 0.1% FPR	Operational Recall	Fixed Investigator Capacity	$[0.0, 1.0]$	$\ge 0.60$	OCC 2011-12 SLA Bounds	metrics_service.py
ECE	Expected Calibration Error	Probability Reliability	$[0.0, 1.0]$	$\le 0.03$	EU AI Act Art. 15 (Accuracy)	run_factorial_ablation.py
BS	Brier Score	Proper Forecast Accuracy	$[0.0, 1.0]$	$\le 0.02$	SR 11-7 Probability Accuracy	run_factorial_ablation.py
PSI	Population Stability Index	Feature & Concept Drift	$[0.0, \infty)$	$< 0.10$	SR 11-7 Ongoing Monitoring	banks.py, drift_service.py
JSD	Jensen-Shannon Divergence	Symmetric Distribution Drift	$[0.0, 1.0]$	$\le 0.15$	BCBS 32 Model Risk	banks.py, drift_service.py
DIR	Disparate Impact Ratio	Algorithmic Non-Discrimination	$[0.0, \infty)$	$[0.80, 1.25]$	ECOA Reg B / EEOC 80% Rule	demographic_audit.py