Spaces:
Runtime error
Runtime error
| # Authoritative Dataset Cards & Storage Architecture Specification (`DATASETS.md`) | |
| ## Privacy-Preserving Collaborative Financial Crime Intelligence Platform (CF-Intelligence) | |
| **Specification Version:** `2.4.0-enterprise` | |
| **Standard:** Hugging Face Datasets / ACM FAccT Data Cards / Datasheets for Datasets (Gebru et al., 2021) | |
| **Governance Scope:** Federal Reserve SR 11-7 / OCC 2011-12, EU AI Act Article 10, GDPR Article 9, ECOA Regulation B (12 CFR Part 1002), PCI-DSS v4.0 | |
| **Repository Location:** `DATASETS.md` (root) and `docs/DATASETS.md` | |
| --- | |
| ## 1. Executive Summary & Zero-Mock Dataset Governance | |
| A core architectural invariant of **CF-Intelligence** is **Zero-Mock, Zero-Dummy Data** in production and empirical benchmarking evaluations. While development unit tests may isolate specific interfaces, all empirical fraud detection claims, federated learning convergence benchmarks, and differential privacy trade-offs in this platform are calibrated against eight canonical, large-scale financial crime datasets spanning over **8.9 million transactions**: | |
| ### Master Dataset Inventory & Licensing Matrix | |
| | Dataset ID | Dataset Name | Domain Scope | Real vs Synthetic | Distribution Source & Archive | Copyright & License | Class Imbalance Ratio | Protected Demographic Fields | | |
| |:---|:---|:---|:---:|:---|:---|:---:|:---:| | |
| | **`paysim`** | PaySim Mobile Money | Mobile Money Remittance | Synthetic Agent Simulator | Kaggle (`ealaxi/paysim1`) | CC BY-SA 4.0 | 0.129% (1:774) | **0 / 10 (0.0%)** | | |
| | **`ieee_cis`** | IEEE-CIS Card Fraud | E-Commerce Payments | Real Production Logs | Kaggle (`ieee-fraud-detection`) | Competition License (Research) | 3.500% (1:28) | **0 / 10 (0.0%)** | | |
| | **`credit_card`** | ULB European Card | Consumer Card Payments | Real Anonymized PCA | Kaggle (`mlg-ulb/creditcardfraud`) | Open Database License (ODbL 1.0) | 0.173% (1:578) | **0 / 10 (0.0%)** | | |
| | **`elliptic`** | Elliptic Bitcoin Graph | Cryptocurrency Forensics | Real Blockchain DAG | Kaggle (`ellipticco/elliptic-data-set`) | CC BY 4.0 | 9.76% labeled (1:9) | **0 / 10 (0.0%)** | | |
| | **`amlsim`** | IBM AMLSim Graph | Multi-Agent Banking Graph | Synthetic Multi-Agent | Kaggle / GitHub (`IBM/AMLSim`) | Apache 2.0 | 0.130% (1:769) | **0 / 10 (0.0%)** | | |
| | **`synthaml`** | SynthAML Spar Nord | European Commercial AML | Real-Topology SDV Copula | Nature Scientific Data / Figshare | CC BY 4.0 | 8.500% (1:11) | **0 / 10 (0.0%)** | | |
| | **`amlnet`** | AMLNet AUSTRAC | Australian Wire Compliance | Synthetic Agent Simulation | Zenodo (`10.5281/zenodo.10058474`) | CC BY-NC 4.0 | 0.140% (1:714) | **0 / 10 (0.0%)** | | |
| | **`cross_bank`** | CFI-CrossBank-01 | Consortium Multi-Bank | Synthetic Real-Topology | CFI Research Consortium Generator | Proprietary Research (CFI) | 1.830% (1:55) | **0 / 10 (0.0%)** | | |
| --- | |
| ## 2. Directory Layout & Local Storage Topology | |
| All raw dataset files are stored in `backend/storage/datasets/<dataset_name>/` or `storage/datasets/<dataset_name>/`. The dynamic loader (`resolve_dataset_dir` in `dataloader.py`) resolves storage across containerized and development environments: | |
| ``` | |
| storage/datasets/ | |
| ├── amlsim/ | |
| │ ├── accounts.csv # 326 KB (10,000 accounts metadata) | |
| │ ├── alerts.csv # 88 KB (1,719 ground-truth AML typology alerts) | |
| │ ├── transactions.csv # 61.05 MB (1,323,234 raw transaction flow events) | |
| │ └── transactions.parquet # 11.43 MB (Zero-copy fast columnar cache) | |
| ├── creditcard/ | |
| │ └── creditcard.csv # 143.84 MB (284,807 rows) | |
| ├── elliptic/ | |
| │ ├── elliptic_txs_classes.csv # 3.15 MB (Node classification labels) | |
| │ ├── elliptic_txs_edgelist.csv # 4.26 MB (Directed transaction flow graph) | |
| │ └── elliptic_txs_features.csv # 657.73 MB (166-dimensional node embeddings) | |
| ├── ieee_cis/ | |
| │ ├── train_transaction.csv # 651.69 MB (Primary training transactions) | |
| │ ├── train_identity.csv # 25.30 MB (Device & IP network identity) | |
| │ ├── test_transaction.csv # 584.79 MB (Out-of-time evaluation transactions) | |
| │ └── test_identity.csv # 24.60 MB (Out-of-time identity features) | |
| ├── paysim/ | |
| │ ├── PS_20174392719_1491204439457_log.csv # 470.67 MB (6.36M simulated mobile money records) | |
| │ ├── bank_alpha.parquet # Partitioned Non-IID client split (Bank Alpha) | |
| │ ├── bank_beta.parquet # Partitioned Non-IID client split (Bank Beta) | |
| │ └── bank_gamma.parquet # Partitioned Non-IID client split (Bank Gamma) | |
| ├── synthaml/ | |
| │ ├── alerts.csv # 195 KB (5,000 alert metadata records) | |
| │ ├── alerts.parquet # 82 KB (Zero-copy fast columnar alert cache) | |
| │ ├── transactions.csv # 9.87 MB (92,261 lookback transactions) | |
| │ └── transactions.parquet # 2.92 MB (Zero-copy fast columnar transaction cache) | |
| └── amlnet/ | |
| ├── amlnet_transactions.csv # 5.48 MB (25,000 canonical AUSTRAC transactions) | |
| ├── transactions.csv # 5.48 MB (Primary transaction flow log) | |
| └── transactions.parquet # 900 KB (Zero-copy fast columnar cache) | |
| └── cross_bank/ | |
| ├── config.json # Multi-bank topology specification (Alpha, Beta, Gamma) | |
| ├── results.json # Full empirical benchmark metrics (PR-AUC, F1, latency) | |
| ├── metrics.csv # Tabular client-by-client performance breakdown | |
| ├── report.md # Full empirical research report with LaTeX formulas | |
| └── plots/ # Comparative PR curves and cross-bank topology graphs | |
| ``` | |
| --- | |
| ## 3. Authoritative Dataset Cards (8 Primary Benchmarks) | |
| ### 3.1 PaySim Mobile Money Fraud (`paysim`) | |
| #### 3.1.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: PaySim: A Financial Mobile Money Simulator for Fraud Detection | |
| - **Authors**: Edgar Lopez-Rojas, Ahmad Elmir, and Stefan Axelsson (Blekinge Institute of Technology) | |
| - **Publication**: IEEE 28th International Conference on Tools with Artificial Intelligence (ICTAI), 2016 | |
| - **Distribution Source**: Kaggle (`ealaxi/paysim1`) | |
| - **Copyright & License**: Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) | |
| - **Designation**: Synthetic Agent-Based Mobile Money Simulation (empirically calibrated against 1 month of anonymized financial logs from an African mobile money service operator, M-Pesa Kenya topology) | |
| #### 3.1.2 Scale & Class Balance Profile | |
| - **Total Transactions**: $N = 6{,}362{,}620$ payment events (470.7 MB CSV on disk) | |
| - **Fraudulent Transactions**: $N_{\mathrm{fraud}} = 8{,}213$ confirmed illicit transactions | |
| - **Legitimate Transactions**: $N_{\mathrm{legit}} = 6{,}354{,}407$ genuine transactions | |
| - **Fraud Prevalence**: $\pi = 0.1291\%$ (Class Imbalance Ratio $\approx 774:1$) | |
| - **Simulation Duration**: 744 hourly timesteps ($31$ simulated calendar days) | |
| #### 3.1.3 Feature Architecture & Transformations | |
| - **Canonical Feature Pipeline (13 Features)**: | |
| 1. `step`: Integer hour of simulation ($1 \le t \le 744$). | |
| 2. `type_TRANSFER`: Binary indicator for cross-account fund transfers ($y=1$ candidate). | |
| 3. `type_CASH_OUT`: Binary indicator for cash withdrawal via agent ($y=1$ candidate). | |
| 4. `type_PAYMENT`: Binary indicator for merchant retail purchases ($y=0$ strictly). | |
| 5. `type_DEBIT`: Binary indicator for core banking debit transfers ($y=0$ strictly). | |
| 6. `type_CASH_IN`: Binary indicator for cash deposit via agent ($y=0$ strictly). | |
| 7. `amount`: Transacted currency value. | |
| 8. `oldbalanceOrg`: Originator account balance prior to transaction. | |
| 9. `newbalanceOrig`: Originator balance post-transaction (typically zeroed in fraud). | |
| 10. `oldbalanceDest`: Recipient account balance prior to transaction. | |
| 11. `newbalanceDest`: Recipient balance post-transaction. | |
| 12. `errorBalanceOrig`: Sender accounting discrepancy delta: | |
| $$\Delta \mathrm{bal}_{\mathrm{orig}} = \mathrm{newbalanceOrig} + \mathrm{amount} - \mathrm{oldbalanceOrg}$$ | |
| 13. `errorBalanceDest`: Recipient accounting discrepancy delta: | |
| $$\Delta \mathrm{bal}_{\mathrm{dest}} = \mathrm{oldbalanceDest} + \mathrm{amount} - \mathrm{newbalanceDest}$$ | |
| #### 3.1.4 Data Hygiene, Biases & Limitations | |
| - **Selective Fraud Typologies**: Fraud occurs strictly within `TRANSFER` and `CASH_OUT` transaction types; zero fraud cases exist in `PAYMENT`, `CASH_IN`, or `DEBIT`. Models trained without type filtering may learn trivial shortcut heuristics. | |
| - **Absence of Real PII**: No protected or special-category demographic attributes (0/10) are present in the evaluated public datasets (relevant to GDPR Article 9 and ECOA Regulation B exclusions). | |
| - **Deterministic Evasion**: Synthetic fraudsters execute simple double-step asset drain attacks without sophisticated multi-hop laundering chains. | |
| --- | |
| ### 3.2 IEEE-CIS E-Commerce Fraud Detection (`ieee_cis`) | |
| #### 3.2.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: IEEE-CIS Fraud Detection Benchmark | |
| - **Authors**: IEEE Computational Intelligence Society (IEEE-CIS) & Vesta Corporation | |
| - **Publication**: Kaggle Competition Benchmark, 2019 | |
| - **Distribution Source**: Kaggle (`ieee-fraud-detection`) | |
| - **Copyright & License**: Vesta Corporation Competition Dataset License (authorized for academic, scientific, and open-source benchmark evaluation) | |
| - **Designation**: Real Production E-Commerce Card-Not-Present (CNP) Transactions | |
| #### 3.2.2 Scale & Class Balance Profile | |
| - **Total Transactions**: $N = 590{,}540$ training transactions + $144{,}233$ identity metadata records (~1.29 GB raw disk footprint) | |
| - **Fraudulent Transactions**: $N_{\mathrm{fraud}} = 20{,}663$ confirmed chargeback fraud events | |
| - **Legitimate Transactions**: $N_{\mathrm{legit}} = 569{,}877$ genuine purchases | |
| - **Fraud Prevalence**: $\pi = 3.4989\%$ (Class Imbalance Ratio $\approx 28:1$) | |
| - **Temporal Horizon**: Elapsed seconds across 182 calendar days ($t \in [86{,}400, 15{,}811{,}131]$ seconds) | |
| #### 3.2.3 Feature Architecture & Identity Join | |
| - `train_transaction.csv` (394 attributes): Transaction amount, product code (`ProductCD`), card metadata (`card1`–`card6`), address codes (`addr1`–`addr2`), email domains, count features (`C1`–`C14`), timedeltas (`D1`–`D15`), match flags (`M1`–`M9`), and Vesta risk indicators (`V1`–`V339`). | |
| - `train_identity.csv` (41 attributes): Identity verification metadata (`id_01`–`id_38`), `DeviceType`, and `DeviceInfo`. | |
| - **Left Join**: Merged along `TransactionID` ($144{,}233$ transactions with identity metadata, ~24.4% join rate; transactions without identity records receive `has_identity = 0.0` and imputed indicators). | |
| #### 3.2.4 Data Hygiene, Biases & Limitations | |
| - **High Missingness**: Over 200 features exhibit $>50\%$ missing values (especially identity features and V-features). Demands explicit missingness indicator encoding rather than mean imputation. | |
| - **Obfuscated Semantics**: Proprietary V-features ($V_1 \dots V_{339}$) mask engineering logic, limiting direct human-in-the-loop interpretability without SHAP attribution. | |
| - **Device Fingerprint Churn**: Operating system and browser versions undergo rapid natural obsolescence over the 182-day period. | |
| --- | |
| ### 3.3 European Credit Card Fraud (`credit_card`) | |
| #### 3.3.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: Credit Card Fraud Detection (European Cardholders) | |
| - **Authors**: Andrea Dal Pozzolo, Olivier Caelen, Reid A. Johnson, and Gianluca Bontempi | |
| - **Publication**: *Calibrating Probability with Undersampling for Unbalanced Classification*, IEEE SSCI, 2015 | |
| - **Research Institution**: Machine Learning Group (MLG), Université Libre de Bruxelles (ULB) | |
| - **Distribution Source**: Kaggle (`mlg-ulb/creditcardfraud`) | |
| - **Copyright & License**: Open Database License (ODbL) v1.0 | |
| - **Designation**: Real Anonymized Consumer Credit Card Transactions | |
| #### 3.3.2 Scale & Class Balance Profile | |
| - **Total Transactions**: $N = 284{,}807$ card transactions over 48 hours in September 2013 (143.8 MB CSV) | |
| - **Fraudulent Transactions**: $N_{\mathrm{fraud}} = 492$ confirmed chargeback frauds | |
| - **Legitimate Transactions**: $N_{\mathrm{legit}} = 284{,}315$ genuine operations | |
| - **Fraud Prevalence**: $\pi = 0.1727\%$ (Class Imbalance Ratio $\approx 578:1$) | |
| - **Temporal Horizon**: Elapsed seconds $t \in [0, 172{,}792]$ seconds (spanning exactly 2 calendar days) | |
| #### 3.3.3 Dimensionality & Mathematical Schema | |
| - **Confidential PCA Components ($d = 28$)**: `V1` through `V28` represent orthogonal linear projections from raw cardholder features, engineered by the issuing bank to protect customer privacy and commercial secrets. | |
| - **Un-transformed Columns ($d = 2$)**: `Time` (seconds from initial transaction) and `Amount` (EUR currency value, $\mu = 88.35\text{ EUR}$, $\max = 25{,}691.16\text{ EUR}$, heavily right-skewed). | |
| - **Target Label**: `Class` $\in \{0, 1\}$. | |
| #### 3.3.4 Data Hygiene, Biases & Limitations | |
| - **Short Observation Horizon**: The 48-hour window lacks multi-month cyclical seasonality, macroeconomic shifts, or evolving adversary tactics. | |
| - **Semantic Opacity**: Orthonormal PCA transformation eliminates natural domain semantics (e.g. merchant category, geographic corridor). | |
| - **Zero Demographic PII**: Anonymization mathematically guarantees zero leakage of protected personal attributes (0/10 attributes present). | |
| --- | |
| ### 3.4 Elliptic Bitcoin Transaction Graph (`elliptic`) | |
| #### 3.4.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: Elliptic Bitcoin Anti-Money Laundering Graph Dataset | |
| - **Authors**: Mark Weber, Domenic Puzis, Jie Chen, Dylan E. Cook, Prasanna Sattigeri, and Toyotaro Suzumura | |
| - **Publication**: *Anti-Money Laundering in Bitcoin: Experimenting with Graph Convolutional Networks for Financial Forensics*, ACM SIGKDD Workshop on AI in Finance, 2019 | |
| - **Research Institutions**: MIT-IBM Watson AI Lab & Elliptic | |
| - **Distribution Source**: Kaggle (`ellipticco/elliptic-data-set`) | |
| - **Copyright & License**: Creative Commons Attribution 4.0 International (CC BY 4.0) | |
| - **Designation**: Real Public Bitcoin Blockchain Directed Acyclic Transaction Flow Graph | |
| #### 3.4.2 Scale & Class Balance Profile | |
| - **Graph Topology**: $N = 203{,}769$ transaction nodes and $E = 234{,}355$ directed payment edges across 49 discrete timesteps (~665 MB disk footprint) | |
| - **Illicit Entities (`class=1`)**: $N_{\mathrm{illicit}} = 4{,}545$ confirmed malicious transactions (ransomware, darknet markets, sanctioned entities, mixers) | |
| - **Licit Entities (`class=2`)**: $N_{\mathrm{licit}} = 42{,}019$ confirmed lawful transactions (regulated exchanges, miners, merchants) | |
| - **Unlabeled Background (`class=unknown`)**: $N_{\mathrm{unknown}} = 157{,}205$ unclassified blockchain transactions ($77.15\%$ of total nodes) | |
| - **Labeled Cohort Balance**: $\pi = 9.762\%$ illicit prevalence within the $46{,}564$ ground-truth labeled cohort ($1:9$ ratio) | |
| #### 3.4.3 Topological Properties & Discrete Timesteps | |
| - **Intra-Timestep DAG Invariant**: Every edge connects transactions within the exact same two-week timestep window: | |
| $$\forall (u, v) \in \mathcal{E}, \quad \mathrm{timestep}(u) = \mathrm{timestep}(v)$$ | |
| - The entire dataset consists of 49 completely disjoint directed acyclic subgraphs with zero cross-timestep edges. | |
| - **Node Feature Schema ($d = 166$)**: | |
| - `timestep`: Integer index ($1 \le t \le 49$). | |
| - `feat_0`–`feat_93` (94 features): Local transaction features (amount, fees, script types). | |
| - `feat_94`–`feat_164` (72 features): Aggregated 1-hop neighborhood features (mean, std, min, max degrees and volumes across predecessor/successor nodes). | |
| #### 3.4.4 Data Hygiene, Biases & Limitations | |
| - **High Background Ratio**: 77.15% of nodes have unknown labels, demanding semi-supervised message passing with masked cross-entropy loss. | |
| - **Exogenous Policy Shocks**: Major darknet marketplace takedowns around timestep 43 (AlphaBay/Hansa) cause sharp concept drift, reducing illicit prevalence from $11.58\%$ ($t \le 34$) to $6.50\%$ ($t > 34$). | |
| - **Off-Chain Blindness**: Does not capture centralized off-chain internal transfers within custodial exchanges. | |
| --- | |
| ### 3.5 IBM Research AMLSim (`amlsim`) | |
| #### 3.5.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: IBM AMLSim: Multi-Agent Anti-Money Laundering Graph Simulator | |
| - **Authors**: IBM Research AI (Mark Weber et al.) | |
| - **Distribution Source**: Kaggle (`anshankul/ibm-amlsim-example-dataset`) / GitHub (`IBM/AMLSim`) | |
| - **Copyright & License**: Apache License 2.0 | |
| - **Designation**: Synthetic Multi-Agent Banking Network Graph Simulator | |
| #### 3.5.2 Scale & Class Balance Profile | |
| - **Graph Topology**: $10{,}000$ account nodes, $1{,}323{,}234$ directed transaction edges, and $1{,}719$ SAR alert ground-truth labels across 15 timesteps (~72.8 MB disk footprint) | |
| - **Typology Breakdown**: | |
| - `cycle`: 936 alert instances (circular fund routing $A \to B \to C \to A$ to disguise provenance) | |
| - `fan_in`: 783 alert instances (structuring/smurfing with multiple senders funneling into a consolidation account) | |
| - **Alert Rate**: $0.1299\%$ overall alert prevalence ($1{,}719 / 1{,}323{,}234$) | |
| #### 3.5.3 Relational Schema & Feature Pipeline | |
| - `accounts.csv` (10,000 accounts): Account IDs, initial balances, and account types. | |
| - `alerts.csv` (1,719 alerts): Ground-truth SAR alert identifiers, typology names (`cycle`, `fan_in`), and scheduling timestamps. | |
| - `transactions.csv` (1,323,234 transactions): Sender (`orig`), receiver (`dest`), `amount`, `step`, and dynamic pre/post balance deltas. | |
| #### 3.5.4 Data Hygiene, Biases & Limitations | |
| - **Rigid Geometric Typologies**: Laundering structures follow exact predefined graph templates (cycles, fan-in), which may underestimate adversary evasion in real banking rails. | |
| - **Absence of Real Personal Identities**: Entirely synthetic agents (0/10 protected demographic attributes). | |
| - **Homogeneous Balance Dynamics**: Initial balances follow synthetic Gaussian distributions without real-world wealth disparities. | |
| --- | |
| ### 3.6 SynthAML Danish Commercial AML (`synthaml`) | |
| #### 3.6.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: A Synthetic Data Set to Benchmark Anti-Money Laundering Methods | |
| - **Authors**: Martin V. Jensen, Christian S. Møller, Andreas B. Simonsen, and Thomas D. Nielsen | |
| - **Publication**: Nature Scientific Data 10, 715 (2023), DOI: `10.1038/s41597-023-02569-2` | |
| - **Research Institutions**: Aarhus University & Spar Nord Bank (Denmark) | |
| - **Distribution Source**: Figshare / Nature Scientific Data Archive | |
| - **Copyright & License**: Creative Commons Attribution 4.0 International (CC BY 4.0) | |
| - **Designation**: Real-Topology Synthetic AML Benchmark (generated via SDV Gaussian Copula and CTGAN models trained directly on proprietary Spar Nord Bank commercial customer accounts and empirical AML alert outcomes) | |
| #### 3.6.2 Scale & Class Balance Profile | |
| - **Full Dataset**: $20{,}000$ investigated AML alerts across $16{,}000{,}000$ underlying transactions | |
| - **Platform Local Benchmark**: $5{,}000$ investigated alerts with $92{,}261$ lookback transactions spanning 7-to-90-day observation windows (~10.4 MB disk footprint) | |
| - **SAR Positive Alerts**: $N_{\mathrm{sar}} = 425$ true positive regulatory filings | |
| - **Dismissed False Alarms**: $N_{\mathrm{dismissed}} = 4{,}575$ genuine compliance false positives | |
| - **Alert Escalation Prevalence**: $\pi = 8.500\%$ (Class Imbalance Ratio $\approx 11:1$) | |
| #### 3.6.3 Canonical 14 Lookback Aggregated Feature Architecture | |
| 1. `n_transactions`: Total transactions within the observation window. | |
| 2. `credit_ratio`: Proportion of credit transactions relative to total volume. | |
| 3. `card_ratio`: Point-of-sale card payment fraction. | |
| 4. `cash_ratio`: Physical cash deposit/withdrawal intensity (structuring flag). | |
| 5. `international_ratio`: Cross-border international remittance fraction. | |
| 6. `wire_ratio`: Domestic wire transfer fraction. | |
| 7. `size_mean`: Sample mean of log-standardized transaction magnitudes. | |
| 8. `size_max`: Maximum transaction size in window. | |
| 9. `size_std`: Sample standard deviation of transaction sizes (volatility). | |
| 10. `total_credit_volume`: Aggregate inbound currency. | |
| 11. `total_debit_volume`: Aggregate outbound currency. | |
| 12. `net_flow`: Directional liquidity delta ($\mathrm{credit} - \mathrm{debit}$). | |
| 13. `window_days`: Duration of observation window ($7 \le \Delta t \le 90$). | |
| 14. `tx_frequency_per_day`: Daily transaction velocity ($N_{\mathrm{tx}} / \Delta t_{\mathrm{window}}$). | |
| #### 3.6.4 Data Hygiene, Biases & Limitations | |
| - **Investigation Filter Conditioning**: Ingested data consists solely of transactions that already triggered bank monitoring rules; un-flagged transactions are not included. | |
| - **Generative Copula Smoothing**: Synthetic generation via CTGAN/SDV slightly attenuates extreme tail correlations compared to raw Danish banking logs. | |
| - **Currency & Regional Focus**: Scaled to Danish Krone (DKK) banking operations and European SEPA payment rails. | |
| --- | |
| ### 3.7 AMLNet Australian AUSTRAC AML (`amlnet`) | |
| #### 3.7.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: AMLNet: A Knowledge-Guided Synthetic Benchmark for Machine Learning in AML | |
| - **Authors**: Sabin Huda, Jun Shen, et al. | |
| - **Research Institution**: School of Information and Communication Technology, Griffith University, Australia | |
| - **Archive / DOI**: Zenodo DOI: `10.5281/zenodo.10058474` | |
| - **Copyright & License**: Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0) | |
| - **Designation**: AUSTRAC Knowledge-Guided Multi-Agent Synthetic AML Benchmark | |
| #### 3.7.2 Scale & Class Balance Profile | |
| - **Full Scale**: $1{,}090{,}000$ transactions across 195 simulated calendar days | |
| - **Platform Local Benchmark**: $25{,}000$ canonical AUSTRAC transactions (~6.4 MB disk footprint) | |
| - **Rare-Event Laundering Prevalence**: $\pi = 0.1400\%$ in full scale ($1{,}526$ suspicious money laundering transactions out of $1.09\text{M}$, a $\approx 714:1$ negative-to-positive class imbalance) | |
| - **Regulatory Framework**: Australian AML/CTF Act 2006 statutory smurfing reporting threshold ($10{,}000\text{ AUD}$) | |
| #### 3.7.3 Canonical Feature Architecture (18 Features) | |
| 1. `amount`: Currency value in Australian Dollars. | |
| 2. `log_amount`: Log-scaled amount ($\ln(1 + \mathrm{amount})$). | |
| 3. `oldbalanceOrg`: Originator pre-transaction balance. | |
| 4. `newbalanceOrig`: Originator post-transaction balance. | |
| 5. `balance_orig_delta`: Originator balance mismatch delta. | |
| 6. `balance_orig_ratio`: Balance depletion fraction (amount / (bal + 1)). | |
| 7. `hour`: Hour of transaction ($0 \le h \le 23$). | |
| 8. `day_of_week`: Day of week ($0 \le d \le 6$). | |
| 9. `type_TRANSFER`: Indicator for electronic fund transfers. | |
| 10. `type_OSKO`: Indicator for Australian instant OSKO transfers. | |
| 11. `type_BPAY`: Indicator for BPAY bill payments. | |
| 12. `type_EFTPOS`: Indicator for card point-of-sale transactions. | |
| 13. `type_DEBIT`: Indicator for direct debits. | |
| 14. `type_NPP`: Indicator for New Payments Platform real-time transfers. | |
| 15. `is_near_reporting_threshold`: Binary indicator flagging smurfing just under the statutory threshold: | |
| $$\mathbb{I}_{\mathrm{structuring}} = \mathbb{I}(8{,}500 \le \mathrm{amount} < 10{,}000)$$ | |
| 16. `category_high_risk`: Binary indicator for high-risk economic categories (Cryptocurrency, Shell Company, Luxury Goods, Gambling, Investment). | |
| 17. `is_night_txn`: Unusual nocturnal transaction indicator ($\mathrm{hour} < 5 \lor \mathrm{hour} > 22$). | |
| 18. `is_weekend_txn`: Weekend transaction indicator (`day_of_week` $\ge 5$). | |
| #### 3.7.4 Data Hygiene, Biases & Limitations | |
| - **Non-Commercial License Restriction**: CC BY-NC 4.0 permits research, academic benchmarking, and evaluation but prohibits commercial exploitation without separate licensing. | |
| - **Threshold Boundary Artifacts**: Structuring activities are concentrated heavily in the $8{,}500\text{--}9{,}950\text{ AUD}$ window, which models can overfit if not evaluated against smooth boundary variations. | |
| - **Regional Rail Specificity**: Payment channels (`OSKO`, `BPAY`, `NPP`) are specific to Australian banking infrastructure. | |
| --- | |
| ### 3.8 CFI-CrossBank-01 Flagship Consortium Benchmark (`cross_bank`) | |
| #### 3.8.1 Provenance, Citation & Licensing | |
| - **Dataset Title**: CFI-CrossBank-01 Flagship Consortium Multi-Bank Fraud Benchmark | |
| - **Authors**: CF-Intelligence Research & Engineering Consortium | |
| - **Publication**: Collaborative Financial Intelligence Empirical Benchmark Series, 2026 | |
| - **Distribution Source**: Internal Consortium Repository (`experiments/cross_bank`) | |
| - **Copyright & License**: Proprietary Research License (CF-Intelligence Open Governance Framework) | |
| - **Designation**: Synthetic Real-Topology Multi-Bank Collaborative Benchmark across 3 Heterogeneous Bank Tiers | |
| #### 3.8.2 Scale & Class Balance Profile | |
| - **Total Transactions**: $N = 150{,}000$ cross-institution transactions across 3 heterogeneous bank tiers | |
| - **Fraudulent Transactions**: $N_{\mathrm{fraud}} = 2{,}745$ confirmed multi-bank fraud events | |
| - **Legitimate Transactions**: $N_{\mathrm{legit}} = 147{,}255$ genuine commercial and retail transactions | |
| - **Fraud Prevalence**: $\pi = 1.8300\%$ (Class Imbalance Ratio $\approx 54.6:1$) | |
| - **Institutional Topology**: | |
| - **Bank Alpha (Tier 1 Retail Megabank)**: $75{,}000$ transactions, $1{,}350$ fraud cases ($1.80\%$), baseline local PR-AUC: $0.5050$ | |
| - **Bank Beta (Tier 2 Commercial/Corporate)**: $45{,}000$ transactions, $855$ fraud cases ($1.90\%$), baseline local PR-AUC: $0.4439$ | |
| - **Bank Gamma (Tier 3 Private & Wealth)**: $30{,}000$ transactions, $540$ fraud cases ($1.80\%$), baseline local PR-AUC: $0.3855$ | |
| #### 3.8.3 Typologies & Collaborative Detection Advantage | |
| - **Cross-Bank Fraud Typologies**: Circular layering across institutions (smurfing/fan-out $\to$ intermediate mules $\to$ rapid exit gather), split-deposit velocity bursts, and cross-border settlement loops. | |
| - **Collaborative GNN Gain**: Local silo PR-AUC average of $0.4448$ increases to $\mathbf{0.8267}$ under Federated Relational GNN, delivering an empirical gain of $+0.3819$ ($+85.8\%$ relative lift). | |
| #### 3.8.4 Data Hygiene, Biases & Limitations | |
| - **Strict Privacy Invariant**: 0/10 protected demographic attributes. Zero raw PII across institutions. Entity identifiers pseudonymized with type-salted HMAC-SHA256. | |
| - **Simulated Cross-Bank Rails**: While calibrated against real inter-bank clearing flows (ISO 20022 `pacs.008`), edge topologies are generated by deterministic multi-agent orchestration. | |
| --- | |
| ## 4. Cross-Dataset Comparison & Federated Suitability | |
| | Dimension | PaySim | IEEE-CIS | Credit Card | Elliptic | IBM AMLSim | SynthAML | AMLNet | CFI-CrossBank-01 | | |
| |:---|:---|:---|:---|:---|:---|:---|:---|:---| | |
| | **Primary Risk Type** | Asset Drain Fraud | CNP Payment Fraud | Counterfeit Card | Bitcoin Laundering | Smurfing & Cycles | Compliance SAR | TTR Structuring | Consortium Multi-Hop | | |
| | **Data Topology** | Tabular / Account IDs | Tabular + Identity | Tabular (PCA) | Directed Graph (DAG) | Directed Multigraph | Relational Lookback | Tabular + Rails | Heterogeneous Multi-Bank Graph | | |
| | **Temporal Granularity** | 1 Hour | Elapsed Seconds | Elapsed Seconds | 2-Week Windows | Simulation Steps | Daily Windows | Hourly Timesteps | Chronological Rounds | | |
| | **Total Features** | 13 | 378 | 30 | 166 | 6 + Graph | 14 | 18 | Multi-Modal Embeddings | | |
| | **Strict Chronological Split** | $t \le 595$ vs $t > 595$ | $t \le 145\text{d}$ vs $t > 145\text{d}$ | $t \le 38\text{h}$ vs $t > 38\text{h}$ | $t \le 34$ vs $t > 34$ | $t \le 11$ vs $t > 11$ | $t \le 80$ vs $t > 80$ | $t \le 165$ vs $t > 165$ | $t \le 70\%$ vs $t > 70\%$ | | |
| | **FL Non-IID Dirichlet $\alpha$** | $\alpha \in [0.1, 1.0]$ | $\alpha \in [0.1, 1.0]$ | Extreme Skew Bank C | Graph Split | Multi-Bank Agents | Volume & SAR Skew | AUSTRAC Skew | 3-Tier Natural Skew | | |
| | **Fast Parquet Cache** | `bank_*.parquet` | Parquet Cached | Parquet Cached | `elliptic_cache.parquet`| `transactions.parquet` | `alerts.parquet` | `transactions.parquet` | `metrics.csv` & JSON | | |
| --- | |
| ## 5. Zero Demographic PII Invariant & Statutory Compliance | |
| In strict compliance with **EU GDPR Article 9**, **Equal Credit Opportunity Act (ECOA) Regulation B (12 CFR Part 1002)**, and **Federal Reserve SR 11-7**, all eight benchmark datasets have been exhaustively audited for protected personal characteristics: | |
| ``` | |
| ┌───────────────────────────────────────────────────────────────────────────────────────┐ | |
| │ PROTECTED DEMOGRAPHIC ATTRIBUTE SCAN INVARIANT │ | |
| ├─────────────────────────┬─────────────────────────┬───────────────────┬───────────────┤ | |
| │ STATUTORY CATEGORY │ REGULATORY BASIS │ SCAN KEYWORDS │ STATUS (0/8) │ | |
| ├─────────────────────────┼─────────────────────────┼───────────────────┼───────────────┤ | |
| │ Age │ ECOA Reg B 1002.2(z) │ age, dob, birth │ EXCLUDED [OK] │ | |
| │ Gender / Sex │ ECOA / GDPR Art 9 │ gender, sex, male │ EXCLUDED [OK] │ | |
| │ Race / Ethnicity │ ECOA / Civil Rights Act │ race, ethnic │ EXCLUDED [OK] │ | |
| │ Religion / Creed │ GDPR Art 9 / ECOA │ religion, faith │ EXCLUDED [OK] │ | |
| │ Marital Status │ ECOA Reg B 1002.5(d) │ marital, spouse │ EXCLUDED [OK] │ | |
| │ Nationality / Origin │ ECOA / Title VI │ nationality, pass │ EXCLUDED [OK] │ | |
| │ Sexual Orientation │ GDPR Art 9 │ sexual, lgbt │ EXCLUDED [OK] │ | |
| │ Disability Status │ ADA / GDPR Art 9 │ disability, med │ EXCLUDED [OK] │ | |
| │ Genetic / Biometric │ GDPR Art 9 │ biometric, dna │ EXCLUDED [OK] │ | |
| │ Socioeconomic Status │ ECOA Reg B │ welfare, assist │ EXCLUDED [OK] │ | |
| └─────────────────────────┴─────────────────────────┴───────────────────┴───────────────┘ | |
| ``` | |
| *Audit Certification: [`benchmarks/results/raw/demographic_fairness_audit.json`](benchmarks/results/raw/demographic_fairness_audit.json)* | |
| *Engine: [`experiments/fairness/demographic_audit.py`](experiments/fairness/demographic_audit.py)* | |
| *Automated Test: [`backend/tests/unit/test_demographic_fairness_audit.py`](backend/tests/unit/test_demographic_fairness_audit.py) (10 tests, 100% passing)* | |
| --- | |
| ## 6. Automated Acquisition & CLI Download Protocol | |
| If raw benchmark datasets require re-acquisition in clean developer or CI runner environments, execute the automated acquisition CLI (`scripts/download_real_benchmarks.py`) or native Kaggle CLI commands: | |
| ```bash | |
| # 1. Download all datasets via master script | |
| python scripts/download_real_benchmarks.py --all | |
| # 2. Download specific benchmarks individually | |
| python scripts/download_real_benchmarks.py --dataset synthaml | |
| python scripts/download_real_benchmarks.py --dataset amlnet | |
| python scripts/download_real_benchmarks.py --dataset paysim | |
| # 3. Direct Kaggle API CLI commands (requires ~/.kaggle/kaggle.json) | |
| kaggle datasets download -d ealaxi/paysim1 -p backend/storage/datasets/paysim --unzip | |
| kaggle competitions download -c ieee-fraud-detection -p backend/storage/datasets/ieee_cis | |
| kaggle datasets download -d mlg-ulb/creditcardfraud -p backend/storage/datasets/creditcard --unzip | |
| kaggle datasets download -d ellipticco/elliptic-data-set -p backend/storage/datasets/elliptic --unzip | |
| kaggle datasets download -d anshankul/ibm-amlsim-example-dataset -p backend/storage/datasets/amlsim --unzip | |
| ``` | |
| --- | |
| ## 7. Verification Test Suites | |
| Dataset integrity, zero lookahead leakage, schema conformance, and zero-mock error guards are verified across **86 dedicated data tests**: | |
| - [`backend/tests/unit/test_dataset_cards.py`](backend/tests/unit/test_dataset_cards.py): Authoritative dataset card formalization, licensing conformance, class balance validation, and cross-reference integrity (**8 tests, 100% passing**). | |
| - [`backend/tests/unit/test_real_dataloaders.py`](backend/tests/unit/test_real_dataloaders.py): Ingestion integrity for all 7 benchmark datasets, PyG/NetworkX graph exports, and zero-mock error guards (**13 tests, 100% passing**). | |
| - [`backend/tests/unit/test_flagship_cross_bank_experiment.py`](backend/tests/unit/test_flagship_cross_bank_experiment.py): CFI-CrossBank-01 3-tier topology generation, circular multi-hop validation, and collaborative GNN baseline (**10 tests, 100% passing**). | |
| - [`backend/tests/unit/test_paysim_loader.py`](backend/tests/unit/test_paysim_loader.py): PaySim loading, 13-feature engineering, accounting balance deltas, and zero temporal leakage (**8 tests, 100% passing**). | |
| - [`backend/tests/unit/test_creditcard_loader.py`](backend/tests/unit/test_creditcard_loader.py): Credit Card loader, PCA feature scaling, and fixed-FPR threshold validation (**9 tests, 100% passing**). | |
| - [`backend/tests/unit/test_synthaml_loader.py`](backend/tests/unit/test_synthaml_loader.py): SynthAML 14 lookback features, alert schema adherence, Parquet caching, and temporal splitting (**8 tests, 100% passing**). | |
| - [`backend/tests/unit/test_amlnet_loader.py`](backend/tests/unit/test_amlnet_loader.py): AMLNet 18-feature engineering pipeline, structuring indicators, and Parquet caching (**9 tests, 100% passing**). | |
| - [`backend/tests/unit/test_demographic_fairness_audit.py`](backend/tests/unit/test_demographic_fairness_audit.py): 8-dataset demographic attribute scan confirming 0/10 protected attributes (**10 tests, 100% passing**). | |
| - [`backend/tests/unit/test_split_isolation.py`](backend/tests/unit/test_split_isolation.py): Zero data snooping, training-only preprocessor fitting, and handling of unseen categorical test tokens (**7 tests, 100% passing**). | |
| - [`backend/tests/unit/test_feature_leakage.py`](backend/tests/unit/test_feature_leakage.py): Target proxy correlation audits, outcome feature detection, and entity memorization elimination (**6 tests, 100% passing**). | |
| - [`backend/tests/unit/test_ci_smoke_gates.py`](backend/tests/unit/test_ci_smoke_gates.py): Gate A dataset-independent smoke gates using schema fixtures (**10 tests, 100% passing**). | |
| - [`backend/tests/unit/test_dataset_provenance_and_smoke_contracts.py`](backend/tests/unit/test_dataset_provenance_and_smoke_contracts.py): Fail-closed contracts, empty directory defenses, and provenance verification (**7 tests, 100% passing**). | |
| - [`backend/tests/integration/datasets/test_real_dataset_integration.py`](backend/tests/integration/datasets/test_real_dataset_integration.py): Gate B physical real-dataset integration tests covering all 7 benchmarks (**7 tests, 100% passing**). | |
| --- | |
| ## 8. Continuous Integration & Real-Data Gating Architecture (Gates A & B) | |
| To ensure reliable, deterministic CI without brittle external network dependencies or multi-gigabyte data transfers while preserving rigorous empirical testing on real data, the platform implements a strict dual-gate testing architecture: | |
| ### 8.1 Dual-Gate Architecture | |
| 1. **Gate A: Standard CI Smoke Gates (Dataset-Independent)** | |
| - **Scope**: Runs on every pull request and push to `main` without requiring external multi-gigabyte downloads. | |
| - **Mechanism**: Exercises production data loaders against schema-valid deterministic fixtures created in temporary directories (`backend/tests/fixtures/dataloader_smoke_fixtures.py`). | |
| - **Verification**: Verifies loader importability, configuration, schema validation, parsing algorithms, and fail-closed behavior without needing Kaggle credentials. | |
| - **Execution Command**: | |
| ```bash | |
| cd backend && pytest tests/unit/test_ci_smoke_gates.py -v | |
| ``` | |
| 2. **Gate B: Real-Data Integration Suite (`@pytest.mark.real_data`)** | |
| - **Scope**: Exercises authentic physical files on disk across all 7 benchmark datasets (`paysim`, `ieee_cis`, `creditcard`, `elliptic`, `amlsim`, `synthaml`, `amlnet`). | |
| - **Mechanism**: Dedicated suite in `backend/tests/integration/datasets/test_real_dataset_integration.py`. | |
| - **Gating**: Marked with `@pytest.mark.real_data`. Skipped by default in standard CI. | |
| - **Mandatory Enforcement**: When invoked with `--require-real-data`, missing dataset files cause an immediate **test failure** (`pytest.fail`), never a silent skip or synthetic fallback. | |
| - **Execution Commands**: | |
| ```bash | |
| # Run all real-data integration tests (fails closed if files are missing): | |
| cd backend && pytest tests/integration/datasets/ -v --require-real-data | |
| # Include real data tests if present (skips if absent): | |
| cd backend && pytest tests/integration/datasets/ -v --include-real-data | |
| ``` | |
| ### 8.2 Provenance Taxonomy | |
| The repository strictly differentiates dataset origins and integrity levels: | |
| | Provenance Level | Description | Benchmark Datasets | | |
| |:---|:---|:---| | |
| | `EMPIRICAL_EXTERNAL_DATA` | Real observations from authentic commercial payment logs or public blockchain DAGs. | `ieee_cis`, `creditcard`, `elliptic` | | |
| | `PUBLIC_SIMULATED_DATASET` | Public agent-based simulations published with physical source files and published baseline distributions. | `paysim`, `amlsim` | | |
| | `CONTROLLED_PROJECT_SYNTHETIC` | Reproducible benchmark datasets generated via domain-specific statistical models or academic methodology. | `synthaml`, `amlnet`, `cross_bank` | | |
| | `TEST_FIXTURE` | Minimal in-process deterministic schema fixtures for unit testing and CI smoke gates (`is_synthetic=True`). | Generated in `dataloader_smoke_fixtures.py` | | |
| ### 8.3 Provisioning Matrix & CI Reality | |
| | Dataset ID | Authoritative Source | Required Files & Format | Storage Location | Acquisition Method | Auth & License Requirements | Standard CI | Real-Data CI Gate | Local Execution | | |
| |:---|:---|:---|:---|:---|:---|:---:|:---:|:---| | |
| | **`paysim`** | Kaggle `ealaxi/paysim1` | `PS_20174392719_1491204439457_log.csv` (CSV) | `backend/storage/datasets/paysim/` | Kaggle API / CLI | Kaggle API Key; CC BY-SA 4.0 | Schema Fixture | Provisions via Kaggle API | Real file on disk | | |
| | **`ieee_cis`** | Kaggle `c/ieee-fraud-detection` | `train_transaction.csv`, `train_identity.csv` (CSV) | `backend/storage/datasets/ieee_cis/` | Kaggle Competition CLI | Kaggle API Key + Competition Rules Acceptance | Schema Fixture | Requires Kaggle credentials & rules acceptance | Real file on disk | | |
| | **`creditcard`** | Kaggle `mlg-ulb/creditcardfraud` | `creditcard.csv` (CSV) | `backend/storage/datasets/creditcard/` | Kaggle API / CLI | Kaggle API Key; ODbL 1.0 | Schema Fixture | Provisions via Kaggle API | Real file on disk | | |
| | **`elliptic`** | Kaggle `ellipticco/elliptic-data-set` | `elliptic_txs_features.csv`, `elliptic_txs_classes.csv`, `elliptic_txs_edgelist.csv` (CSV) | `backend/storage/datasets/elliptic/` | Kaggle API / CLI | Kaggle API Key; CC BY 4.0 | Schema Fixture | Provisions via Kaggle API | Real file on disk | | |
| | **`amlsim`** | Kaggle `anshankul/ibm-amlsim-example-dataset` | `transactions.csv`, `accounts.csv`, `alerts.csv` (CSV) | `backend/storage/datasets/amlsim/` | Kaggle API / CLI | Kaggle API Key; Apache 2.0 | Schema Fixture | Provisions via Kaggle API | Real file on disk | | |
| | **`synthaml`** | Figshare / Nature Sci Data DOI: `10.1038/s41597-023-02569-2` | `alerts.csv`, `transactions.csv` (CSV/Parquet) | `backend/storage/datasets/synthaml/` | Direct Download / `generate_synthaml_dataset.py` | Open Access; CC BY 4.0 | Schema Fixture | Local cache / Generator script | Real file on disk | | |
| | **`amlnet`** | Zenodo DOI: `10.5281/zenodo.10058474` | `transactions.csv` (CSV/Parquet) | `backend/storage/datasets/amlnet/` | Direct Download / `generate_amlnet_dataset.py` | Open Access; CC BY-NC 4.0 | Schema Fixture | Local cache / Generator script | Real file on disk | | |
| ### 8.4 Automated Provisioning Workflow | |
| The standalone GitHub Actions workflow `.github/workflows/real_data_integration.yml` provides a secure, gated pipeline for real-data integration: | |
| - Triggered manually via `workflow_dispatch` or on a weekly schedule. | |
| - Uses repository secrets `KAGGLE_USERNAME` and `KAGGLE_KEY`. | |
| - Executes `python scripts/download_real_benchmarks.py --dataset all`. | |
| - Fails closed if physical datasets cannot be provisioned when mandatory execution is requested. | |