docs/collaborative_aml_architecture.md

Architecture & System Design: Collaborative AML Platform

This document describes the architectural additions, system components, protocols, and end-to-end data flows of the collaborative cross-bank Anti-Money Laundering (AML) platform.

---

Component Overview

The collaborative AML platform builds upon the foundational Federated Learning architecture, adding real-time alert processing, case management with four-eyes dual-control governance, privacy-preserving entity resolution (DH-PSI and Fuzzy MinHash LSH), relationship graph analysis (Neo4j / Memgraph), federated graph embeddings (GraphSAGE), and regulatory e-filing (FinCEN SAR 2.0).

graph TD
    subgraph PresentationLayer ["Presentation Layer"]
        UI["React Web Console / React Flow<br/>[InvestigationDashboard, CaseDetailPage, PsiPage]"]
        WS["WebSocket Gateway (/ws/alerts, /ws/scenarios)"]
        CLI["CFI Operator CLI (cfi)"]
    end

    subgraph ApiLayer ["API & Orchestration Layer (Presentation Routers)"]
        FastAPI["FastAPI Control Plane (Gateway)"]
        AlertsR["Alerts Router (/api/v1/alerts)"]
        CasesR["Cases Router (/api/v1/cases)"]
        EntitiesR["Entities Router (/api/v1/entities)"]
        GraphR["Graph Router (/api/v1/graph)"]
        ScenariosR["Scenarios Router (/api/v1/scenarios)"]
        DashboardR["Dashboard Router (/api/v1/dashboard)"]
        PredictR["Prediction Router (/api/v1/predict)"]
        CoordR["Coordinator Router (/api/v1/coordinator)"]
        PrivacyR["Privacy Defense Router (/api/v1/privacy-defense)"]
        ComplianceR["Compliance Router (/api/v1/compliance)"]
        OnboardR["Onboarding Router (/api/v1/onboarding)"]
        SecurityR["Security Router (/api/v1/security)"]
        MonitoringR["Monitoring Router (/api/v1/monitoring)"]
        Psd2R["PSD2 Router (/api/v1/psd2)"]
    end

    subgraph AppLayer ["Application & Business Logic Services"]
        RiskEng["Risk Scoring Engine (9 Composite Signals)"]
        AlertSvc["Alert Intelligence Service"]
        CaseWorkbench["Investigator Case Workbench Service"]
        RegReporter["Regulatory Reporter Service (FinCEN SAR XML)"]
        EntitySvc["Entity Resolution Service (MinHash LSH)"]
        PsiSvc["PSI Service (Diffie-Hellman Commutative PSI)"]
        GraphEng["Graph Engine (Neo4j Bolt / In-Memory Adjacency)"]
        GraphAnalytics["Graph Analytics Service (PageRank Decay, Velocity)"]
        GraphEmbed["Graph Embedding Service (FedGNN / GraphSAGE)"]
        StreamEng["Streaming Engine & Flink Graph Streaming"]
        CoordSvc["Federated Coordinator Service (Registry & Heartbeats)"]
        PrivacyAudit["Privacy Defense & Audit Service (MIA, Inversion, DLG)"]
        FeedbackLoop["Label Feedback Loop Pipeline"]
    end

    subgraph DataLayer ["Data & Persistence Layer"]
        DB[("PostgreSQL Multi-Tenant Database")]
        Cache[("Redis Event Broker & Pub/Sub Cache")]
        GraphDB[("Neo4j / Memgraph Distributed Graph DB")]
        Vault[("HashiCorp Vault PKI & KMS Secrets Engine")]
    end

    UI --> FastAPI
    CLI --> FastAPI
    WS --> StreamEng
    FastAPI --> AlertsR & CasesR & EntitiesR & GraphR & ScenariosR & DashboardR & PredictR & CoordR & PrivacyR & ComplianceR & OnboardR & SecurityR & MonitoringR & Psd2R

    AlertsR --> AlertSvc
    PredictR --> AlertSvc
    CasesR --> CaseWorkbench
    EntitiesR --> EntitySvc & PsiSvc
    GraphR --> GraphEng & GraphAnalytics & GraphEmbed
    ScenariosR --> StreamEng
    DashboardR --> RiskEng
    CoordR --> CoordSvc
    PrivacyR --> PrivacyAudit
    ComplianceR --> RegReporter

    AlertSvc --> RiskEng
    AlertSvc --> EntitySvc
    EntitySvc --> PsiSvc
    EntitySvc --> GraphEng
    CaseWorkbench --> FeedbackLoop
    CaseWorkbench --> RegReporter
    StreamEng --> Cache
    Cache --> WS

    AlertSvc & CaseWorkbench & EntitySvc & GraphEng & CoordSvc --> DB
    GraphEng --> GraphDB
    FastAPI --> Vault

Layer Responsibilities

Layer	Primary Components	Key Files	Responsibility
Presentation	Web UI, React Flow, WebSockets, CLI	InvestigationDashboard.tsx, CaseDetailPage.tsx, PsiPage.tsx, CoordinatorPage.tsx	Live interactive dashboards, graph visualizer, case detail workbench, PSI sandbox, operator CLI.
API & Routers	FastAPI REST endpoints & WebSockets	cases.py, coordinator.py, privacy_defense.py, compliance.py	Inbound payload validation, JWT / mTLS authentication, ABAC policy enforcement, HTTP/WebSocket serialization.
Application Services	Domain services & business logic	case_workbench.py, regulatory_reporter.py, psi_service.py, graph_analytics_service.py	Risk scoring (9 signals), entity resolution (LSH MinHash), PSI execution, GraphSAGE embeddings, case lifecycle state machines, SAR e-filing.
Domain Layer	Dataclasses & pure value objects	case_management.py, investigation_entities.py, value_objects_investigation.py	Framework-independent domain models, immutable value objects, type-safe status enums, deterministic hashing formulas.
Infrastructure & Storage	Persistence & cryptography	otel_tracer.py, kms_service.py, FinCEN_SAR_2.0.xsd	PostgreSQL schemas, Redis pub/sub broker, Neo4j Bolt driver, HashiCorp Vault PKI engine, OpenTelemetry / Prometheus telemetry.

---

Data Flow: End-to-End Real-Time Detection, Resolution, Investigation & Retraining

The complete data lifecycle coordinates streaming transactions, collaborative privacy-preserving intelligence, four-eyes case investigation, and model feedback loops:

[Core Banking Systems / CBS Adapters]
  (ISO 20022 pacs.008, SWIFT MT103, REST CBS, PSD2 XS2A)
        │
        ▼ (Streaming Event Ingestion)
[Streaming Engine & Flink Graph Streaming] ──(Pub/Sub)──► [Redis Channels] ──► [WebSockets] ──► [Frontend UI]
        │
        ▼ (Real-Time Scoring)
[Risk Scoring Engine]
   ├── Evaluates 9 signals (Federated ML Model, Graph Risk, Velocity, Geolocation, Amount, etc.)
   └── Outputs Composite Risk Score (0 - 1000)
        │
        ▼ (If Risk Score > Threshold [e.g. ≥ 700])
[Alert Intelligence Service]
   ├── Generates Alert Entity with audit metadata
   ├── Converts PII to PrivacyPreservingIdentifiers (Type-Salted HMAC-SHA256)
   └── Broadcasts SharedIntelligence indicator across consortium banks
        │
        ▼ (Trigger Cross-Bank Matching)
[Entity Resolution Service & PSIService]
   ├── Executes Diffie-Hellman Commutative PSI (DH-PSI) over attribute sets
   ├── Computes Locality-Sensitive Hashing (MinHash LSH) over character 3-grams
   └── Identifies cross-institution overlaps without exposing non-matching records
        │
        ▼ (Update Graph Topology)
[Graph Engine & Graph Analytics Service]
   ├── Registers resolved nodes & cross-bank edges in Neo4j / Redis
   ├── Propagates risk scores across neighbors using decay factor (γ = 0.85)
   ├── Detects high-risk communities and fraud syndicates
   └── Computes 12-dimensional node embeddings via Federated GraphSAGE (FedGNN)
        │
        ▼ (Case Assembly & Workbench)
[Investigator Case Workbench Service]
   ├── Groups related alerts and graph nodes into FraudCaseRecord
   ├── Manages CaseLifecycleStateMachine (NEW ➔ ASSIGNED ➔ UNDER_INVESTIGATION ➔ ESCALATED)
   └── Enforces Four-Eyes Dual Control (Requires 2 distinct supervisor signatures: SIG_SUPERVISOR_<ID>)
        │
        ├──► (If Confirmed True Positive) ──► [Regulatory Reporter Service]
        │                                         ├── Serializes FinCEN SAR XML 2.0 filing
        │                                         └── Validates against FinCEN_SAR_2.0.xsd schema
        │
        └──► (Verified Label Ingestion) ──► [Label Feedback Loop Pipeline]
                                                  ├── Updates local bank training partitions
                                                  ├── Applies Dirichlet non-IID class weighting
                                                  └── Triggers next Federated Learning Round

---

Privacy-Preserving Mechanics

To satisfy strict data protection regulations (GDPR Art. 9/22, CCPA, Bank Secrecy Act 31 U.S.C. 5318(g), Turkish KVKK), the architecture enforces four foundational security invariants:

1. Zero Raw PII Transmission

• No plaintext customer names, emails, telephone numbers, bank account numbers (IBAN), or device fingerprints are ever transmitted across bank boundaries or to the central coordinator.
• All PII is normalized and converted to deterministic cryptographic hashes locally within the bank's secure perimeter prior to any collaborative protocol execution.

2. Type-Salted HMAC-SHA256 Deterministic Hashing

• Hashes are computed using keyed-hash message authentication codes:

$$\mathrm{P}\mathrm{r}\mathrm{i}\mathrm{v}\mathrm{a}\mathrm{c}\mathrm{y}\mathrm{H}\mathrm{a}\mathrm{s}\mathrm{h}=\mathrm{H}\mathrm{M}\mathrm{A}\mathrm{C}\text{-}\mathrm{S}\mathrm{H}\mathrm{A}256(\mathrm{S}\mathrm{h}\mathrm{a}\mathrm{r}\mathrm{e}\mathrm{d}\mathrm{S}\mathrm{a}\mathrm{l}\mathrm{t},\mathrm{E}\mathrm{n}\mathrm{t}\mathrm{i}\mathrm{t}\mathrm{y}\mathrm{T}\mathrm{y}\mathrm{p}\mathrm{e}\mathrm{\parallel }\mathrm{S}\mathrm{t}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{a}\mathrm{r}\mathrm{d}\mathrm{i}\mathrm{z}\mathrm{e}\mathrm{d}\mathrm{V}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e})\backslash mathrm\{PrivacyHash\}\; =\; \backslash mathrm\{HMAC\backslash text\{-\}SHA256\}(\backslash mathrm\{SharedSalt\},\; \backslash mathrm\{EntityType\}\; \backslash mathbin\{\backslash Vert\}\; \backslash mathrm\{StandardizedValue\})$$

• Prefixing each value with its entity type (customer:, phone:, email:, device:, iban:) eliminates cross-attribute collision and prevents rainbow table matching across different identifier categories.
• Output hashes are formatted as compact 16-character hexadecimal strings for simulation tracking and audit chaining.

3. Commutative Diffie-Hellman Private Set Intersection (DH-PSI)

• Enables two banks ($A$ and $B$) to discover overlapping fraud identifiers ($x \in X_A \cap X_B$) without disclosing non-matching elements ($x \in X_A \setminus X_B$ or $x \in X_B \setminus X_A$):

• By modular exponentiation commutativity $(H(x)^a)^b \equiv (H(x)^b)^a \pmod p$, equality $c_{AB,i} = c_{BA,j}$ indicates an exact match while leaking zero information about unmatched values.

4. MinHash LSH Fuzzy Private Set Intersection

• Extracts overlapping character 3-grams over standardized entity strings.
• Applies $H = 16$ independent polynomial rolling hash seeds modulo a large prime $M$ to compute deterministic 16-dimensional MinHash signature vectors:

$$\mathrm{s}\mathrm{i}\mathrm{g}[i]=\underset{s\in S}{\min }((a_i\cdot h(s)+b_i)M)\backslash mathrm\{sig\}[i]\; =\; \backslash min\_\{s\; \backslash in\; S\}\; \backslash left(\; (a\_i\; \backslash cdot\; h(s)\; +\; b\_i)\; \backslash bmod\; M\; \backslash right)$$

• Enables near-duplicate detection and typo-resilient entity matching under strict zero raw PII policies.

5. Federated Learning Alignment

• Local models train on partitioned bank data using Local SGD with Differential Privacy (Opacus DP-SGD: $\epsilon, \delta$ guarantees).
• Model updates (gradients / weights) are aggregated securely using Byzantine-robust optimizers (FedAvg, FedProx, FedYogi, SCAFFOLD, Trimmed Mean, Bulyan) without sharing training samples.

---

Design Patterns & Architectural Choices

• Signal-Combiner Pattern: The RiskScoringEngine decouples 9 independent risk scoring strategies (federated ML inference, transaction velocity, high-risk destination geolocation, time-of-day anomaly, transaction amount deviation, device sharing risk, graph risk score, and historical fraud incidence). This enables dynamic weight rebalancing without altering the core pipeline.
• Separation of Concerns (Clean Architecture):
  • Domain Layer (case_management.py, investigation_entities.py, value_objects_investigation.py) maintains framework-independent business entities, status enums, and mathematical algorithms.
  • Application Layer (case_workbench.py, regulatory_reporter.py, psi_service.py, coordinator_service.py) orchestrates business operations and AML algorithms.
  • Presentation Layer (cases.py, coordinator.py, privacy_defense.py) handles network endpoints, OpenAPI contracts, and serializations.
• Finite State Machine (FSM) Pattern: The CaseLifecycleStateMachine governs case progression across immutable transition paths, enforcing actor identity logging and four-eyes dual authorization.
• Pub/Sub Scenario Replay: Redis Pub/Sub decouples high-throughput simulation engines from FastAPI WebSocket connections, eliminating latency jitter on browser dashboards during real-time load tests.

---

Distributed Federated Learning Engine (HTTP Engine)

When configured for distributed operation (fl_engine_type = "distributed"), the system transitions from an in-memory simulation to an enterprise multi-node distributed topology:

Node Layout and Networking

• Coordinator Node: The central fl-coordinator orchestrates federated rounds, checks client capabilities, distributes global model parameters, and executes aggregation.
• Bank Client Nodes: Bank clients (bank-a, bank-b, bank-c) execute within isolated container environments listening on dedicated HTTP ports (8011, 8012, 8013).

sequenceDiagram
    autonumber
    participant Coord as FL Coordinator
    participant BA as Bank Client A (bank-a:8011)
    participant BB as Bank Client B (bank-b:8012)
    participant BC as Bank Client C (bank-c:8013)

    Note over Coord, BC: 1. Initialization Phase
    Coord->>BA: POST /api/v1/bank-client/initialize
    BA-->>Coord: 200 OK (Dataset Initialized)
    Coord->>BB: POST /api/v1/bank-client/initialize
    BB-->>Coord: 200 OK
    Coord->>BC: POST /api/v1/bank-client/initialize
    BC-->>Coord: 200 OK

    loop Training Rounds (1 to N)
        Note over Coord, BC: 2. Local Training Phase
        Coord->>BA: POST /api/v1/bank-client/train (Global Model Weights)
        Coord->>BB: POST /api/v1/bank-client/train (Global Model Weights)
        Coord->>BC: POST /api/v1/bank-client/train (Global Model Weights)
        
        Note over BA: Train on Local Partition (SGD/DP)
        Note over BB: Train on Local Partition (SGD/DP)
        Note over BC: Train on Local Partition (SGD/DP)

        BA-->>Coord: 200 OK (Updated Weights + Sample Count + Local Loss)
        BB-->>Coord: 200 OK (Updated Weights + Sample Count + Local Loss)
        BC-->>Coord: 200 OK (Updated Weights + Sample Count + Local Loss)

        Note over Coord: 3. Secure Aggregation & Weight Update

        Note over Coord, BC: 4. Evaluation Phase
        Coord->>BA: POST /api/v1/bank-client/evaluate (Aggregated Weights)
        Coord->>BB: POST /api/v1/bank-client/evaluate (Aggregated Weights)
        Coord->>BC: POST /api/v1/bank-client/evaluate (Aggregated Weights)

        BA-->>Coord: 200 OK (Local Test Metrics)
        BB-->>Coord: 200 OK (Local Test Metrics)
        BC-->>Coord: 200 OK (Local Test Metrics)
    end

---

Event-Driven Federated Learning Engine (Redis Pub/Sub Engine)

When configured as event_driven (fl_engine_type = "event_driven"), communication shifts from synchronous HTTP requests to asynchronous event exchanges using Redis Pub/Sub:

Security & Networking Advantages

• Zero Inbound Port Exposure: Bank clients connect strictly as outbound consumers. No external listening ports are opened, adhering to strict corporate banking network perimeter policies.
• Decoupled Topologies: Neither the coordinator nor bank nodes require fixed IP routing tables or DNS host records.
• Robust Correlation Tracking: All transactions, task events, and replies maintain unique message correlation_id headers.

sequenceDiagram
    autonumber
    participant Coord as FL Coordinator
    participant Broker as Redis Event Broker
    participant BA as Bank Client A (Outbound Consumer)

    Note over Coord, BA: 1. Initialization Phase
    Coord->>Broker: Publish: bank_client_bank_a_init (CorrelationID: init_123)
    Broker->>BA: Deliver event
    Note over BA: Generate Local Partition Dataset
    BA->>Broker: Publish: bank_client_bank_a_init_response (CorrelationID: init_123)
    Broker-->>Coord: Deliver event (Init Complete)

    loop Training Rounds (1 to N)
        Note over Coord, BA: 2. Local Training Phase
        Coord->>Broker: Publish: bank_client_bank_a_train (Global Weights, CorrelationID: train_1)
        Broker->>BA: Deliver event
        Note over BA: Train on Local Partition (SGD/DP)
        BA->>Broker: Publish: bank_client_bank_a_train_response (Updated Weights, CorrelationID: train_1)
        Broker-->>Coord: Deliver event

        Note over Coord: 3. Secure Aggregation & Weight Update

        Note over Coord, BA: 4. Evaluation Phase
        Coord->>Broker: Publish: bank_client_bank_a_evaluate (Aggregated Weights, CorrelationID: eval_1)
        Broker->>BA: Deliver event
        Note over BA: Evaluate on Local Test Data
        BA->>Broker: Publish: bank_client_bank_a_evaluate_response (Metrics, CorrelationID: eval_1)
        Broker-->>Coord: Deliver event
    end

---

Privacy-Preserving Graph Intelligence

Phase 2 introduces Privacy-Preserving Entity Resolution (PSI) and Graph-Based Fraud Detection (Graph Analytics). These components operate jointly to match entities securely across banks and propagate threat risks over the transaction graph.

1. Diffie-Hellman Private Set Intersection (DH-PSI)

To match customers, cards, or device IDs without disclosing raw identifiers of non-overlapping records, we implement a commutative modular exponentiation protocol (DH-PSI):

1. Parameters: A shared 512-bit modular prime $p$ and a generator $g$.
2. Local Keys: Bank A generates private scalar key $a$; Bank B generates private scalar key $b$.
3. Pass 1:
   • Bank A encrypts its set of hashes $X_A$: $Y_A = { x^a \pmod p \mid x \in X_A }$.
   • Bank B encrypts its set of hashes $X_B$: $Y_B = { x^b \pmod p \mid x \in X_B }$.
   • They exchange their encrypted sets.
4. Pass 2:
   • Bank A encrypts Bank B's set: $Z_B = { y^a \pmod p \mid y \in Y_B } = { x^{ab} \pmod p }$.
   • Bank B encrypts Bank A's set: $Z_A = { y^b \pmod p \mid y \in Y_A } = { x^{ba} \pmod p }$.
5. Intersection: Since modular exponentiation is commutative ($x^{ab} \equiv x^{ba} \pmod p$), matching elements in $Z_A$ and $Z_B$ reveal the shared entities without exposing any other values.

sequenceDiagram
    autonumber
    participant BA as Bank Client A
    participant BB as Bank Client B
    Note over BA, BB: Pass 1: Local Encryption
    Note over BA: Encrypts set with private key 'a'
    Note over BB: Encrypts set with private key 'b'
    BA->>BB: Send A's encrypted set
    BB->>BA: Send B's encrypted set
    Note over BA, BB: Pass 2: Commutative Cross-Encryption
    Note over BA: Encrypts received set with key 'a'
    Note over BB: Encrypts received set with key 'b'
    BA->>BB: Send double-encrypted set (Z_B)
    BB->>BA: Send double-encrypted set (Z_A)
    Note over BA: Computes Z_A & Z_B intersection -> Matches

1.1 Fuzzy & Probabilistic Private Entity Resolution (LSH / Fuzzy PSI)

Deterministic exact-string matching fails when bank records differ slightly in spelling, accents, or formatting (e.g., "Yusuf Çalışır" vs "Yusuf Calisir"). Phase 2 implements a four-stage privacy-preserving fuzzy entity matching pipeline:

Stage 1 — Standardization Pipeline (standardize_input())

value_objects_investigation.py :: standardize_input(raw_value, entity_type) applies a pre-hashing normalization pipeline to all entity identifiers before any matching is attempted:

Entity Type	Transformation Applied
customer / merchant	Unicode NFC → Turkish character transliteration (ı→i, ş→s, ç→c, ğ→g, ö→o, ü→u, ß→ss) → NFD accent stripping → lowercase → strip non-alphanumeric → collapse whitespace
phone	Extract all digit characters; if original input starts with +, preserve + prefix (E.164 format)
email	Strip surrounding whitespace → lowercase
device / others	Strip → lowercase

This ensures that "Yusuf Çalışır" and "Yusuf Calisir" both standardize to "yusuf calisir" before any hash is computed.

Stage 2 — Locality-Sensitive Hashing (LSH) on Character n-grams (compute_minhash_signature())

value_objects_investigation.py :: compute_minhash_signature(text, num_hashes=16) generates a compact probabilistic fingerprint:

1. 3-gram Extraction: The standardized name is decomposed into a set of overlapping character 3-grams:

$$S=\{\mathrm{t}\mathrm{e}\mathrm{x}\mathrm{t}[i:i+3]\mid 0\le i\le \mathrm{l}\mathrm{e}\mathrm{n}(\mathrm{t}\mathrm{e}\mathrm{x}\mathrm{t})-3\}S\; =\; \backslash \{\; \backslash mathrm\{text\}[i:i+3]\; \backslash mid\; 0\; \backslash le\; i\; \backslash le\; \backslash mathrm\{len\}(\backslash mathrm\{text\})\; -\; 3\; \backslash \}$$

e.g., "yusuf calisir" → {"yus", "usu", "suf", "uf ", ...}

2. MinHash Signature: $H=16$ independent hash seeds $i$ are applied to each shingle $s$ using a deterministic polynomial rolling hash modulo a large prime $M$:

$$\mathrm{s}\mathrm{i}\mathrm{g}[i]=\underset{s\in S}{\min }((a_i\cdot h(s)+b_i)M)\backslash mathrm\{sig\}[i]\; =\; \backslash min\_\{s\; \backslash in\; S\}\; \backslash left(\; (a\_i\; \backslash cdot\; h(s)\; +\; b\_i)\; \backslash bmod\; M\; \backslash right)$$

3. Jaccard Approximation: Similarity between two signatures is estimated as the fraction of matching positions:

$$\widehat{J}(S_1,S_2)=\frac{\mathrm{\mid }\{i\mid {\mathrm{s}\mathrm{i}\mathrm{g}}_1[i]={\mathrm{s}\mathrm{i}\mathrm{g}}_2[i]\}\mathrm{\mid }}{H}\backslash widehat\{J\}(S\_1,\; S\_2)\; =\; \backslash frac\{|\backslash \{\; i\; \backslash mid\; \backslash mathrm\{sig\}\_1[i]\; =\; \backslash mathrm\{sig\}\_2[i]\; \backslash \}|\}\{H\}$$

The 16-dimensional MinHash vector is stored directly in the entity's attributes["minhash_signature"] field, eliminating the need for a separate LSH index store.

Stage 3 — Multi-Attribute Fuzzy PSI (PSIService.run_psi(enable_fuzzy=True))

psi_service.py :: run_psi(bank_a_id, bank_b_id, entity_type, enable_fuzzy=True, fuzzy_threshold=3) executes a threshold-based multi-attribute matching protocol:

1. Attribute Set: 5 key PII attributes are evaluated per entity pair: phone, email, device_id, birthdate, surname.
2. Independent DH-PSI per Attribute: Standard Diffie-Hellman commutative exponentiation is executed independently over each attribute's PrivacyPreservingIdentifier hash:

$$Z_a^{(\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{r})}={(H({\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{r}}_a))}^{k_b\cdot k_a}(\mathrm{m}\mathrm{o}\mathrm{d}p)Z\_a^\{(\backslash mathrm\{attr\})\}\; =\; \backslash left(\; H(\backslash mathrm\{attr\}\_a)\; \backslash right)^\{k\_b\; \backslash cdot\; k\_a\}\; \backslash pmod\; p$$

3. k-of-n Threshold Gate: A cross-bank pair is declared a match if:

$$\mid \mathrm{m}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}\mathrm{e}\mathrm{d}\mathrm{\_}\mathrm{a}\mathrm{t}\mathrm{t}\mathrm{r}\mathrm{s}\mid \ge k(\text{default: }k=3\text{ of }n=5)\backslash left|\; \backslash mathrm\{matched\backslash \_attrs\}\; \backslash right|\; \backslash ge\; k\; \backslash quad\; (\backslash text\{default:\; \}\; k\; =\; 3\; \backslash text\{\; of\; \}\; n\; =\; 5)$$

4. Similarity Score: The output match record includes a continuous overlap score $= |\mathrm{matched_attrs}| / n$.

Stage 4 — Central LSH Registry (EntityResolutionService.resolve_fuzzy_entities())

entity_resolution.py :: resolve_fuzzy_entities(query_name, entity_type, threshold=0.70) provides a direct name-to-entity fuzzy lookup:

• Standardizes and computes the MinHash signature of the query string.
• Iterates all stored entities, reads their minhash_signature attribute, and computes Jaccard similarity.
• Returns all entities above the configurable similarity threshold, sorted by descending similarity.
• Exposed to the frontend via POST /api/v1/entities/fuzzy-resolve.

Frontend Integration (PsiPage.tsx)

The PsiPage.tsx React component provides an interactive three-panel UI:

• PSI Protocol Control Center: Configures bank pair, entity type, Intel SGX TEE toggle, fuzzy enable/disable, and threshold slider ($k = 1–5$).
• MinHash Spelling Playground: Local JavaScript implementation of standardize_input() and compute_minhash_signature() that shows real-time 16-dimensional signature comparison and Jaccard score for any two name inputs without making API calls.
• Central LSH Registry Query Panel: Calls /api/v1/entities/fuzzy-resolve and displays matched entities with similarity scores, bank, risk level, standardized form, and privacy hash.

---

2. Graph Analytics & Risk Propagation

Once matches are resolved, a multi-bank transaction graph is constructed. The engine executes three structural graph algorithms:

• PageRank-Like Risk Propagation: Starting with known high-risk/critical alerts, risk scores are propagated to neighboring nodes using a decay factor ($\gamma = 0.85$) and weight adjustments based on relationship type (e.g. SHARES_DEVICE has higher weight than SHARES_IP).
• Community Analytics: Connected components are grouped, and community-level statistics (size, average risk, and fraud density) are computed to isolate potential fraud rings.
• Temporal Velocity Anomalies: Edges are grouped in sliding time windows (e.g., 5 minutes) to detect high-velocity relationship creation bursts that signal structuring or automated laundering networks.

2.5 Dedicated Distributed Graph Database Integration (Neo4j / Memgraph)

To handle massive scales of customer relationships, transactions, and alert linkages at sub-second latency, the Graph Engine supports a dedicated, distributed graph database backend (Neo4j or Memgraph) via the Bolt protocol.

• Cypher Queries: Replaces CPU-bound custom Python traversal loops (BFS/DFS) with highly optimized Cypher queries executed directly in the database:
  • Neighbor Search query:

    MATCH (s:Entity {id: $entity_id})-[r]-(n:Entity)
    WHERE ($relationship_types IS NULL OR r.relationship_type IN $relationship_types)
    RETURN DISTINCT n
    

  • Subgraph Extraction query:

    MATCH (s:Entity {id: $center_id})
    OPTIONAL MATCH p = (s)-[*1..$radius]-(n:Entity)
    RETURN s, collect(p) as paths
    

• Real-time GNN Serving: Enables compatibility with real-time Graph Neural Network inference runtimes (e.g. Memgraph GNN modules / DGL) to update node embeddings dynamically as new transaction edges are written.
• Dual-Storage & Fallback: Configured via graph_db_type in settings (supporting "redis", "neo4j", "memgraph"). If the graph database is not reachable or not installed, the engine gracefully falls back to the Redis / in-memory adjacency list to maintain complete backward compatibility in development.
• Topological Ring Analytics & Mule Loop Detection:
  • Detects closed transaction cycles of length $L \in [3, 7]$ using directed DFS traversal in memory or native Cypher variable-length path matching MATCH path = (start:Entity)-[r*3..7]->(start).
  • Implements canonical rotation deduplication: rotates each cycle to begin with its lexicographically minimum entity ID, preventing identical cycle duplicates under cyclic permutations. Assigns deterministic SHA-256 ring digests.
  • Multi-bank risk scoring: evaluates cross-bank boundaries ($\mathrm{banks}_{\mathrm{involved}} \ge 2$), entity risk baselines, and cycle compactness.
• Multi-Hop Financial Smurfing Detection:
  • Fan-In (Aggregation Mules): Identifies hubs receiving convergent micro-deposits from $\ge 3$ distinct sources.
  • Fan-Out (Dispersion Mules): Identifies hubs dispersing funds to $\ge 3$ target accounts to circumvent reporting thresholds.
  • Multi-Hop Layering: Identifies transit hubs exhibiting concurrent high fan-in and fan-out across multiple hops (depth $\ge 2$).
• Safe Parameterized Cypher Execution:
  • Rejects mutation keywords (CREATE, MERGE, DELETE, SET, REMOVE, DROP, DETACH) when read_only=True via strict regex parsing.
  • Thread safety enforced across all graph state mutations and queries via threading.RLock().

2.6 Advanced AI Explainability Portal (Counterfactuals, Decision Replay, GNNExplainer)

To satisfy strict regulatory requirements ("Right to Explanation" under GDPR Art. 22) and provide compliance officers with audit-grade transparency:

1. Counterfactual Remediation Engine:
   • Computes actionable feature modifications ($x \to x'$) that lower an alert's risk score below the remediation threshold (e.g. $<350.0$).
   • Formulates human-understandable remediation statements (e.g., "Reduce amount by $45.00 AND originate transaction from US instead of RU").
2. Deterministic Decision Replay (Inference Audit):
   • Reproduces historical risk scoring decisions deterministically ($| \text{score}{\text{replay}} - \text{score}{\text{orig}} | < 0.01$).
   • Retrieves model version metadata (v2.4.0-champion), feature vector snapshots, 9-signal policy rule outcomes, and graph snapshots at transaction timestamp.
3. GNNExplainer Subgraph Attribution:
   • Calculates edge contribution percentages over entity 2-hop neighborhoods via message-passing masking.
   • Highlights specific relationship types (e.g., SHARES_DEVICE with known mule account) driving the GraphSAGE risk embedding.
4. LIME Local Linear Surrogate (Interpretable Model-Agnostic Explanations):
   • Evaluates local decision surfaces via continuous Gaussian feature perturbations $z \sim \mathcal{N}(x, \sigma^2)$ weighted by an exponential distance kernel $w(z) = \exp(-D(x, z)^2 / \sigma^2)$.
   • Solves closed-form L2 regularized Ridge regression $\beta = (X_{\mathrm{surr}}^T W X_{\mathrm{surr}} + \lambda I)^{-1} X_{\mathrm{surr}}^T W y$ to extract signed feature attributions and compute surrogate fidelity $R^2$.
5. Mathematical Shapley Efficiency & Process-Safe Caching:
   • Enforces the efficiency axiom $\sum_{i=1}^M \phi_i = f(x) - \mathbb{E}[f(x)]$ with exact normalization.
   • Preserves process-wide RNG integrity during KernelSHAP evaluation and protects real-time fast-path caching with threading.RLock().

2.7 Production Enterprise Security Suite (mTLS, OIDC, Vault, ABAC, Immutable Audit Chain)

To satisfy enterprise banking security standards (ISO 27001, SOC2, PCI-DSS):

1. Mutual TLS 1.3 (mTLS): Enforces TLS 1.3 with client certificate requirement (CERT_REQUIRED), HashiCorp Vault PKI Root CA integration, automated zero-downtime certificate rotation (cert rotation), Subject Alternative Name (SAN) validation, and CRL revocation checks for all inter-microservice communication.
2. OIDC / OAuth2 JWT Authentication: Authenticates users with signed JWT bearer tokens, parsing standard (sub, iss, exp) and custom claims (bank_id, roles, clearance_level, shift_hours, approval_tier).
3. Dynamic Attribute-Based Access Control (ABAC): Evaluates dynamic policy rules matching user attributes against resource properties:
   • Tenant Isolation: Restricts data access strictly to the user's home bank ID unless holding cross_bank_investigator or super_admin roles.
   • Shift Hours Restriction: Enforces access windows (e.g. 08:00-18:00).
   • Approval Tier Limit: Limits high-value operations (> 50,000 USD) to qualified authorization tiers.
   • Security Clearance: Restricts classified intelligence by clearance level.
4. HashiCorp Vault & Live PKI Integration: Centralizes secrets management via Vault KV v2 secret engine and provisions dynamic X.509 certificates via HashiCorp Vault PKI Secrets Engine (/v1/pki/issue/cfi-bank-role), backed by automated bootstrap script (scripts/init_vault_pki.py) and environment fallbacks.
5. Tamper-Evident Cryptographic Audit Chain: Chains every system event using SHA-256 hash chaining ($H_i = \mathrm{SHA256}(L_i \mathbin{\Vert} H_{i-1})$) with a 1-click verify_chain_integrity() tool to detect retrospective log tampering.

2.8 Enterprise Observability, Log Aggregation & Model Drift Engine

To maintain continuous MLOps model quality and infrastructure health:

1. PLG Log Aggregation Stack: Grafana Loki + Promtail scrape container log streams. Backend uses structured JSONLogFormatter tagging log entries with tenant_id, bank_id, and OpenTelemetry trace_id.
2. Prometheus Alertmanager: Evaluates real-time metric thresholds (monitoring/prometheus/alert_rules.yml):
   • High Gateway Latency: $p95 \text{ latency} > 100\text{ms}$ for 5m.
   • High Client Dropout: $>50%$ bank client dropout during FL rounds.
   • Significant Concept Drift: Concept drift $PSI > 0.20$ for 1m.
   • Poor Model Calibration: Brier score $> 0.15$ for 1m.
3. Statistical Model Drift Engine (ModelDriftService):
   • Feature Drift: Kolmogorov-Smirnov 2-sample test ($p$-value threshold $<0.05$) and Wasserstein distance across incoming features against reference baselines.
   • Concept Drift: Population Stability Index (PSI) over transaction risk scores:

$$\mathrm{P}\mathrm{S}\mathrm{I}=\sum _{i=1}^k(A_i-E_i)\times \ln \left(\frac{A_i}{E_i}\right)\backslash mathrm\{PSI\}\; =\; \backslash sum\_\{i=1\}^k\; (A\_i\; -\; E\_i)\; \backslash times\; \backslash ln\backslash left(\backslash frac\{A\_i\}\{E\_i\}\backslash right)$$

4. Model Calibration Monitoring: Evaluates probability calibration via Brier score ($\frac{1}{N}\sum (p_i - y_i)^2$), Expected Calibration Error (ECE), and 10-bin reliability curve generation.
5. Automated Re-training Triggers: When concept drift $PSI \ge 0.20$, the platform automatically alerts MLOps engineers and triggers a new federated training round (trigger_auto_retraining()).

2.9 GitOps & Container Orchestration Pipeline (Kubernetes, Helm & ArgoCD)

To ensure high availability, zero-downtime rolling updates, and declarative environment alignment:

1. Kubernetes Orchestration: The containerized microservices leverage Horizontal Pod Autoscaling (HPA) to scale between 2 and 10 replicas based on CPU demand.
2. Helm Charts Packaging: Standardizes packaging across services (gateway, fl-coordinator, identity-graph, fraud-alert, frontend) under a unified chart (helm/cfi-platform/). Parameters for resources, storage classes, ingress hosts, database connectivity, and secrets are dynamically injected via values.yaml.
3. Declarative GitOps (ArgoCD): An ArgoCD Application manifest (argocd/application.yaml) tracks target repositories and syncs Kubernetes resources automatically whenever changes are pushed to git, maintaining a strict source of truth.
4. CI Pipeline Linting: The GitHub Actions pipeline (ci.yml) executes helm lint validation on all pull requests to verify manifest syntax correctness before build promotion.

3. Federated Graph Embedding (FedGNN)

To move beyond heuristic relationship weights, the platform introduces a Federated GraphSAGE (Sample and Aggregate) pipeline to learn structural graph embeddings collaboratively:

1. Local Graph Representation: Each bank constructs a graph mapping its entities to a 12-dimensional numerical feature representation (entity types, risk levels, alert logs, local degrees, and activity recency).
2. GraphSAGE Model: A 2-layer GraphSAGE architecture performs message-passing:

$${\mathbf{h}}_{\mathcal{N}(v)}^{(k)}=\mathrm{A}\mathrm{G}\mathrm{G}\mathrm{R}\mathrm{E}\mathrm{G}\mathrm{A}\mathrm{T}\mathrm{E}\left(\{{\mathbf{h}}_u^{(k-1)},\mathrm{\forall }u\in \mathcal{N}(v)\}\right)\backslash mathbf\{h\}\_\{\backslash mathcal\{N\}(v)\}^\{(k)\}\; =\; \backslash mathrm\{AGGREGATE\}\backslash left(\backslash left\backslash lbrace\backslash mathbf\{h\}\_u^\{(k-1)\},\; \backslash forall\; u\; \backslash in\; \backslash mathcal\{N\}(v)\backslash right\backslash rbrace\backslash right)$$

$${\mathbf{h}}_v^{(k)}=\sigma ({\mathbf{W}}^{(k)}\cdot [{\mathbf{h}}_v^{(k-1)}\mathrm{\parallel }{\mathbf{h}}_{\mathcal{N}(v)}^{(k)}])\backslash mathbf\{h\}\_v^\{(k)\}\; =\; \backslash sigma\backslash left(\backslash mathbf\{W\}^\{(k)\}\; \backslash cdot\; \backslash left[\backslash mathbf\{h\}\_v^\{(k-1)\}\; \backslash mathbin\{\backslash Vert\}\; \backslash mathbf\{h\}\_\{\backslash mathcal\{N\}(v)\}^\{(k)\}\backslash right]\backslash right)$$

3. Federated Aggregation: Only GNN parameters ($\mathbf{W}^{(k)}$ projection weights) are sent to the coordinator. The coordinator aggregates GNN parameters using Krum or FedAvg, then redistributes the global GNN.
4. Downstream Analytics:
   • Embedding-Enhanced Propagation: Connected node risk transfer weights are calculated dynamically using cosine similarity of GNN embeddings rather than hardcoded heuristics.
   • Unconnected Clustering: Nodes are clustered across banks by embedding similarity, isolating coordinated syndicate networks that share a modus operandi without having a direct edge connection in the graph.

---

Decentralized Infrastructure & Network Isolation (Production Design)

To move away from monolithic or shared environments, the production-grade simulator models a multi-VPC decentralized cloud architecture using distinct networks and database containers.

1. Security Boundaries & Network-Level VPC Simulation

The system is divided into four private, isolated security zones and one shared federation channel:

• Bank Client Private Zones (net-bank-a, net-bank-b, net-bank-c): Contains the respective Bank Client microservice and its isolated database container (e.g. postgres-bank-a). No database port is exposed to other networks or the host, preventing cross-bank data leakage.
• Coordinator Private Zone (net-coordinator): Hosts the central control plane, including the Gateway, Federated Learning Coordinator, Identity & Graph Service, Fraud & Alert Engine, Celery Workers, Flower, Redis Event Broker, Jaeger, Prometheus, Grafana, and the coordinator database.
• Federation Network Channel (net-federation): A dedicated network bridge restricted to cross-zone communications. Only the fl-coordinator and the bank clients (bank-a, bank-b, bank-c) have interfaces on this network.

graph TD
    subgraph VPC Bank A
        postgres-a[(Postgres Bank A)]
        bank-a[Bank Client A Service]
        bank-a --- postgres-a
    end

    subgraph VPC Bank B
        postgres-b[(Postgres Bank B)]
        bank-b[Bank Client B Service]
        bank-b --- postgres-b
    end

    subgraph VPC Bank C
        postgres-c[(Postgres Bank C)]
        bank-c[Bank Client C Service]
        bank-c --- postgres-c
    end

    subgraph VPC Coordinator Control Plane
        fl-coord[FL Coordinator]
        postgres-coord[(Coordinator Postgres)]
        gateway[API Gateway]
        redis[Redis Event Bus]
        fl-coord --- postgres-coord
    end

    %% Federation Links
    fl-coord ===|net-federation| bank-a
    fl-coord ===|net-federation| bank-b
    fl-coord ===|net-federation| bank-c

2. Mutual Auth & Cryptographic Payload Signing

To protect the REST communication channel between the fl-coordinator and the bank-client nodes on net-federation, the API implements end-to-end payload signing:

1. Outbound Request Signing: The coordinator computes an HMAC-SHA256 signature using the shared PAYLOAD_SIGNING_SECRET, the current Unix timestamp, and the request body:

$$\mathrm{S}\mathrm{i}\mathrm{g}\mathrm{n}\mathrm{a}\mathrm{t}\mathrm{u}\mathrm{r}\mathrm{e}=\mathrm{H}\mathrm{M}\mathrm{A}\mathrm{C}\text{-}\mathrm{S}\mathrm{H}\mathrm{A}256(\mathrm{S}\mathrm{e}\mathrm{c}\mathrm{r}\mathrm{e}\mathrm{t},\mathrm{T}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{m}\mathrm{p}\mathrm{\parallel }\mathrm{P}\mathrm{a}\mathrm{y}\mathrm{l}\mathrm{o}\mathrm{a}\mathrm{d}\mathrm{B}\mathrm{y}\mathrm{t}\mathrm{e}\mathrm{s})\backslash mathrm\{Signature\}\; =\; \backslash mathrm\{HMAC\backslash text\{-\}SHA256\}(\backslash mathrm\{Secret\},\; \backslash mathrm\{Timestamp\}\; \backslash mathbin\{\backslash Vert\}\; \backslash mathrm\{PayloadBytes\})$$

2. Transmission: The request is sent with the custom headers X-Payload-Signature and X-Payload-Timestamp.
3. Inbound Validation: The bank client validates that:
   • The timestamp header is present and is within a $\pm 300\text{s}$ tolerance window (mitigating replay attacks).
   • The HMAC computed locally over the received raw request body matches the signature header.
4. Rejection: If signature verification fails, the request is rejected with a 401 Unauthorized response.

---

Real-Bank Connector Integrations (Core Banking Architecture)

To support seamless transitions from simulation to production banking architectures, the platform implements standardized, production-ready interfaces for core banking systems (CBS), standard messaging formats, open banking APIs, and message queues.

graph TD
    subgraph Core Banking Integration
        REST[RESTBankConnector] -->|OAuth2 / mTLS| CBS[Core Banking System]
        MQ[RabbitMQBankConnector] -->|AMQP / Pika| Queue[RabbitMQ Broker]
    end

    subgraph Standard Messaging
        Parser[FinancialMessageParser]
        Parser -->|Ingests| ISO[ISO 20022 XML pacs.008]
        Parser -->|Ingests| SWIFT[SWIFT MT103]
        Parser -->|Ingests| SEPA[SEPA Credit Transfer XML]
    end

    subgraph Open Banking PSD2 XS2A
        Router[PSD2 Router] -->|Bearer JWT Auth| AISP[Third-Party AISP Client]
        Router -->|Checks Consent| Consents[(Consent Store)]
    end

1. Production-Grade CBS Adapters

• mTLS Integration: The RESTBankConnector checks for configured client certificate files at runtime. If present, it establishes secure HTTPS connections using mutual TLS certificate verification.
• OAuth2 Authentication: For systems requiring token-based access, the adapter dynamically requests OAuth2 Client Credentials tokens from the configured authorization server (oauth_token_url), caches them locally, and attaches them as Bearer tokens to outbound requests.

2. Financial Message Parsers

The financial_message_parser.py normalizes real-time and bulk financial messages into transaction entities:

• ISO 20022 (pacs.008): Parses structured XML schemas to extract end-to-end IDs, settlement amounts, currencies, debtor (sender) and creditor (receiver) names, IBANs, and BICs.
• SWIFT MT103: Parses flat legacy SWIFT messages by scanning block 4 tags (e.g., :20: for reference, :32A: for value date/amount, :50K:/:59: for customer info).
• SEPA credit transfers: Standardizes incoming instant and credit transfer pain.001 or pacs.008 messages into the platform's schema.

3. Open Banking PSD2 API

The platform exposes standardized endpoints compliant with the PSD2 XS2A (Access to Account) mandate:

• Consent Management (/api/v1/psd2/consents): Allows third-party providers (AISPs) to register account access scopes and expiration dates.
• Account Access (/api/v1/psd2/accounts): Retrieves details of accounts matching active consents.
• Transaction Ingestion (/api/v1/psd2/accounts/{account_id}/transactions): Exposes historical transactions under active AISP consents.
• Bearer JWT Security: Endpoints require JWT validation signed with a secure secret, verifying standard claims (sub, scope, and expiration timestamp).

4. Enterprise Message Queues (RabbitMQ)

• Concrete RabbitMQ Connector: Implements BankConnectorInterface using the pika library. It publishes tasks to durable queues (fl.queue.{bank_id}.train, etc.) and consumes responses on dynamic, exclusive callback queues using matching correlation IDs.
• Resilient Failover: If the RabbitMQ broker is unreachable, the connector falls back to local in-memory nodes (MockBankConnector), preventing orchestration failure during local testing.

---

Enterprise Case Management Workbench & Regulatory SAR Architecture

The case_workbench.py and case_management.py services implement full-lifecycle case handling with strict Four-Eyes governance and automated FinCEN Suspicious Activity Report (SAR) XML generation.

stateDiagram-v2
    [*] --> NEW: create_case(alert_ids)
    NEW --> ASSIGNED: assign_investigator(investigator_id)
    ASSIGNED --> UNDER_INVESTIGATION: transition_to_investigation()
    UNDER_INVESTIGATION --> ESCALATED: escalate_case(reason)
    ESCALATED --> UNDER_INVESTIGATION: reassign()
    
    UNDER_INVESTIGATION --> PENDING_SECOND_SIGNATURE: add_supervisor_signature(sig_1)
    ESCALATED --> PENDING_SECOND_SIGNATURE: add_supervisor_signature(sig_1)
    
    PENDING_SECOND_SIGNATURE --> RESOLVED_TRUE_POSITIVE: resolve_case(sig_2, TRUE_POSITIVE)
    PENDING_SECOND_SIGNATURE --> RESOLVED_FALSE_POSITIVE: resolve_case(sig_2, FALSE_POSITIVE)
    PENDING_SECOND_SIGNATURE --> CLOSED_NO_ACTION: resolve_case(sig_2, CLOSED)

    RESOLVED_TRUE_POSITIVE --> FinCEN_SAR_Filing: generate_sar_xml()
    RESOLVED_TRUE_POSITIVE --> Retraining_Feedback_Loop: label=1 (Fraud)
    RESOLVED_FALSE_POSITIVE --> Retraining_Feedback_Loop: label=0 (Legitimate)

1. Four-Eyes Dual-Control Governance

• Dual Supervisor Requirement: High-risk cases cannot be closed or submitted to regulatory authorities without two independent supervisor cryptographic signatures (SIG_SUPERVISOR_<ID>).
• State Machine Invariants:
  • A single supervisor cannot supply both signatures (anti-collusion invariant).
  • Attempting to bypass PENDING_SECOND_SIGNATURE raises a domain state transition exception.
  • Every transition records the actor_id, timestamp, and reason notes into an append-only audit trail.

2. FinCEN BSA SAR 2.0 XML Generation & Validation

• The regulatory_reporter.py service generates e-filing compliant XML documents for confirmed money laundering cases.
• Documents are validated against the official XML Schema Definition at FinCEN_SAR_2.0.xsd, asserting:
  • <EFilingSubmission> root element.
  • Standard <SubmissionHeader> with ActivityType, SubmissionType, and timestamps.
  • <Activity> body containing institution identifiers, suspect subjects, detailed narratives, and financial transaction amounts.

3. Verified Retraining Label Feedback Loop

• Confirmed case outcomes are ingested by the label_feedback_pipeline.py.
• Validated labels (label = 1 for RESOLVED_TRUE_POSITIVE, label = 0 for RESOLVED_FALSE_POSITIVE) are merged into local bank training partitions using Dirichlet non-IID class weighting, continuously improving model detection accuracy in successive FL rounds.

---

Enterprise Federated Coordinator Suite

The coordinator_service.py transforms static node topologies into a production-grade, self-healing FL network where bank nodes register dynamically, send heartbeats, and negotiate hardware-aware training parameters.

Architecture

sequenceDiagram
    autonumber
    participant Bank as Bank Node (Client)
    participant Coord as CoordinatorService
    participant Prom as Prometheus Gauge

    Note over Bank, Coord: Phase 1 — Handshake & Version Validation
    Bank->>Coord: POST /api/v1/coordinator/handshake<br/>{bank_id, pytorch_version, python_version, hardware_type, ram_gb}
    Coord->>Coord: Validate PyTorch ≥ 2.x AND Python ≥ 3.10
    alt Compatible
        Coord-->>Bank: {registered: true, status: "COMPATIBLE"}
        Coord->>Coord: Store ClientCapability in registry dict
    else Incompatible
        Coord-->>Bank: {registered: false, status: "INCOMPATIBLE", reason: "..."}
    end

    Note over Bank, Coord: Phase 2 — Live Heartbeat Loop (every ~10s)
    loop Every 10 seconds while ONLINE
        Bank->>Coord: POST /api/v1/coordinator/heartbeat<br/>{bank_id}
        Coord->>Coord: Update last_heartbeat = time.time()
        Coord-->>Bank: {alive: true, status: "ONLINE"}
    end

    Note over Coord, Prom: Background Monitor (sweeps every call to get_active_clients)
    Coord->>Coord: For each client: if now - last_heartbeat > timeout → mark OFFLINE
    Coord->>Prom: cfi_active_clients_count.record(len(active_clients))

    Note over Bank, Coord: Phase 3 — Parameter Negotiation
    Bank->>Coord: GET /api/v1/coordinator/negotiate?bank_id=X&base_batch_size=64&base_epochs=5
    Coord->>Coord: Lookup hardware_type and ram_gb from registry
    alt CUDA and RAM ≥ 16 GB
        Coord-->>Bank: {batch_size: 64, local_epochs: 5, gradient_accumulation_steps: 1, use_cuda: true, status: "COMPATIBLE"}
    else CPU or RAM < 8 GB
        Coord-->>Bank: {batch_size: 16, local_epochs: 3, gradient_accumulation_steps: 4, use_cuda: false, status: "DEGRADED"}
    end

Components

Component	File	Responsibility
CoordinatorService	coordinator_service.py	Client registry, heartbeat tracking, runtime version validator, hardware-aware parameter negotiation.
coordinator Router	coordinator.py	REST endpoints: /handshake, /heartbeat, /clients, /negotiate.
cfi_active_clients_count	otel_tracer.py	Prometheus gauge tracking live online client count.
CoordinatorPage	CoordinatorPage.tsx	Live registry UI, heartbeat health indicators, and API reference.

Heartbeat Timeout Logic

The coordinator uses a passive sweep model: get_active_clients() iterates the registry on each call and marks clients OFFLINE when last_heartbeat age exceeds heartbeat_timeout_seconds (default: 15s). This avoids background thread contention and works transparently with FastAPI async request handlers.

Parameter Negotiation Strategy

Hardware Profile	Batch Size	Local Epochs	Gradient Accumulation	CUDA	Status
CUDA + RAM ≥ 16 GB	Base (e.g. 64)	Base (e.g. 5)	1	✅	COMPATIBLE
CUDA + RAM 8–16 GB	Base × 0.75	Base − 1	2	✅	COMPATIBLE
CPU + RAM ≥ 8 GB	32	min(base, 3)	2	❌	DEGRADED
CPU + RAM < 8 GB	16	min(base, 3)	4	❌	DEGRADED
Unregistered	16	min(base, 3)	4	❌	DEGRADED

REST API Endpoints

Method	Path	Description
POST	/api/v1/coordinator/handshake	Register a new bank node with runtime version validation
POST	/api/v1/coordinator/heartbeat	Record a heartbeat ping for a registered bank
GET	/api/v1/coordinator/clients	List all registered clients with status and heartbeat age
GET	/api/v1/coordinator/negotiate	Return negotiated training parameters for a bank's hardware profile

---

Advanced Privacy Defense & Adversarial Attack Benchmarking

To audit and protect aggregated model weights against adversarial attacks, the system integrates Byzantine-robust optimizers and the privacy_audit_service.py.

                  ┌──────────────────────────────┐
                  │      FL Server (Server)      │
                  └──────────────┬───────────────┘
                                 │
               Runs aggregate_parameters() (fl_engine.py)
                                 │
         ┌───────────────────────┴───────────────────────┐
         ▼                                               ▼
[Trimmed Mean (coordinate)]                      [Bulyan (nested)]
  - Removes largest/smallest f updates             - Selects closest candidate (Krum)
  - Averages remaining updates                     - Trims coordinate-wise extremes
                                                   - Averages remaining parameters

Components

Component	File	Responsibility
Bulyan & Trimmed Mean	fl_engine.py	Byzantine-robust aggregation algorithms defending against colluding malicious banks.
PrivacyAuditService	privacy_audit_service.py	Evaluates Membership Inference Attacks (MIA), Model Inversion (gradient variance), and DLG (Pearson correlation).
PrivacyService	privacy_service.py	Multi-simulation $\epsilon$ privacy budget consumption tracking with exhaustion flags.
privacy_defense Router	privacy_defense.py	REST API routes: /aggregation-methods, /audit/mia, /audit/model-inversion, /audit/dlg, and /budget-log.
PrivacyDefensePage	PrivacyDefensePage.tsx	Dashboard displaying Byzantine defense methods, active attack simulations, and budget gauges.

Robust Aggregation Algorithms

1. Coordinate-wise Trimmed Mean

For each parameter coordinate, sorts received weights from $N$ clients. Removes the $f$ lowest and $f$ highest updates, where $f$ represents the estimated number of Byzantine/malicious workers ($2f < N$). Computes the mean of the remaining $N - 2f$ values:

$${\mathrm{T}\mathrm{r}\mathrm{i}\mathrm{m}\mathrm{m}\mathrm{e}\mathrm{d}\mathrm{M}\mathrm{e}\mathrm{a}\mathrm{n}}_i=\frac{1}{N-2f}\sum _{k=f+1}^{N-f}{w}_{(k),i}\backslash mathrm\{TrimmedMean\}\_i\; =\; \backslash frac\{1\}\{N\; -\; 2f\}\; \backslash sum\_\{k=f+1\}^\{N-f\}\; w\_\{(k),\; i\}$$

2. Bulyan

Defends against colluding attackers that Krum or Median alone cannot mitigate:

1. Selects $\theta = N - 2f$ candidate weight vectors using Krum (minimizing multi-client Euclidean distance).
2. For each coordinate $i$, sorts the parameters of the $\theta$ selected updates.
3. Applies Trimmed Mean on the sorted parameters by discarding $f'$ largest and $f'$ smallest values ($f' = \theta - 2f$), averaging the remainder.

REST API Endpoints

Method	Path	Description
GET	/api/v1/privacy-defense/aggregation-methods	List Byzantine robust aggregation methods (Krum, Median, Trimmed Mean, Bulyan)
POST	/api/v1/privacy-defense/audit/mia	Run Membership Inference Attack (MIA) audit based on loss distributions
POST	/api/v1/privacy-defense/audit/model-inversion	Run Model Inversion Audit based on variance of gradient norms
POST	/api/v1/privacy-defense/audit/dlg	Evaluate Deep Leakage from Gradients (DLG) via Pearson correlation
GET	/api/v1/privacy-defense/budget-log	Get sorted multi-simulation privacy budget consumption logs

---

Public Dataset Benchmarks & Advanced FL Optimization

Public Dataset Loaders (dataloader.py)

dataloader.py loads standard benchmarks with automatic synthetic mock fallback for offline CI/CD testing:

Dataset	Source	Feature Dim	Fraud Ratio	Graph Structure?
Elliptic Bitcoin	Kaggle / EllipticDataset	166	~2 %	✅ node-edge CSV
AMLSim	IBM Research	6 (tabular)	~1.5 %	—
PaySim / Kaggle CC	Kaggle Credit Card Fraud	29 (V1-V28 + Amount)	~0.17 %	—

Each loader conforms to the contract:

data = load_dataset("elliptic", n_mock_nodes=2000)
# Returns: {"X": np.ndarray, "y": np.ndarray, "edges": list, "source": "real"|"mock"}

Real CSV files are looked up under storage/datasets/<name>/. When absent, synthetic mocks matching exact feature dimensions and label distributions are generated automatically.

FedYogi — Adaptive Server Optimizer

Reference: Reddi et al., "Adaptive Federated Optimization" (ICLR 2021)

FedYogi introduces a server-side adaptive second-moment update that prevents effective learning rates from collapsing in sparse gradient regimes:

Compared to FedAdam, FedYogi slows variance growth and maintains larger effective updates on parameters where $v$ already exceeds $\Delta^2$, which is critical for highly skewed fraud distributions.

SCAFFOLD — Control-Variate Client-Drift Correction

Reference: Karimireddy et al., "SCAFFOLD: Stochastic Controlled Averaging for Federated Learning" (ICML 2020)

SCAFFOLD mitigates client drift induced by non-IID distributions by maintaining control variates $c$ (global) and $c_i$ (client-local):

g_i ← ∇L_i(w) - c_i + c        # corrected gradient
w_i ← w_i - lr · g_i
c_i+ ← c_i - c + (1/K·lr)·(w_old - w_new)

Server updates compute weighted FedAvg on parameter updates, correcting directional bias across heterogeneous bank partitions.

Offline Benchmark Runner (benchmark_real_data.py)

Evaluates cross-products over Datasets (Elliptic, AMLSim, PaySim), Optimizers (FedAvg, FedProx, SCAFFOLD, MOON, FedYogi), and Byzantine Defenses (None, Krum, Bulyan):

# Full benchmark run:
python benchmark_real_data.py

# Offline CI/CD smoke test:
python benchmark_real_data.py --mock-only --n-samples 300 --rounds 2 --epochs 1

Results are saved to storage/benchmark_results.md with ROC-AUC, PR-AUC, and F1 metrics.

---

Cross-Bank Collaborative Intelligence & Case Evidence Ledger

The cross-bank intelligence subsystem operationalizes democratic consortium governance into an active, privacy-preserving investigation workflow:

sequenceDiagram
    autonumber
    actor Analyst as Bank A Investigator
    participant API as FastAPI Gateway
    participant CBI as CrossBankIntelligenceService
    participant Gov as ConsortiumGovernanceService
    participant Entity as EntityResolutionService
    participant Case as CaseWorkbench
    participant Ledger as EvidenceRegistryService

    Analyst->>API: POST /api/v1/intelligence/cross-bank/query
    API->>CBI: query_cross_bank_intelligence(request, principal)
    CBI->>Gov: get_consortium(consortium_id)
    Gov-->>CBI: Consortium state (ACTIVE, members)
    Note over CBI: Assert Bank A active member<br/>Assert Bank B active member<br/>Evaluate sharing policy
    alt Hardware Attestation Requested & Missing
        CBI-->>API: 412 Precondition Failed (AttestationError)
        API-->>Analyst: Fail Closed
    else Software Emulation Mode
        CBI->>Entity: resolve_entity(query, target_bank)
        Entity-->>CBI: Matching entities (confidence, category)
        Note over CBI: Strip raw identifiers<br/>Generate sanitized signal
        CBI-->>API: CrossBankIntelligenceQueryResponse
        API-->>Analyst: Sanitized signals (Zero Raw PII)
    end

    Analyst->>API: POST /api/v1/cases/{case_id}/evidence/cross-bank-intelligence
    API->>CBI: attach_evidence_to_case(payload, principal)
    CBI->>Case: get_case(case_id)
    Case-->>CBI: Case record (status, bank_id)
    Note over CBI: Assert case not in terminal state<br/>Assert owning bank matches principal
    CBI->>Ledger: record_evidence(case_id, signal_hash, metadata)
    Ledger-->>CBI: EvidenceRecord (SHA-256 hash)
    CBI-->>API: CrossBankEvidenceAttachResponse
    API-->>Analyst: Evidence anchored to case

Privacy & Anti-Fabrication Principles

1. Zero-Raw-PII Disclosure Guarantee: Cross-bank query responses never transmit raw customer names, tax identification numbers, cleartext account numbers, phone numbers, or private internal notes. Responses are strictly limited to sanitized match categories (e.g., SUSPECTED_MULE_NETWORK), normalized confidence scores ($0.0 - 1.0$), pseudonymized cross-bank indicators, and evidence timestamps.
2. Truthful Cryptographic Enclave Boundary & Simulation Isolation: In compliance with AGENTS.md truthfulness rules, the system explicitly marks software-emulated Private Set Intersection and entity resolution as SIMULATED_SOFTWARE_ONLY (emulation_mode: True, is_authoritative: False, is_synthetic: True). Software simulation is strictly disabled in production (enable_cross_bank_simulation = False), failing closed with invariant validation. Querying operational entities via simulation is prohibited. When institutional policy mandates physical hardware enclave execution (require_hardware_attestation: True), the platform refuses to fabricate measurement quotes and fails closed with AttestationError (HTTP 412 Precondition Failed).
3. Immutable Evidence Anchoring & Collision-Resistant Durability: Signals evaluated by an analyst are verified via server-side SHA-256 integrity validation (content_hash) and anchored to the institution's case evidence ledger via EvidenceRegistryService. Evidence files are persisted atomically with collision-resistant paths (case_{case_id}_signal_{resolved_signal_id}_{payload_digest[:16]}.json). If file writes fail, execution fails closed with EvidencePersistenceError without claiming durability. If optimistic concurrency (CAS) updates fail, newly written files and registry entries are rolled back. Simulated or synthetic signals cannot be attached to authoritative fraud cases, SAR preparations, or escalated cases. Terminal cases (CONFIRMED_FRAUD, FALSE_POSITIVE, CLOSED_NO_ACTION) reject further mutations (HTTP 409 Conflict), preserving regulatory audit integrity under Four-Eyes dual control.

---

Automated Verification & Test Coverage Matrix

All platform components are continuously verified through unit and integration suites under backend/tests/unit/ and backend/tests/integration/:

Test Suite	Targeted Module / Service	Verified Features	Test Count	Status
test_fuzzy_psi.py	value_objects_investigation, psi_service	Turkish transliteration, MinHash LSH, 3-of-5 attribute threshold gate	3	✅ 100% Pass
test_psi_service.py	psi_service	Commutative DH-PSI exponentiation, match identification, cross-encryption	3	✅ 100% Pass
test_psi_fuzzy_domain.py	value_objects_investigation	Normalization edge cases, punctuation stripping, phone E.164 standardization	4	✅ 100% Pass
test_coordinator_service.py	coordinator_service	Dynamic handshake, heartbeat sweep, CUDA/RAM parameter negotiation	19	✅ 100% Pass
test_privacy_defense_router.py	privacy_defense.py	Aggregation method listing, MIA / Inversion / DLG audit endpoints	7	✅ 100% Pass
test_privacy_audit.py	privacy_audit_service	Loss distribution MIA, reconstruction score, Pearson leakage metric	15	✅ 100% Pass
test_privacy_service.py	privacy_service	Multi-simulation $\epsilon$ log tracking, threshold exhaustion alerts	15	✅ 100% Pass
test_graph_analytics.py	graph_analytics_service	PageRank risk propagation with decay ($\gamma = 0.85$), community density, velocity	3	✅ 100% Pass
test_graph_embedding.py	graph_embedding_service	12-dim node feature extraction, GraphSAGE forward pass, FedAvg GNN aggregation	21	✅ 100% Pass
test_graph_embedding_hardening.py	graph_embedding_service	Inductive inference on unseen nodes, DP noise injection, weight shape validation, similarity query budget & thread safety	10	✅ 100% Pass
test_neo4j_graph.py	graph_engine	Neo4j Bolt driver init, Cypher entity/relationship merges, Redis fallback	8	✅ 100% Pass
test_graph_analytics_hardening.py	graph_engine	Cyclic mule rings ($L \in [3, 7]$), canonical deduplication, smurfing, safe Cypher, thread locks	10	✅ 100% Pass
test_flink_graph_streaming.py	flink_graph_streaming	High-velocity streaming edge sliding windows, anomaly triggers	3	✅ 100% Pass
test_streaming_gnn_hardening.py	streaming_gnn_model, flink_graph_streaming, streaming_graph_service	Multi-neighbor index_add_, exponential time decay w(t), Flink burst rate/volume, thread safety, streaming API	10	✅ 100% Pass
test_elliptic_benchmark_hardening.py	elliptic_benchmark_service	Real Bitcoin graph parsing, row alignment, genuine PyTorch GraphSAGE training, metrics, concurrency & REST API	10	✅ 100% Pass
test_advanced_explainability.py	explainability_service	Counterfactual explanations, deterministic decision replay, GNNExplainer	6	✅ 100% Pass
test_explainability_hardening.py	explainability_service, realtime_explainer	Shapley efficiency, LIME Ridge surrogate, R^2 fidelity, RNG non-pollution, thread caching & purged mock edges	10	✅ 100% Pass
test_case_management_workbench.py	case_workbench	Case FSM transitions, four-eyes supervisor signatures (SIG_SUPERVISOR_<ID>)	4	✅ 100% Pass
test_regulatory_reporter.py	regulatory_reporter	FinCEN SAR XML 2.0 serialization, XML structure & XSD schema validation	5	✅ 100% Pass
test_cross_bank_intelligence_workflow.py	cross_bank_intelligence	Consortium authority checks, zero raw PII sanitization, fail-closed TEE boundary, evidence ledger anchoring	7	✅ 100% Pass
test_cross_bank_intelligence_integration.py	cross_bank_intelligence	FastAPI REST integration flow, 403 non-member rejection, 412 TEE fail-closed, 409 terminal immutability	5	✅ 100% Pass
Total Verified	21 Dedicated Suites	Collaborative AML Platform Architecture	178 Tests	✅ 100% Pass