chrisjcc commited on
Commit
ecfbd81
Β·
1 Parent(s): 39a338c

Add mock Confluence pages for the 'cquisition-and-Credit-Risk' space, designed specifically to demonstrate the Fraud Model Explainability Assistant with Confluence integration.

Browse files
Files changed (1) hide show
  1. docs/MOCK_CONFLUENCE_PAGES_README.md +324 -0
docs/MOCK_CONFLUENCE_PAGES_README.md ADDED
@@ -0,0 +1,324 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Mock Confluence Pages for Demo
2
+
3
+ This directory contains three comprehensive mock Confluence pages designed for demonstrating the Fraud Model Explainability Assistant with Confluence integration.
4
+
5
+ ---
6
+
7
+ ## Files Created
8
+
9
+ ### 1. fraud-model-governance.md
10
+ **Purpose**: Model documentation, SR 11-7 compliance, validation reports
11
+
12
+ **Key Sections**:
13
+ - Model Inventory
14
+ - XGBoost v3.2 Model Validation Report (detailed)
15
+ - Performance metrics and backtesting
16
+ - Fair lending analysis
17
+ - Champion/Challenger analysis
18
+ - SR 11-7 compliance documentation
19
+ - Approval records
20
+
21
+ **Example Queries This Answers**:
22
+ - "Find the model validation report for XGBoost v3.2"
23
+ - "What documentation do we have for model v3.2?"
24
+ - "What are the performance metrics for our fraud model?"
25
+ - "Show me the champion/challenger analysis"
26
+
27
+ **Content Highlights**:
28
+ - Comprehensive validation report with AUC, KS, precision, recall metrics
29
+ - Fair lending testing results by protected class
30
+ - Backtesting results showing model performance post-deployment
31
+ - Phone type feature analysis (relevant to compliance queries)
32
+ - SR 11-7 compliance checklist
33
+
34
+ ---
35
+
36
+ ### 2. compliance-policies.md
37
+ **Purpose**: Fair lending policy, ECOA compliance, adverse action guidelines
38
+
39
+ **Key Sections**:
40
+ - Fair Lending Policy
41
+ - ECOA Compliance Procedures
42
+ - Adverse Action Guidelines with fraud-specific procedures
43
+ - Synthetic ID Detection and Fair Lending
44
+ - **Phone Type Features Policy** (critical for demo queries)
45
+ - Model Feature Fair Lending Requirements
46
+ - Regulatory Examination Preparation
47
+
48
+ **Example Queries This Answers**:
49
+ - "What does our fair lending policy say about synthetic ID detection?"
50
+ - "What does our fair lending policy say about phone type features?"
51
+ - "Check fair lending compliance for APP-55555 and cite relevant policies"
52
+ - "How do we document this for the OCC exam?"
53
+ - "What are our adverse action procedures for suspected fraud?"
54
+
55
+ **Content Highlights**:
56
+ - Detailed phone type features policy with fair lending analysis
57
+ - Adverse impact ratios for phone type features by protected class
58
+ - Synthetic ID detection fair lending requirements
59
+ - Adverse action reason codes for fraud cases
60
+ - Sample examiner Q&A responses
61
+
62
+ ---
63
+
64
+ ### 3. fraud-investigation-playbooks.md
65
+ **Purpose**: Investigation procedures, fraud patterns, escalation procedures
66
+
67
+ **Key Sections**:
68
+ - Investigation Framework
69
+ - **Synthetic ID Fraud Investigation** (with detailed checklist)
70
+ - High-Risk Application Investigation
71
+ - Identity Network Analysis
72
+ - Escalation Procedures
73
+ - Case Studies (3 detailed examples)
74
+ - Fraud Patterns and Red Flags
75
+
76
+ **Example Queries This Answers**:
77
+ - "What are the procedures for escalating high-risk applications?"
78
+ - "What should I check for synthetic ID fraud?"
79
+ - "How do I investigate a high-risk application?"
80
+ - "What are the red flags for bust-out fraud?"
81
+ - "Show me examples of fraud investigation cases"
82
+
83
+ **Content Highlights**:
84
+ - Complete synthetic ID investigation checklist (6 steps)
85
+ - Real investigation example with full documentation
86
+ - Network fraud analysis procedures
87
+ - Escalation path diagram
88
+ - 3 case studies with lessons learned
89
+ - Comprehensive red flag categories with severity ratings
90
+
91
+ ---
92
+
93
+ ## Using These Files for Demo
94
+
95
+ ### Option 1: Direct Use as Markdown
96
+ The files are already in Markdown format and can be used directly with the confluence-ingestor if it supports Markdown input.
97
+
98
+ ### Option 2: Convert to Confluence Format
99
+ If you need actual Confluence pages for the demo:
100
+
101
+ 1. **Manual Upload**:
102
+ - Create a Confluence space called "Acquisition-and-Credit-Risk"
103
+ - Create three pages with the content from these files
104
+ - Confluence will convert the Markdown automatically when you paste it
105
+
106
+ 2. **Automated Import**:
107
+ - Use Confluence's bulk import feature
108
+ - Convert Markdown to Confluence wiki format using pandoc:
109
+ ```bash
110
+ pandoc fraud-model-governance.md -f markdown -t confluence -o fraud-model-governance.confluence
111
+ ```
112
+
113
+ 3. **Mock Ingestion**:
114
+ - Use these files as mock data for testing the RAG pipeline
115
+ - Store them in a local directory and ingest directly if your system supports file-based ingestion
116
+
117
+ ### Option 3: Use as Mock Vector Database Content
118
+ If you want to simulate Confluence without actually setting it up:
119
+
120
+ 1. Ingest these three Markdown files directly into ChromaDB
121
+ 2. Tag them with mock Confluence metadata:
122
+ - Space key: "Acquisition-and-Credit-Risk"
123
+ - Page IDs: fraud-model-governance (001), compliance-policies (002), fraud-investigation-playbooks (003)
124
+ - URLs: Mock Confluence URLs
125
+
126
+ ---
127
+
128
+ ## Demo Query Examples
129
+
130
+ These mock pages are specifically designed to answer the example queries in the Gradio interface:
131
+
132
+ ### Query 1: Model Governance
133
+ **User**: "Find the model validation report for XGBoost v3.2"
134
+
135
+ **Expected Result**: Should retrieve fraud-model-governance.md and cite:
136
+ - Model validation report section
137
+ - Performance metrics (AUC 0.870, KS 0.542)
138
+ - Fair lending testing results
139
+ - Approval signatures and dates
140
+
141
+ ---
142
+
143
+ ### Query 2: Fair Lending - Synthetic ID
144
+ **User**: "What does our fair lending policy say about synthetic ID detection?"
145
+
146
+ **Expected Result**: Should retrieve compliance-policies.md and cite:
147
+ - Synthetic ID Detection and Fair Lending section
148
+ - Approved detection features (SSN validity, credit file characteristics)
149
+ - Fair lending testing requirements
150
+ - Prohibited methods
151
+ - Quarterly disparate impact testing procedures
152
+
153
+ ---
154
+
155
+ ### Query 3: Fair Lending - Phone Type Features
156
+ **User**: "What does our fair lending policy say about phone type features?"
157
+
158
+ **Expected Result**: Should retrieve compliance-policies.md and cite:
159
+ - Phone Type Features Policy section
160
+ - Fair lending analysis table showing adverse impact ratios
161
+ - Policy statement (must pass 80% rule, quarterly testing)
162
+ - Approved phone type features (phone type flag, tenure, velocity)
163
+ - Monitoring and governance requirements
164
+
165
+ ---
166
+
167
+ ### Query 4: Investigation Procedures
168
+ **User**: "What are the procedures for escalating high-risk applications?"
169
+
170
+ **Expected Result**: Should retrieve fraud-investigation-playbooks.md and cite:
171
+ - Escalation Procedures section
172
+ - When to escalate (fraud score > 700, network fraud, etc.)
173
+ - Escalation path (Level 1 β†’ Level 2 β†’ Level 3 β†’ Level 4)
174
+ - Escalation process (4 steps)
175
+ - Escalation contacts table with response times
176
+
177
+ ---
178
+
179
+ ### Query 5: Synthetic ID Investigation
180
+ **User**: "What should I check for synthetic ID fraud?"
181
+
182
+ **Expected Result**: Should retrieve fraud-investigation-playbooks.md and cite:
183
+ - Synthetic ID Fraud Investigation section
184
+ - Synthetic ID red flags (primary and secondary indicators)
185
+ - 6-step investigation checklist
186
+ - SSN validation procedures
187
+ - Example investigation with full documentation
188
+
189
+ ---
190
+
191
+ ### Query 6: Compliance Check with Policy Citation
192
+ **User**: "Check fair lending compliance for APP-55555 and cite relevant company policies"
193
+
194
+ **Expected Result**: Should retrieve both compliance-policies.md AND fraud-model-governance.md:
195
+ - Fair lending policy statement
196
+ - ECOA compliance procedures
197
+ - Model feature fair lending requirements
198
+ - XGBoost v3.2 fair lending testing results
199
+ - Specific adverse impact ratios for protected classes
200
+ - Phone type features analysis
201
+
202
+ ---
203
+
204
+ ## Content Design Principles
205
+
206
+ These mock pages were designed with the following principles:
207
+
208
+ ### 1. Realism
209
+ - Corporate Confluence page style and formatting
210
+ - Realistic policy language and regulatory references
211
+ - Actual financial services terminology and frameworks
212
+ - Proper compliance and governance structures
213
+
214
+ ### 2. Searchability
215
+ - Rich keyword coverage for semantic search
216
+ - Multiple ways to refer to the same concept
217
+ - Specific examples that match Gradio query examples
218
+ - Cross-references between pages
219
+
220
+ ### 3. Demo-Optimized
221
+ - Content specifically answers the 9 example queries in the Gradio interface
222
+ - Includes specific examples mentioned (XGBoost v3.2, phone type features)
223
+ - Provides enough detail for impressive demo responses
224
+ - Includes tables, metrics, and structured data
225
+
226
+ ### 4. Educational Value
227
+ - Real fraud patterns and investigation techniques
228
+ - Actual regulatory frameworks (SR 11-7, ECOA, Regulation B)
229
+ - Genuine fair lending considerations
230
+ - Professional best practices
231
+
232
+ ---
233
+
234
+ ## Statistics
235
+
236
+ **Total Content**:
237
+ - 3 comprehensive Confluence pages
238
+ - ~23,000 words total
239
+ - ~150KB of text content
240
+ - 50+ sections across all pages
241
+ - 15+ tables with structured data
242
+ - 3 detailed case studies
243
+ - 20+ code examples and templates
244
+
245
+ **Coverage**:
246
+ - Model governance: βœ… Complete
247
+ - Fair lending compliance: βœ… Complete
248
+ - Fraud investigation procedures: βœ… Complete
249
+ - Synthetic ID detection: βœ… Extensive coverage
250
+ - Phone type features: βœ… Detailed policy
251
+ - Escalation procedures: βœ… Complete workflows
252
+ - Regulatory examination prep: βœ… Sample Q&A
253
+
254
+ ---
255
+
256
+ ## Next Steps
257
+
258
+ ### For Demo Setup:
259
+
260
+ 1. **Copy files to appropriate location**:
261
+ ```bash
262
+ # If using local file ingestion
263
+ mkdir -p demo_confluence_pages
264
+ cp fraud-*.md compliance-*.md demo_confluence_pages/
265
+ ```
266
+
267
+ 2. **Ingest into RAG pipeline**:
268
+ ```python
269
+ from confluence_ingestor import ConfluenceRAG
270
+
271
+ # Option A: If using actual Confluence (after uploading)
272
+ rag = ConfluenceRAG.from_env()
273
+ rag.ingest_space("Acquisition-and-Credit-Risk", max_pages=10)
274
+
275
+ # Option B: If using local files (modify as needed)
276
+ # Implement file-based ingestion using your RAG pipeline
277
+ ```
278
+
279
+ 3. **Test with example queries**:
280
+ - Run each of the 9 Gradio example queries
281
+ - Verify relevant sections are retrieved
282
+ - Check citation accuracy (page titles, URLs)
283
+
284
+ 4. **Fine-tune retrieval**:
285
+ - Adjust `k` parameter (number of results)
286
+ - Test space filtering
287
+ - Verify semantic search quality
288
+
289
+ ---
290
+
291
+ ## Maintenance
292
+
293
+ To update the mock pages for future demos:
294
+
295
+ 1. **Add new content**: Follow existing structure and formatting
296
+ 2. **Update dates**: Change "Last Updated" dates to current
297
+ 3. **Add new examples**: Create additional queries in Gradio that these pages can answer
298
+ 4. **Expand case studies**: Add more fraud investigation examples
299
+ 5. **Update regulations**: Keep regulatory references current
300
+
301
+ ---
302
+
303
+ ## License and Usage
304
+
305
+ **Usage**: These files are mock examples for demonstration purposes only.
306
+
307
+ **Not for Production**: This is fictitious content designed for demos and testing. Do not use as actual compliance or governance documentation.
308
+
309
+ **Customization**: Feel free to modify content to match your demo needs, add additional sections, or create additional mock pages.
310
+
311
+ ---
312
+
313
+ ## Questions?
314
+
315
+ For questions about these mock Confluence pages or how to use them in your demo:
316
+ - See CONFLUENCE_SETUP_GUIDE.md for integration instructions
317
+ - See QUICK_START.md for 10-minute setup
318
+ - See IMPLEMENTATION_SUMMARY.md for complete integration overview
319
+
320
+ ---
321
+
322
+ **Created**: January 2026
323
+ **Purpose**: Fraud Model Explainability Assistant Demo
324
+ **Status**: Ready for use βœ