Hugging Face's logo

Yoge-2004
/
expense-intelligence-model

Text Classification
Joblib
English
finance
expense-tracker
tabular
scikit-learn
tf-idf
Model card Files
xet
 Community
Yoge-2004 commited on
Commit
73b95ee
·
verified ·
1 Parent(s): 30f333a

Release Expense Tracker intelligence models and reports

Browse files
Files changed (28) hide show
  1. .gitattributes +4 -0
  2. README.md +55 -0
  3. manifest.json +1530 -0
  4. metadata.json +18 -0
  5. model.joblib +3 -0
  6. models/category-tfidf/metadata.json +18 -0
  7. models/category-tfidf/model.joblib +3 -0
  8. models/duplicate-similarity/model.joblib +3 -0
  9. models/merchant-similarity/model.joblib +3 -0
  10. reports/REPORT.md +21 -0
  11. reports/category_india_holdout.json +207 -0
  12. reports/category_test.json +207 -0
  13. reports/category_test_country_metrics.json +1037 -0
  14. reports/dataset_quality.json +30 -0
  15. reports/dataset_summary.json +29 -0
  16. reports/duplicate_candidates.json +5007 -0
  17. reports/figures/dataset_class_distribution.png +0 -0
  18. reports/figures/text_length_distribution.png +0 -0
  19. reports/figures/tfidf-test_confusion_matrix.png +3 -0
  20. reports/figures/tfidf-test_per_class_metrics.png +3 -0
  21. reports/figures/tfidf-validation_confusion_matrix.png +3 -0
  22. reports/figures/tfidf-validation_per_class_metrics.png +3 -0
  23. reports/model_comparison.json +629 -0
  24. reports/model_selection_validation.json +213 -0
  25. reports/split_summary.json +10 -0
  26. reports/tfidf_validation.json +207 -0
  27. reports/tfidf_validation_country_metrics.json +1037 -0
  28. reports/training_sampling.json +43 -0
.gitattributes CHANGED
@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ reports/figures/tfidf-test_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text
37
+ reports/figures/tfidf-test_per_class_metrics.png filter=lfs diff=lfs merge=lfs -text
38
+ reports/figures/tfidf-validation_confusion_matrix.png filter=lfs diff=lfs merge=lfs -text
39
+ reports/figures/tfidf-validation_per_class_metrics.png filter=lfs diff=lfs merge=lfs -text
README.md ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language:
3
+ - en
4
+ license: mit
5
+ tags:
6
+ - finance
7
+ - expense-tracker
8
+ - text-classification
9
+ - tabular
10
+ - scikit-learn
11
+ - tf-idf
12
+ datasets:
13
+ - Ranjit0034/finee-dataset
14
+ - Sumeetgpt/indian-transaction-categorization-synthetic
15
+ - mitulshah/transaction-categorization
16
+ metrics:
17
+ - accuracy
18
+ - f1
19
+ pipeline_tag: text-classification
20
+ widget:
21
+ - text: "Uber ride from airport to downtown"
22
+ - text: "Dominos Pizza slice and coke"
23
+ - text: "Netflix monthly subscription fee"
24
+ - text: "Monthly salary credited from employer"
25
+ - text: "Apollo Pharmacy medicines and prescription"
26
+ ---
27
+
28
+ # Expense Intelligence Model
29
+
30
+ Production-ready financial transaction classification and auxiliary intelligence suite for the Expense Tracker System.
31
+
32
+ ## Model Overview
33
+ - Architecture: Dual-Gram TF-IDF Vectorizer (Word 1-2 grams + Char-WB 3-5 grams, 200,000 max features) coupled with Calibrated Multiclass SAGA Logistic Regression.
34
+ - Inference Speed: Under 5ms per transaction on standard CPU.
35
+ - Accuracy: 99.30% overall test accuracy (99.32% Macro-F1) across 240,000+ holdout transactions.
36
+ - India Holdout: 98.85% accuracy (98.96% Macro-F1) on Indian banking and UPI transactions.
37
+
38
+ ## Supported Canonical Categories
39
+ 1. food_dining
40
+ 2. transportation
41
+ 3. shopping_retail
42
+ 4. entertainment_recreation
43
+ 5. healthcare_medical
44
+ 6. utilities_services
45
+ 7. financial_services
46
+ 8. income
47
+ 9. government_legal
48
+ 10. charity_donations
49
+
50
+ ## Auxiliary Artifacts Included
51
+ - `models/category-tfidf/`: Primary category classifier
52
+ - `models/merchant-similarity/`: Normalized merchant similarity search index
53
+ - `models/duplicate-similarity/`: Near-duplicate transaction detection model
54
+ - `reports/`: Complete validation and test metrics across countries (India, USA, UK, Canada, Australia)
55
+ - `manifest.json`: Full training provenance, environment, and quality gate scores
manifest.json ADDED
@@ -0,0 +1,1530 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "status": "completed",
3
+ "run_id": "20260918T064102Z",
4
+ "pipeline_version": "2.0.0",
5
+ "created_at_utc": "2026-09-18T06:41:02.474685+00:00",
6
+ "python": "3.14.5 (main, May 10 2026, 19:28:16) [Clang 22.1.3 ]",
7
+ "platform": "Linux-6.12.90+-x86_64-with-glibc2.35",
8
+ "seed": 42,
9
+ "taxonomy_categories": [
10
+ "food_dining",
11
+ "transportation",
12
+ "shopping_retail",
13
+ "entertainment_recreation",
14
+ "healthcare_medical",
15
+ "utilities_services",
16
+ "financial_services",
17
+ "income",
18
+ "government_legal",
19
+ "charity_donations"
20
+ ],
21
+ "resources": {
22
+ "cpu_threads": 4,
23
+ "torch_threads": 4,
24
+ "device": "cpu",
25
+ "cuda_devices": 0,
26
+ "torch_configured": false,
27
+ "platform": "Linux-6.12.90+-x86_64-with-glibc2.35"
28
+ },
29
+ "datasets": [],
30
+ "models": {
31
+ "category_tfidf": {
32
+ "status": "trained",
33
+ "artifact": "models/category-tfidf",
34
+ "validation_accuracy": 0.9922900091368398,
35
+ "validation_macro_f1": 0.9924938150790148,
36
+ "final_training_rows": 1359243,
37
+ "test_accuracy": 0.9929848825051134,
38
+ "test_macro_f1": 0.9932028965957418
39
+ },
40
+ "category_transformer": {
41
+ "status": "disabled"
42
+ },
43
+ "merchant_similarity": {
44
+ "status": "trained",
45
+ "artifact": "models/merchant-similarity",
46
+ "training_rows": 250000
47
+ },
48
+ "duplicate_similarity": {
49
+ "status": "trained",
50
+ "artifact": "models/duplicate-similarity",
51
+ "training_rows": 500000
52
+ },
53
+ "transaction_anomaly": {
54
+ "status": "not_applicable",
55
+ "reason": "No amount feature is present in the configured category datasets."
56
+ },
57
+ "spending_forecast": {
58
+ "status": "not_applicable",
59
+ "reason": "Category datasets do not provide transaction date and amount fields."
60
+ }
61
+ },
62
+ "selected_model": {
63
+ "name": "tfidf",
64
+ "artifact": "models/category-tfidf",
65
+ "selection_metric": "validation_macro_f1",
66
+ "validation_accuracy": 0.9922900091368398,
67
+ "validation_macro_f1": 0.9924938150790148,
68
+ "test_accuracy": 0.9929848825051134,
69
+ "test_macro_f1": 0.9932028965957418
70
+ },
71
+ "quality_gates": {
72
+ "minimum_accuracy": 0.9,
73
+ "minimum_macro_f1": 0.9,
74
+ "minimum_country_macro_f1": 0.85,
75
+ "minimum_india_macro_f1": 0.85,
76
+ "minimum_confidence_coverage": 0.75,
77
+ "minimum_high_confidence_accuracy": 0.98
78
+ },
79
+ "test_evaluation": {
80
+ "model_name": "tfidf",
81
+ "accuracy": 0.9929848825051134,
82
+ "macro_f1": 0.9932028965957418,
83
+ "weighted_f1": 0.9929778167820927,
84
+ "report": {
85
+ "charity_donations": {
86
+ "precision": 0.9999548451187573,
87
+ "recall": 1.0,
88
+ "f1-score": 0.9999774220496264,
89
+ "support": 22145.0
90
+ },
91
+ "entertainment_recreation": {
92
+ "precision": 0.999746589517253,
93
+ "recall": 0.9980183826629564,
94
+ "f1-score": 0.9988817385800189,
95
+ "support": 23718.0
96
+ },
97
+ "financial_services": {
98
+ "precision": 0.9880473946784922,
99
+ "recall": 0.9998247090169682,
100
+ "f1-score": 0.9939011640064125,
101
+ "support": 28524.0
102
+ },
103
+ "food_dining": {
104
+ "precision": 0.9960777698114799,
105
+ "recall": 0.9931457886548085,
106
+ "f1-score": 0.9946096184620568,
107
+ "support": 23781.0
108
+ },
109
+ "government_legal": {
110
+ "precision": 0.9907501211293661,
111
+ "recall": 0.9937704338605637,
112
+ "f1-score": 0.9922579791340407,
113
+ "support": 22634.0
114
+ },
115
+ "healthcare_medical": {
116
+ "precision": 0.9814401400744145,
117
+ "recall": 0.9882316643159379,
118
+ "f1-score": 0.9848241934421189,
119
+ "support": 22688.0
120
+ },
121
+ "income": {
122
+ "precision": 0.995253164556962,
123
+ "recall": 0.9974627339042182,
124
+ "f1-score": 0.9963567242198638,
125
+ "support": 22071.0
126
+ },
127
+ "shopping_retail": {
128
+ "precision": 0.9878508434881254,
129
+ "recall": 0.973775340884496,
130
+ "f1-score": 0.980762593325946,
131
+ "support": 27722.0
132
+ },
133
+ "transportation": {
134
+ "precision": 0.9976043374101626,
135
+ "recall": 0.9926396788223486,
136
+ "f1-score": 0.9951158159522062,
137
+ "support": 23912.0
138
+ },
139
+ "utilities_services": {
140
+ "precision": 0.9951458433550532,
141
+ "recall": 0.9955376673374748,
142
+ "f1-score": 0.9953417167851285,
143
+ "support": 22858.0
144
+ },
145
+ "accuracy": 0.9929848825051134,
146
+ "macro avg": {
147
+ "precision": 0.9931871049140065,
148
+ "recall": 0.9932406399459772,
149
+ "f1-score": 0.9932028965957418,
150
+ "support": 240053.0
151
+ },
152
+ "weighted avg": {
153
+ "precision": 0.9929953943768574,
154
+ "recall": 0.9929848825051134,
155
+ "f1-score": 0.9929778167820927,
156
+ "support": 240053.0
157
+ }
158
+ },
159
+ "confusion_matrix": [
160
+ [
161
+ 22145,
162
+ 0,
163
+ 0,
164
+ 0,
165
+ 0,
166
+ 0,
167
+ 0,
168
+ 0,
169
+ 0,
170
+ 0
171
+ ],
172
+ [
173
+ 0,
174
+ 23671,
175
+ 44,
176
+ 1,
177
+ 0,
178
+ 1,
179
+ 0,
180
+ 1,
181
+ 0,
182
+ 0
183
+ ],
184
+ [
185
+ 0,
186
+ 1,
187
+ 28519,
188
+ 1,
189
+ 0,
190
+ 0,
191
+ 3,
192
+ 0,
193
+ 0,
194
+ 0
195
+ ],
196
+ [
197
+ 1,
198
+ 2,
199
+ 82,
200
+ 23618,
201
+ 0,
202
+ 0,
203
+ 0,
204
+ 77,
205
+ 1,
206
+ 0
207
+ ],
208
+ [
209
+ 0,
210
+ 1,
211
+ 2,
212
+ 1,
213
+ 22493,
214
+ 0,
215
+ 102,
216
+ 34,
217
+ 1,
218
+ 0
219
+ ],
220
+ [
221
+ 0,
222
+ 0,
223
+ 47,
224
+ 0,
225
+ 0,
226
+ 22421,
227
+ 0,
228
+ 220,
229
+ 0,
230
+ 0
231
+ ],
232
+ [
233
+ 0,
234
+ 0,
235
+ 0,
236
+ 0,
237
+ 56,
238
+ 0,
239
+ 22015,
240
+ 0,
241
+ 0,
242
+ 0
243
+ ],
244
+ [
245
+ 0,
246
+ 0,
247
+ 60,
248
+ 89,
249
+ 154,
250
+ 423,
251
+ 0,
252
+ 26995,
253
+ 0,
254
+ 1
255
+ ],
256
+ [
257
+ 0,
258
+ 1,
259
+ 65,
260
+ 0,
261
+ 0,
262
+ 0,
263
+ 0,
264
+ 0,
265
+ 23736,
266
+ 110
267
+ ],
268
+ [
269
+ 0,
270
+ 1,
271
+ 45,
272
+ 1,
273
+ 0,
274
+ 0,
275
+ 0,
276
+ 0,
277
+ 55,
278
+ 22756
279
+ ]
280
+ ],
281
+ "samples": 240053,
282
+ "confidence_threshold": 0.7,
283
+ "confidence_coverage": 0.9808167363040662,
284
+ "high_confidence_accuracy": 0.9995540416567564
285
+ },
286
+ "india_holdout_evaluation": {
287
+ "model_name": "tfidf-india-holdout",
288
+ "accuracy": 0.9885360423757005,
289
+ "macro_f1": 0.9896056616563857,
290
+ "weighted_f1": 0.988553516466768,
291
+ "report": {
292
+ "charity_donations": {
293
+ "precision": 1.0,
294
+ "recall": 1.0,
295
+ "f1-score": 1.0,
296
+ "support": 5227.0
297
+ },
298
+ "entertainment_recreation": {
299
+ "precision": 0.9986962190352021,
300
+ "recall": 0.9855611150822016,
301
+ "f1-score": 0.9920851921139733,
302
+ "support": 6995.0
303
+ },
304
+ "financial_services": {
305
+ "precision": 0.9673148793310952,
306
+ "recall": 0.9997381342062193,
307
+ "f1-score": 0.9832592878758611,
308
+ "support": 15275.0
309
+ },
310
+ "food_dining": {
311
+ "precision": 0.9956268221574344,
312
+ "recall": 0.9859968240219431,
313
+ "f1-score": 0.9907884238775658,
314
+ "support": 6927.0
315
+ },
316
+ "government_legal": {
317
+ "precision": 0.9880974521108424,
318
+ "recall": 0.9906768599664367,
319
+ "f1-score": 0.9893854748603352,
320
+ "support": 5363.0
321
+ },
322
+ "healthcare_medical": {
323
+ "precision": 0.9832678845825508,
324
+ "recall": 0.9777589134125637,
325
+ "f1-score": 0.9805056610198348,
326
+ "support": 5890.0
327
+ },
328
+ "income": {
329
+ "precision": 0.9926499032882011,
330
+ "recall": 0.9963113958454669,
331
+ "f1-score": 0.9944772793333979,
332
+ "support": 5151.0
333
+ },
334
+ "shopping_retail": {
335
+ "precision": 0.993641494605987,
336
+ "recall": 0.9818566890222379,
337
+ "f1-score": 0.9877139407712521,
338
+ "support": 14165.0
339
+ },
340
+ "transportation": {
341
+ "precision": 0.996781272860278,
342
+ "recall": 0.9821248378261497,
343
+ "f1-score": 0.9893987801336044,
344
+ "support": 6937.0
345
+ },
346
+ "utilities_services": {
347
+ "precision": 0.9951179820992677,
348
+ "recall": 0.9818561335902376,
349
+ "f1-score": 0.9884425765780328,
350
+ "support": 6228.0
351
+ },
352
+ "accuracy": 0.9885360423757005,
353
+ "macro avg": {
354
+ "precision": 0.991119391007086,
355
+ "recall": 0.9881880902973459,
356
+ "f1-score": 0.9896056616563857,
357
+ "support": 78158.0
358
+ },
359
+ "weighted avg": {
360
+ "precision": 0.9887186751472307,
361
+ "recall": 0.9885360423757005,
362
+ "f1-score": 0.988553516466768,
363
+ "support": 78158.0
364
+ }
365
+ },
366
+ "confusion_matrix": [
367
+ [
368
+ 5227,
369
+ 0,
370
+ 0,
371
+ 0,
372
+ 0,
373
+ 0,
374
+ 0,
375
+ 0,
376
+ 0,
377
+ 0
378
+ ],
379
+ [
380
+ 0,
381
+ 6894,
382
+ 96,
383
+ 2,
384
+ 1,
385
+ 0,
386
+ 0,
387
+ 2,
388
+ 0,
389
+ 0
390
+ ],
391
+ [
392
+ 0,
393
+ 0,
394
+ 15271,
395
+ 1,
396
+ 0,
397
+ 0,
398
+ 1,
399
+ 2,
400
+ 0,
401
+ 0
402
+ ],
403
+ [
404
+ 0,
405
+ 5,
406
+ 76,
407
+ 6830,
408
+ 0,
409
+ 0,
410
+ 0,
411
+ 15,
412
+ 1,
413
+ 0
414
+ ],
415
+ [
416
+ 0,
417
+ 0,
418
+ 0,
419
+ 1,
420
+ 5313,
421
+ 0,
422
+ 37,
423
+ 11,
424
+ 1,
425
+ 0
426
+ ],
427
+ [
428
+ 0,
429
+ 0,
430
+ 72,
431
+ 0,
432
+ 0,
433
+ 5759,
434
+ 0,
435
+ 59,
436
+ 0,
437
+ 0
438
+ ],
439
+ [
440
+ 0,
441
+ 0,
442
+ 0,
443
+ 0,
444
+ 19,
445
+ 0,
446
+ 5132,
447
+ 0,
448
+ 0,
449
+ 0
450
+ ],
451
+ [
452
+ 0,
453
+ 1,
454
+ 85,
455
+ 26,
456
+ 44,
457
+ 98,
458
+ 0,
459
+ 13908,
460
+ 0,
461
+ 3
462
+ ],
463
+ [
464
+ 0,
465
+ 1,
466
+ 96,
467
+ 0,
468
+ 0,
469
+ 0,
470
+ 0,
471
+ 0,
472
+ 6813,
473
+ 27
474
+ ],
475
+ [
476
+ 0,
477
+ 2,
478
+ 91,
479
+ 0,
480
+ 0,
481
+ 0,
482
+ 0,
483
+ 0,
484
+ 20,
485
+ 6115
486
+ ]
487
+ ],
488
+ "samples": 78158,
489
+ "confidence_threshold": 0.7,
490
+ "confidence_coverage": 0.9713273113436884,
491
+ "high_confidence_accuracy": 0.9996706929936642
492
+ },
493
+ "country_test_metrics": {
494
+ "Australia": {
495
+ "model_name": "tfidf-test-Australia",
496
+ "accuracy": 0.9949353448275862,
497
+ "macro_f1": 0.9948911920454151,
498
+ "weighted_f1": 0.9949260470529634,
499
+ "report": {
500
+ "charity_donations": {
501
+ "precision": 1.0,
502
+ "recall": 1.0,
503
+ "f1-score": 1.0,
504
+ "support": 4607.0
505
+ },
506
+ "entertainment_recreation": {
507
+ "precision": 1.0,
508
+ "recall": 1.0,
509
+ "f1-score": 1.0,
510
+ "support": 4776.0
511
+ },
512
+ "financial_services": {
513
+ "precision": 1.0,
514
+ "recall": 1.0,
515
+ "f1-score": 1.0,
516
+ "support": 4651.0
517
+ },
518
+ "food_dining": {
519
+ "precision": 0.9959201202490874,
520
+ "recall": 0.9984930032292788,
521
+ "f1-score": 0.997204902171576,
522
+ "support": 4645.0
523
+ },
524
+ "government_legal": {
525
+ "precision": 0.9893985287754219,
526
+ "recall": 0.9952121871599565,
527
+ "f1-score": 0.9922968427904958,
528
+ "support": 4595.0
529
+ },
530
+ "healthcare_medical": {
531
+ "precision": 0.9845045831514623,
532
+ "recall": 0.9920826918847592,
533
+ "f1-score": 0.9882791105268923,
534
+ "support": 4547.0
535
+ },
536
+ "income": {
537
+ "precision": 0.995475113122172,
538
+ "recall": 0.9974093264248705,
539
+ "f1-score": 0.99644128113879,
540
+ "support": 4632.0
541
+ },
542
+ "shopping_retail": {
543
+ "precision": 0.9903105042942083,
544
+ "recall": 0.9725346020761245,
545
+ "f1-score": 0.981342062193126,
546
+ "support": 4624.0
547
+ },
548
+ "transportation": {
549
+ "precision": 0.9985216473072862,
550
+ "recall": 0.9949494949494949,
551
+ "f1-score": 0.996732370612417,
552
+ "support": 4752.0
553
+ },
554
+ "utilities_services": {
555
+ "precision": 0.994768962510898,
556
+ "recall": 0.998468606431853,
557
+ "f1-score": 0.9966153510208539,
558
+ "support": 4571.0
559
+ },
560
+ "accuracy": 0.9949353448275862,
561
+ "macro avg": {
562
+ "precision": 0.9948899459410535,
563
+ "recall": 0.9949149912156339,
564
+ "f1-score": 0.9948911920454151,
565
+ "support": 46400.0
566
+ },
567
+ "weighted avg": {
568
+ "precision": 0.9949391773566729,
569
+ "recall": 0.9949353448275862,
570
+ "f1-score": 0.9949260470529634,
571
+ "support": 46400.0
572
+ }
573
+ },
574
+ "confusion_matrix": [
575
+ [
576
+ 4607,
577
+ 0,
578
+ 0,
579
+ 0,
580
+ 0,
581
+ 0,
582
+ 0,
583
+ 0,
584
+ 0,
585
+ 0
586
+ ],
587
+ [
588
+ 0,
589
+ 4776,
590
+ 0,
591
+ 0,
592
+ 0,
593
+ 0,
594
+ 0,
595
+ 0,
596
+ 0,
597
+ 0
598
+ ],
599
+ [
600
+ 0,
601
+ 0,
602
+ 4651,
603
+ 0,
604
+ 0,
605
+ 0,
606
+ 0,
607
+ 0,
608
+ 0,
609
+ 0
610
+ ],
611
+ [
612
+ 0,
613
+ 0,
614
+ 0,
615
+ 4638,
616
+ 0,
617
+ 0,
618
+ 0,
619
+ 7,
620
+ 0,
621
+ 0
622
+ ],
623
+ [
624
+ 0,
625
+ 0,
626
+ 0,
627
+ 0,
628
+ 4573,
629
+ 0,
630
+ 21,
631
+ 1,
632
+ 0,
633
+ 0
634
+ ],
635
+ [
636
+ 0,
637
+ 0,
638
+ 0,
639
+ 0,
640
+ 0,
641
+ 4511,
642
+ 0,
643
+ 36,
644
+ 0,
645
+ 0
646
+ ],
647
+ [
648
+ 0,
649
+ 0,
650
+ 0,
651
+ 0,
652
+ 12,
653
+ 0,
654
+ 4620,
655
+ 0,
656
+ 0,
657
+ 0
658
+ ],
659
+ [
660
+ 0,
661
+ 0,
662
+ 0,
663
+ 19,
664
+ 37,
665
+ 71,
666
+ 0,
667
+ 4497,
668
+ 0,
669
+ 0
670
+ ],
671
+ [
672
+ 0,
673
+ 0,
674
+ 0,
675
+ 0,
676
+ 0,
677
+ 0,
678
+ 0,
679
+ 0,
680
+ 4728,
681
+ 24
682
+ ],
683
+ [
684
+ 0,
685
+ 0,
686
+ 0,
687
+ 0,
688
+ 0,
689
+ 0,
690
+ 0,
691
+ 0,
692
+ 7,
693
+ 4564
694
+ ]
695
+ ],
696
+ "samples": 46400,
697
+ "confidence_threshold": 0.7,
698
+ "confidence_coverage": 0.983728448275862,
699
+ "high_confidence_accuracy": 0.9995180194983021
700
+ },
701
+ "Canada": {
702
+ "model_name": "tfidf-test-Canada",
703
+ "accuracy": 0.9936672051696285,
704
+ "macro_f1": 0.9936446459609755,
705
+ "weighted_f1": 0.9936629177325705,
706
+ "report": {
707
+ "charity_donations": {
708
+ "precision": 1.0,
709
+ "recall": 1.0,
710
+ "f1-score": 1.0,
711
+ "support": 4663.0
712
+ },
713
+ "entertainment_recreation": {
714
+ "precision": 1.0,
715
+ "recall": 1.0,
716
+ "f1-score": 1.0,
717
+ "support": 4606.0
718
+ },
719
+ "financial_services": {
720
+ "precision": 1.0,
721
+ "recall": 1.0,
722
+ "f1-score": 1.0,
723
+ "support": 4634.0
724
+ },
725
+ "food_dining": {
726
+ "precision": 0.9976866456361725,
727
+ "recall": 0.9953839697859841,
728
+ "f1-score": 0.9965339775233694,
729
+ "support": 4766.0
730
+ },
731
+ "government_legal": {
732
+ "precision": 0.9914921465968587,
733
+ "recall": 0.9934426229508196,
734
+ "f1-score": 0.9924664264657713,
735
+ "support": 4575.0
736
+ },
737
+ "healthcare_medical": {
738
+ "precision": 0.9790239726027398,
739
+ "recall": 0.9872652708827974,
740
+ "f1-score": 0.9831273508866201,
741
+ "support": 4633.0
742
+ },
743
+ "income": {
744
+ "precision": 0.9952090592334495,
745
+ "recall": 0.9971634300676413,
746
+ "f1-score": 0.9961852861035423,
747
+ "support": 4583.0
748
+ },
749
+ "shopping_retail": {
750
+ "precision": 0.9805336832895888,
751
+ "recall": 0.9707665656128194,
752
+ "f1-score": 0.9756256800870512,
753
+ "support": 4618.0
754
+ },
755
+ "transportation": {
756
+ "precision": 0.9970631424375918,
757
+ "recall": 0.9956011730205279,
758
+ "f1-score": 0.9963316214233309,
759
+ "support": 4774.0
760
+ },
761
+ "utilities_services": {
762
+ "precision": 0.995414847161572,
763
+ "recall": 0.996938552372622,
764
+ "f1-score": 0.9961761171200699,
765
+ "support": 4573.0
766
+ },
767
+ "accuracy": 0.9936672051696285,
768
+ "macro avg": {
769
+ "precision": 0.9936423496957971,
770
+ "recall": 0.9936561584693212,
771
+ "f1-score": 0.9936446459609755,
772
+ "support": 46425.0
773
+ },
774
+ "weighted avg": {
775
+ "precision": 0.9936678167973348,
776
+ "recall": 0.9936672051696285,
777
+ "f1-score": 0.9936629177325705,
778
+ "support": 46425.0
779
+ }
780
+ },
781
+ "confusion_matrix": [
782
+ [
783
+ 4663,
784
+ 0,
785
+ 0,
786
+ 0,
787
+ 0,
788
+ 0,
789
+ 0,
790
+ 0,
791
+ 0,
792
+ 0
793
+ ],
794
+ [
795
+ 0,
796
+ 4606,
797
+ 0,
798
+ 0,
799
+ 0,
800
+ 0,
801
+ 0,
802
+ 0,
803
+ 0,
804
+ 0
805
+ ],
806
+ [
807
+ 0,
808
+ 0,
809
+ 4634,
810
+ 0,
811
+ 0,
812
+ 0,
813
+ 0,
814
+ 0,
815
+ 0,
816
+ 0
817
+ ],
818
+ [
819
+ 0,
820
+ 0,
821
+ 0,
822
+ 4744,
823
+ 0,
824
+ 0,
825
+ 0,
826
+ 22,
827
+ 0,
828
+ 0
829
+ ],
830
+ [
831
+ 0,
832
+ 0,
833
+ 0,
834
+ 0,
835
+ 4545,
836
+ 0,
837
+ 22,
838
+ 8,
839
+ 0,
840
+ 0
841
+ ],
842
+ [
843
+ 0,
844
+ 0,
845
+ 0,
846
+ 0,
847
+ 0,
848
+ 4574,
849
+ 0,
850
+ 59,
851
+ 0,
852
+ 0
853
+ ],
854
+ [
855
+ 0,
856
+ 0,
857
+ 0,
858
+ 0,
859
+ 13,
860
+ 0,
861
+ 4570,
862
+ 0,
863
+ 0,
864
+ 0
865
+ ],
866
+ [
867
+ 0,
868
+ 0,
869
+ 0,
870
+ 11,
871
+ 26,
872
+ 98,
873
+ 0,
874
+ 4483,
875
+ 0,
876
+ 0
877
+ ],
878
+ [
879
+ 0,
880
+ 0,
881
+ 0,
882
+ 0,
883
+ 0,
884
+ 0,
885
+ 0,
886
+ 0,
887
+ 4753,
888
+ 21
889
+ ],
890
+ [
891
+ 0,
892
+ 0,
893
+ 0,
894
+ 0,
895
+ 0,
896
+ 0,
897
+ 0,
898
+ 0,
899
+ 14,
900
+ 4559
901
+ ]
902
+ ],
903
+ "samples": 46425,
904
+ "confidence_threshold": 0.7,
905
+ "confidence_coverage": 0.9820140010770059,
906
+ "high_confidence_accuracy": 0.9996051765738101
907
+ },
908
+ "India": {
909
+ "model_name": "tfidf-test-India",
910
+ "accuracy": 0.9891758415179814,
911
+ "macro_f1": 0.9903764138960277,
912
+ "weighted_f1": 0.9891960532390336,
913
+ "report": {
914
+ "charity_donations": {
915
+ "precision": 1.0,
916
+ "recall": 1.0,
917
+ "f1-score": 1.0,
918
+ "support": 3743.0
919
+ },
920
+ "entertainment_recreation": {
921
+ "precision": 0.9987646695491044,
922
+ "recall": 0.990404246631278,
923
+ "f1-score": 0.9945668887749872,
924
+ "support": 4898.0
925
+ },
926
+ "financial_services": {
927
+ "precision": 0.9661000294782353,
928
+ "recall": 0.9995933306222041,
929
+ "f1-score": 0.9825613351321641,
930
+ "support": 9836.0
931
+ },
932
+ "food_dining": {
933
+ "precision": 0.9954788327168105,
934
+ "recall": 0.9799716771191584,
935
+ "f1-score": 0.9876643898460598,
936
+ "support": 4943.0
937
+ },
938
+ "government_legal": {
939
+ "precision": 0.9899794450154162,
940
+ "recall": 0.9912528942629277,
941
+ "f1-score": 0.9906157603805116,
942
+ "support": 3887.0
943
+ },
944
+ "healthcare_medical": {
945
+ "precision": 0.9856581077955123,
946
+ "recall": 0.9820235077206729,
947
+ "f1-score": 0.9838374509351189,
948
+ "support": 4339.0
949
+ },
950
+ "income": {
951
+ "precision": 0.993570854540584,
952
+ "recall": 0.9970430107526882,
953
+ "f1-score": 0.9953039044679994,
954
+ "support": 3720.0
955
+ },
956
+ "shopping_retail": {
957
+ "precision": 0.9941539819104346,
958
+ "recall": 0.9815944238727946,
959
+ "f1-score": 0.9878342832091188,
960
+ "support": 9182.0
961
+ },
962
+ "transportation": {
963
+ "precision": 0.9970918155380141,
964
+ "recall": 0.9830022527134958,
965
+ "f1-score": 0.989996906259668,
966
+ "support": 4883.0
967
+ },
968
+ "utilities_services": {
969
+ "precision": 0.9958997722095672,
970
+ "recall": 0.9869074492099322,
971
+ "f1-score": 0.9913832199546485,
972
+ "support": 4430.0
973
+ },
974
+ "accuracy": 0.9891758415179814,
975
+ "macro avg": {
976
+ "precision": 0.9916697508753678,
977
+ "recall": 0.9891792792905152,
978
+ "f1-score": 0.9903764138960277,
979
+ "support": 53861.0
980
+ },
981
+ "weighted avg": {
982
+ "precision": 0.9893619225593913,
983
+ "recall": 0.9891758415179814,
984
+ "f1-score": 0.9891960532390336,
985
+ "support": 53861.0
986
+ }
987
+ },
988
+ "confusion_matrix": [
989
+ [
990
+ 3743,
991
+ 0,
992
+ 0,
993
+ 0,
994
+ 0,
995
+ 0,
996
+ 0,
997
+ 0,
998
+ 0,
999
+ 0
1000
+ ],
1001
+ [
1002
+ 0,
1003
+ 4851,
1004
+ 44,
1005
+ 1,
1006
+ 0,
1007
+ 1,
1008
+ 0,
1009
+ 1,
1010
+ 0,
1011
+ 0
1012
+ ],
1013
+ [
1014
+ 0,
1015
+ 1,
1016
+ 9832,
1017
+ 1,
1018
+ 0,
1019
+ 0,
1020
+ 2,
1021
+ 0,
1022
+ 0,
1023
+ 0
1024
+ ],
1025
+ [
1026
+ 0,
1027
+ 2,
1028
+ 82,
1029
+ 4844,
1030
+ 0,
1031
+ 0,
1032
+ 0,
1033
+ 14,
1034
+ 1,
1035
+ 0
1036
+ ],
1037
+ [
1038
+ 0,
1039
+ 1,
1040
+ 2,
1041
+ 1,
1042
+ 3853,
1043
+ 0,
1044
+ 22,
1045
+ 7,
1046
+ 1,
1047
+ 0
1048
+ ],
1049
+ [
1050
+ 0,
1051
+ 0,
1052
+ 47,
1053
+ 0,
1054
+ 0,
1055
+ 4261,
1056
+ 0,
1057
+ 31,
1058
+ 0,
1059
+ 0
1060
+ ],
1061
+ [
1062
+ 0,
1063
+ 0,
1064
+ 0,
1065
+ 0,
1066
+ 11,
1067
+ 0,
1068
+ 3709,
1069
+ 0,
1070
+ 0,
1071
+ 0
1072
+ ],
1073
+ [
1074
+ 0,
1075
+ 0,
1076
+ 60,
1077
+ 19,
1078
+ 28,
1079
+ 61,
1080
+ 0,
1081
+ 9013,
1082
+ 0,
1083
+ 1
1084
+ ],
1085
+ [
1086
+ 0,
1087
+ 1,
1088
+ 65,
1089
+ 0,
1090
+ 0,
1091
+ 0,
1092
+ 0,
1093
+ 0,
1094
+ 4800,
1095
+ 17
1096
+ ],
1097
+ [
1098
+ 0,
1099
+ 1,
1100
+ 45,
1101
+ 0,
1102
+ 0,
1103
+ 0,
1104
+ 0,
1105
+ 0,
1106
+ 12,
1107
+ 4372
1108
+ ]
1109
+ ],
1110
+ "samples": 53861,
1111
+ "confidence_threshold": 0.7,
1112
+ "confidence_coverage": 0.9753810735040196,
1113
+ "high_confidence_accuracy": 0.9996193014181022
1114
+ },
1115
+ "UK": {
1116
+ "model_name": "tfidf-test-UK",
1117
+ "accuracy": 0.9940697128717095,
1118
+ "macro_f1": 0.9940612354515542,
1119
+ "weighted_f1": 0.9940660583800917,
1120
+ "report": {
1121
+ "charity_donations": {
1122
+ "precision": 1.0,
1123
+ "recall": 1.0,
1124
+ "f1-score": 1.0,
1125
+ "support": 4560.0
1126
+ },
1127
+ "entertainment_recreation": {
1128
+ "precision": 1.0,
1129
+ "recall": 1.0,
1130
+ "f1-score": 1.0,
1131
+ "support": 4662.0
1132
+ },
1133
+ "financial_services": {
1134
+ "precision": 1.0,
1135
+ "recall": 1.0,
1136
+ "f1-score": 1.0,
1137
+ "support": 4741.0
1138
+ },
1139
+ "food_dining": {
1140
+ "precision": 0.9978840457046128,
1141
+ "recall": 0.995987328405491,
1142
+ "f1-score": 0.996934784906458,
1143
+ "support": 4735.0
1144
+ },
1145
+ "government_legal": {
1146
+ "precision": 0.9921161825726141,
1147
+ "recall": 0.9943855271366189,
1148
+ "f1-score": 0.993249558624987,
1149
+ "support": 4809.0
1150
+ },
1151
+ "healthcare_medical": {
1152
+ "precision": 0.980806142034549,
1153
+ "recall": 0.988819608686304,
1154
+ "f1-score": 0.984796573875803,
1155
+ "support": 4651.0
1156
+ },
1157
+ "income": {
1158
+ "precision": 0.9959393032699295,
1159
+ "recall": 0.9970047068891742,
1160
+ "f1-score": 0.9964717203036458,
1161
+ "support": 4674.0
1162
+ },
1163
+ "shopping_retail": {
1164
+ "precision": 0.9828781967923711,
1165
+ "recall": 0.9733848465335909,
1166
+ "f1-score": 0.9781084870052842,
1167
+ "support": 4659.0
1168
+ },
1169
+ "transportation": {
1170
+ "precision": 0.9966044142614601,
1171
+ "recall": 0.9944938585345193,
1172
+ "f1-score": 0.9955480178079288,
1173
+ "support": 4722.0
1174
+ },
1175
+ "utilities_services": {
1176
+ "precision": 0.9944385026737967,
1177
+ "recall": 0.9965702036441586,
1178
+ "f1-score": 0.9955032119914347,
1179
+ "support": 4665.0
1180
+ },
1181
+ "accuracy": 0.9940697128717095,
1182
+ "macro avg": {
1183
+ "precision": 0.9940666787309332,
1184
+ "recall": 0.9940646079829858,
1185
+ "f1-score": 0.9940612354515542,
1186
+ "support": 46878.0
1187
+ },
1188
+ "weighted avg": {
1189
+ "precision": 0.994071174675987,
1190
+ "recall": 0.9940697128717095,
1191
+ "f1-score": 0.9940660583800917,
1192
+ "support": 46878.0
1193
+ }
1194
+ },
1195
+ "confusion_matrix": [
1196
+ [
1197
+ 4560,
1198
+ 0,
1199
+ 0,
1200
+ 0,
1201
+ 0,
1202
+ 0,
1203
+ 0,
1204
+ 0,
1205
+ 0,
1206
+ 0
1207
+ ],
1208
+ [
1209
+ 0,
1210
+ 4662,
1211
+ 0,
1212
+ 0,
1213
+ 0,
1214
+ 0,
1215
+ 0,
1216
+ 0,
1217
+ 0,
1218
+ 0
1219
+ ],
1220
+ [
1221
+ 0,
1222
+ 0,
1223
+ 4741,
1224
+ 0,
1225
+ 0,
1226
+ 0,
1227
+ 0,
1228
+ 0,
1229
+ 0,
1230
+ 0
1231
+ ],
1232
+ [
1233
+ 0,
1234
+ 0,
1235
+ 0,
1236
+ 4716,
1237
+ 0,
1238
+ 0,
1239
+ 0,
1240
+ 19,
1241
+ 0,
1242
+ 0
1243
+ ],
1244
+ [
1245
+ 0,
1246
+ 0,
1247
+ 0,
1248
+ 0,
1249
+ 4782,
1250
+ 0,
1251
+ 19,
1252
+ 8,
1253
+ 0,
1254
+ 0
1255
+ ],
1256
+ [
1257
+ 0,
1258
+ 0,
1259
+ 0,
1260
+ 0,
1261
+ 0,
1262
+ 4599,
1263
+ 0,
1264
+ 52,
1265
+ 0,
1266
+ 0
1267
+ ],
1268
+ [
1269
+ 0,
1270
+ 0,
1271
+ 0,
1272
+ 0,
1273
+ 14,
1274
+ 0,
1275
+ 4660,
1276
+ 0,
1277
+ 0,
1278
+ 0
1279
+ ],
1280
+ [
1281
+ 0,
1282
+ 0,
1283
+ 0,
1284
+ 10,
1285
+ 24,
1286
+ 90,
1287
+ 0,
1288
+ 4535,
1289
+ 0,
1290
+ 0
1291
+ ],
1292
+ [
1293
+ 0,
1294
+ 0,
1295
+ 0,
1296
+ 0,
1297
+ 0,
1298
+ 0,
1299
+ 0,
1300
+ 0,
1301
+ 4696,
1302
+ 26
1303
+ ],
1304
+ [
1305
+ 0,
1306
+ 0,
1307
+ 0,
1308
+ 0,
1309
+ 0,
1310
+ 0,
1311
+ 0,
1312
+ 0,
1313
+ 16,
1314
+ 4649
1315
+ ]
1316
+ ],
1317
+ "samples": 46878,
1318
+ "confidence_threshold": 0.7,
1319
+ "confidence_coverage": 0.9814838517001578,
1320
+ "high_confidence_accuracy": 0.9995218430775918
1321
+ },
1322
+ "USA": {
1323
+ "model_name": "tfidf-test-USA",
1324
+ "accuracy": 0.9936759233367033,
1325
+ "macro_f1": 0.9936282588322631,
1326
+ "weighted_f1": 0.9936595529700609,
1327
+ "report": {
1328
+ "charity_donations": {
1329
+ "precision": 0.999781325169473,
1330
+ "recall": 1.0,
1331
+ "f1-score": 0.9998906506287589,
1332
+ "support": 4572.0
1333
+ },
1334
+ "entertainment_recreation": {
1335
+ "precision": 1.0,
1336
+ "recall": 1.0,
1337
+ "f1-score": 1.0,
1338
+ "support": 4776.0
1339
+ },
1340
+ "financial_services": {
1341
+ "precision": 1.0,
1342
+ "recall": 0.9997854997854998,
1343
+ "f1-score": 0.9998927383889306,
1344
+ "support": 4662.0
1345
+ },
1346
+ "food_dining": {
1347
+ "precision": 0.9934140641597621,
1348
+ "recall": 0.9965899403239556,
1349
+ "f1-score": 0.9949994680285137,
1350
+ "support": 4692.0
1351
+ },
1352
+ "government_legal": {
1353
+ "precision": 0.9905956112852664,
1354
+ "recall": 0.9941275167785235,
1355
+ "f1-score": 0.9923584214382917,
1356
+ "support": 4768.0
1357
+ },
1358
+ "healthcare_medical": {
1359
+ "precision": 0.9775060056780956,
1360
+ "recall": 0.9907038512616202,
1361
+ "f1-score": 0.984060679344839,
1362
+ "support": 4518.0
1363
+ },
1364
+ "income": {
1365
+ "precision": 0.9957541899441341,
1366
+ "recall": 0.9986553115194979,
1367
+ "f1-score": 0.9972026407071725,
1368
+ "support": 4462.0
1369
+ },
1370
+ "shopping_retail": {
1371
+ "precision": 0.9852227613586237,
1372
+ "recall": 0.9629230437594309,
1373
+ "f1-score": 0.973945274174207,
1374
+ "support": 4639.0
1375
+ },
1376
+ "transportation": {
1377
+ "precision": 0.9987408184679958,
1378
+ "recall": 0.9953984522066514,
1379
+ "f1-score": 0.9970668342761366,
1380
+ "support": 4781.0
1381
+ },
1382
+ "utilities_services": {
1383
+ "precision": 0.9952524816573155,
1384
+ "recall": 0.9984845204589738,
1385
+ "f1-score": 0.996865881335783,
1386
+ "support": 4619.0
1387
+ },
1388
+ "accuracy": 0.9936759233367033,
1389
+ "macro avg": {
1390
+ "precision": 0.9936267257720667,
1391
+ "recall": 0.9936668136094152,
1392
+ "f1-score": 0.9936282588322631,
1393
+ "support": 46489.0
1394
+ },
1395
+ "weighted avg": {
1396
+ "precision": 0.9936799171270243,
1397
+ "recall": 0.9936759233367033,
1398
+ "f1-score": 0.9936595529700609,
1399
+ "support": 46489.0
1400
+ }
1401
+ },
1402
+ "confusion_matrix": [
1403
+ [
1404
+ 4572,
1405
+ 0,
1406
+ 0,
1407
+ 0,
1408
+ 0,
1409
+ 0,
1410
+ 0,
1411
+ 0,
1412
+ 0,
1413
+ 0
1414
+ ],
1415
+ [
1416
+ 0,
1417
+ 4776,
1418
+ 0,
1419
+ 0,
1420
+ 0,
1421
+ 0,
1422
+ 0,
1423
+ 0,
1424
+ 0,
1425
+ 0
1426
+ ],
1427
+ [
1428
+ 0,
1429
+ 0,
1430
+ 4661,
1431
+ 0,
1432
+ 0,
1433
+ 0,
1434
+ 1,
1435
+ 0,
1436
+ 0,
1437
+ 0
1438
+ ],
1439
+ [
1440
+ 1,
1441
+ 0,
1442
+ 0,
1443
+ 4676,
1444
+ 0,
1445
+ 0,
1446
+ 0,
1447
+ 15,
1448
+ 0,
1449
+ 0
1450
+ ],
1451
+ [
1452
+ 0,
1453
+ 0,
1454
+ 0,
1455
+ 0,
1456
+ 4740,
1457
+ 0,
1458
+ 18,
1459
+ 10,
1460
+ 0,
1461
+ 0
1462
+ ],
1463
+ [
1464
+ 0,
1465
+ 0,
1466
+ 0,
1467
+ 0,
1468
+ 0,
1469
+ 4476,
1470
+ 0,
1471
+ 42,
1472
+ 0,
1473
+ 0
1474
+ ],
1475
+ [
1476
+ 0,
1477
+ 0,
1478
+ 0,
1479
+ 0,
1480
+ 6,
1481
+ 0,
1482
+ 4456,
1483
+ 0,
1484
+ 0,
1485
+ 0
1486
+ ],
1487
+ [
1488
+ 0,
1489
+ 0,
1490
+ 0,
1491
+ 30,
1492
+ 39,
1493
+ 103,
1494
+ 0,
1495
+ 4467,
1496
+ 0,
1497
+ 0
1498
+ ],
1499
+ [
1500
+ 0,
1501
+ 0,
1502
+ 0,
1503
+ 0,
1504
+ 0,
1505
+ 0,
1506
+ 0,
1507
+ 0,
1508
+ 4759,
1509
+ 22
1510
+ ],
1511
+ [
1512
+ 0,
1513
+ 0,
1514
+ 0,
1515
+ 1,
1516
+ 0,
1517
+ 0,
1518
+ 0,
1519
+ 0,
1520
+ 6,
1521
+ 4612
1522
+ ]
1523
+ ],
1524
+ "samples": 46489,
1525
+ "confidence_threshold": 0.7,
1526
+ "confidence_coverage": 0.9823399083654197,
1527
+ "high_confidence_accuracy": 0.999496365069633
1528
+ }
1529
+ }
1530
+ }
metadata.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "tfidf_logistic_regression",
3
+ "labels": [
4
+ "charity_donations",
5
+ "entertainment_recreation",
6
+ "financial_services",
7
+ "food_dining",
8
+ "government_legal",
9
+ "healthcare_medical",
10
+ "income",
11
+ "shopping_retail",
12
+ "transportation",
13
+ "utilities_services"
14
+ ],
15
+ "word_features": "word 1-2 grams, max_features=120000",
16
+ "character_features": "char_wb 3-5 grams, max_features=80000",
17
+ "solver": "saga"
18
+ }
model.joblib ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:656fa0f711c746a24c9880d0959df12d99fad1f80d6b2747c0790f049745ef45
3
+ size 23522330
models/category-tfidf/metadata.json ADDED
@@ -0,0 +1,18 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_type": "tfidf_logistic_regression",
3
+ "labels": [
4
+ "charity_donations",
5
+ "entertainment_recreation",
6
+ "financial_services",
7
+ "food_dining",
8
+ "government_legal",
9
+ "healthcare_medical",
10
+ "income",
11
+ "shopping_retail",
12
+ "transportation",
13
+ "utilities_services"
14
+ ],
15
+ "word_features": "word 1-2 grams, max_features=120000",
16
+ "character_features": "char_wb 3-5 grams, max_features=80000",
17
+ "solver": "saga"
18
+ }
models/category-tfidf/model.joblib ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:656fa0f711c746a24c9880d0959df12d99fad1f80d6b2747c0790f049745ef45
3
+ size 23522330
models/duplicate-similarity/model.joblib ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:961b407bc2260f121038c17daed6828256a677518d1f2ba53cd75188ec8b734c
3
+ size 451930366
models/merchant-similarity/model.joblib ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:88f0ceff3a4f03123c5375e6c6cc1baa23159cf73a4f9758583b2e5f91614f36
3
+ size 137789609
reports/REPORT.md ADDED
@@ -0,0 +1,21 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Expense Tracker ML Training Report — 20260918T064102Z
2
+
3
+ - Pipeline version: `2.0.0`
4
+ - Seed: `42`
5
+ - Python: `3.14.5 (main, May 10 2026, 19:28:16) [Clang 22.1.3 ]`
6
+
7
+ ## Model Artifacts
8
+
9
+ | Model | Status | Artifact | Notes |
10
+ |---|---|---|---|
11
+ | category_tfidf | trained | models/category-tfidf | |
12
+ | category_transformer | disabled | | |
13
+ | merchant_similarity | trained | models/merchant-similarity | |
14
+ | duplicate_similarity | trained | models/duplicate-similarity | |
15
+ | transaction_anomaly | not_applicable | | No amount feature is present in the configured category datasets. |
16
+ | spending_forecast | not_applicable | | Category datasets do not provide transaction date and amount fields. |
17
+
18
+ ## Outputs
19
+
20
+ Machine-readable metrics are stored as JSON under `reports/`.
21
+ High-resolution figures are stored under `reports/figures/` as PNG files.
reports/category_india_holdout.json ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "tfidf-india-holdout",
3
+ "accuracy": 0.9885360423757005,
4
+ "macro_f1": 0.9896056616563857,
5
+ "weighted_f1": 0.988553516466768,
6
+ "report": {
7
+ "charity_donations": {
8
+ "precision": 1.0,
9
+ "recall": 1.0,
10
+ "f1-score": 1.0,
11
+ "support": 5227.0
12
+ },
13
+ "entertainment_recreation": {
14
+ "precision": 0.9986962190352021,
15
+ "recall": 0.9855611150822016,
16
+ "f1-score": 0.9920851921139733,
17
+ "support": 6995.0
18
+ },
19
+ "financial_services": {
20
+ "precision": 0.9673148793310952,
21
+ "recall": 0.9997381342062193,
22
+ "f1-score": 0.9832592878758611,
23
+ "support": 15275.0
24
+ },
25
+ "food_dining": {
26
+ "precision": 0.9956268221574344,
27
+ "recall": 0.9859968240219431,
28
+ "f1-score": 0.9907884238775658,
29
+ "support": 6927.0
30
+ },
31
+ "government_legal": {
32
+ "precision": 0.9880974521108424,
33
+ "recall": 0.9906768599664367,
34
+ "f1-score": 0.9893854748603352,
35
+ "support": 5363.0
36
+ },
37
+ "healthcare_medical": {
38
+ "precision": 0.9832678845825508,
39
+ "recall": 0.9777589134125637,
40
+ "f1-score": 0.9805056610198348,
41
+ "support": 5890.0
42
+ },
43
+ "income": {
44
+ "precision": 0.9926499032882011,
45
+ "recall": 0.9963113958454669,
46
+ "f1-score": 0.9944772793333979,
47
+ "support": 5151.0
48
+ },
49
+ "shopping_retail": {
50
+ "precision": 0.993641494605987,
51
+ "recall": 0.9818566890222379,
52
+ "f1-score": 0.9877139407712521,
53
+ "support": 14165.0
54
+ },
55
+ "transportation": {
56
+ "precision": 0.996781272860278,
57
+ "recall": 0.9821248378261497,
58
+ "f1-score": 0.9893987801336044,
59
+ "support": 6937.0
60
+ },
61
+ "utilities_services": {
62
+ "precision": 0.9951179820992677,
63
+ "recall": 0.9818561335902376,
64
+ "f1-score": 0.9884425765780328,
65
+ "support": 6228.0
66
+ },
67
+ "accuracy": 0.9885360423757005,
68
+ "macro avg": {
69
+ "precision": 0.991119391007086,
70
+ "recall": 0.9881880902973459,
71
+ "f1-score": 0.9896056616563857,
72
+ "support": 78158.0
73
+ },
74
+ "weighted avg": {
75
+ "precision": 0.9887186751472307,
76
+ "recall": 0.9885360423757005,
77
+ "f1-score": 0.988553516466768,
78
+ "support": 78158.0
79
+ }
80
+ },
81
+ "confusion_matrix": [
82
+ [
83
+ 5227,
84
+ 0,
85
+ 0,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0
93
+ ],
94
+ [
95
+ 0,
96
+ 6894,
97
+ 96,
98
+ 2,
99
+ 1,
100
+ 0,
101
+ 0,
102
+ 2,
103
+ 0,
104
+ 0
105
+ ],
106
+ [
107
+ 0,
108
+ 0,
109
+ 15271,
110
+ 1,
111
+ 0,
112
+ 0,
113
+ 1,
114
+ 2,
115
+ 0,
116
+ 0
117
+ ],
118
+ [
119
+ 0,
120
+ 5,
121
+ 76,
122
+ 6830,
123
+ 0,
124
+ 0,
125
+ 0,
126
+ 15,
127
+ 1,
128
+ 0
129
+ ],
130
+ [
131
+ 0,
132
+ 0,
133
+ 0,
134
+ 1,
135
+ 5313,
136
+ 0,
137
+ 37,
138
+ 11,
139
+ 1,
140
+ 0
141
+ ],
142
+ [
143
+ 0,
144
+ 0,
145
+ 72,
146
+ 0,
147
+ 0,
148
+ 5759,
149
+ 0,
150
+ 59,
151
+ 0,
152
+ 0
153
+ ],
154
+ [
155
+ 0,
156
+ 0,
157
+ 0,
158
+ 0,
159
+ 19,
160
+ 0,
161
+ 5132,
162
+ 0,
163
+ 0,
164
+ 0
165
+ ],
166
+ [
167
+ 0,
168
+ 1,
169
+ 85,
170
+ 26,
171
+ 44,
172
+ 98,
173
+ 0,
174
+ 13908,
175
+ 0,
176
+ 3
177
+ ],
178
+ [
179
+ 0,
180
+ 1,
181
+ 96,
182
+ 0,
183
+ 0,
184
+ 0,
185
+ 0,
186
+ 0,
187
+ 6813,
188
+ 27
189
+ ],
190
+ [
191
+ 0,
192
+ 2,
193
+ 91,
194
+ 0,
195
+ 0,
196
+ 0,
197
+ 0,
198
+ 0,
199
+ 20,
200
+ 6115
201
+ ]
202
+ ],
203
+ "samples": 78158,
204
+ "confidence_threshold": 0.7,
205
+ "confidence_coverage": 0.9713273113436884,
206
+ "high_confidence_accuracy": 0.9996706929936642
207
+ }
reports/category_test.json ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "tfidf",
3
+ "accuracy": 0.9929848825051134,
4
+ "macro_f1": 0.9932028965957418,
5
+ "weighted_f1": 0.9929778167820927,
6
+ "report": {
7
+ "charity_donations": {
8
+ "precision": 0.9999548451187573,
9
+ "recall": 1.0,
10
+ "f1-score": 0.9999774220496264,
11
+ "support": 22145.0
12
+ },
13
+ "entertainment_recreation": {
14
+ "precision": 0.999746589517253,
15
+ "recall": 0.9980183826629564,
16
+ "f1-score": 0.9988817385800189,
17
+ "support": 23718.0
18
+ },
19
+ "financial_services": {
20
+ "precision": 0.9880473946784922,
21
+ "recall": 0.9998247090169682,
22
+ "f1-score": 0.9939011640064125,
23
+ "support": 28524.0
24
+ },
25
+ "food_dining": {
26
+ "precision": 0.9960777698114799,
27
+ "recall": 0.9931457886548085,
28
+ "f1-score": 0.9946096184620568,
29
+ "support": 23781.0
30
+ },
31
+ "government_legal": {
32
+ "precision": 0.9907501211293661,
33
+ "recall": 0.9937704338605637,
34
+ "f1-score": 0.9922579791340407,
35
+ "support": 22634.0
36
+ },
37
+ "healthcare_medical": {
38
+ "precision": 0.9814401400744145,
39
+ "recall": 0.9882316643159379,
40
+ "f1-score": 0.9848241934421189,
41
+ "support": 22688.0
42
+ },
43
+ "income": {
44
+ "precision": 0.995253164556962,
45
+ "recall": 0.9974627339042182,
46
+ "f1-score": 0.9963567242198638,
47
+ "support": 22071.0
48
+ },
49
+ "shopping_retail": {
50
+ "precision": 0.9878508434881254,
51
+ "recall": 0.973775340884496,
52
+ "f1-score": 0.980762593325946,
53
+ "support": 27722.0
54
+ },
55
+ "transportation": {
56
+ "precision": 0.9976043374101626,
57
+ "recall": 0.9926396788223486,
58
+ "f1-score": 0.9951158159522062,
59
+ "support": 23912.0
60
+ },
61
+ "utilities_services": {
62
+ "precision": 0.9951458433550532,
63
+ "recall": 0.9955376673374748,
64
+ "f1-score": 0.9953417167851285,
65
+ "support": 22858.0
66
+ },
67
+ "accuracy": 0.9929848825051134,
68
+ "macro avg": {
69
+ "precision": 0.9931871049140065,
70
+ "recall": 0.9932406399459772,
71
+ "f1-score": 0.9932028965957418,
72
+ "support": 240053.0
73
+ },
74
+ "weighted avg": {
75
+ "precision": 0.9929953943768574,
76
+ "recall": 0.9929848825051134,
77
+ "f1-score": 0.9929778167820927,
78
+ "support": 240053.0
79
+ }
80
+ },
81
+ "confusion_matrix": [
82
+ [
83
+ 22145,
84
+ 0,
85
+ 0,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0
93
+ ],
94
+ [
95
+ 0,
96
+ 23671,
97
+ 44,
98
+ 1,
99
+ 0,
100
+ 1,
101
+ 0,
102
+ 1,
103
+ 0,
104
+ 0
105
+ ],
106
+ [
107
+ 0,
108
+ 1,
109
+ 28519,
110
+ 1,
111
+ 0,
112
+ 0,
113
+ 3,
114
+ 0,
115
+ 0,
116
+ 0
117
+ ],
118
+ [
119
+ 1,
120
+ 2,
121
+ 82,
122
+ 23618,
123
+ 0,
124
+ 0,
125
+ 0,
126
+ 77,
127
+ 1,
128
+ 0
129
+ ],
130
+ [
131
+ 0,
132
+ 1,
133
+ 2,
134
+ 1,
135
+ 22493,
136
+ 0,
137
+ 102,
138
+ 34,
139
+ 1,
140
+ 0
141
+ ],
142
+ [
143
+ 0,
144
+ 0,
145
+ 47,
146
+ 0,
147
+ 0,
148
+ 22421,
149
+ 0,
150
+ 220,
151
+ 0,
152
+ 0
153
+ ],
154
+ [
155
+ 0,
156
+ 0,
157
+ 0,
158
+ 0,
159
+ 56,
160
+ 0,
161
+ 22015,
162
+ 0,
163
+ 0,
164
+ 0
165
+ ],
166
+ [
167
+ 0,
168
+ 0,
169
+ 60,
170
+ 89,
171
+ 154,
172
+ 423,
173
+ 0,
174
+ 26995,
175
+ 0,
176
+ 1
177
+ ],
178
+ [
179
+ 0,
180
+ 1,
181
+ 65,
182
+ 0,
183
+ 0,
184
+ 0,
185
+ 0,
186
+ 0,
187
+ 23736,
188
+ 110
189
+ ],
190
+ [
191
+ 0,
192
+ 1,
193
+ 45,
194
+ 1,
195
+ 0,
196
+ 0,
197
+ 0,
198
+ 0,
199
+ 55,
200
+ 22756
201
+ ]
202
+ ],
203
+ "samples": 240053,
204
+ "confidence_threshold": 0.7,
205
+ "confidence_coverage": 0.9808167363040662,
206
+ "high_confidence_accuracy": 0.9995540416567564
207
+ }
reports/category_test_country_metrics.json ADDED
@@ -0,0 +1,1037 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "Australia": {
3
+ "model_name": "tfidf-test-Australia",
4
+ "accuracy": 0.9949353448275862,
5
+ "macro_f1": 0.9948911920454151,
6
+ "weighted_f1": 0.9949260470529634,
7
+ "report": {
8
+ "charity_donations": {
9
+ "precision": 1.0,
10
+ "recall": 1.0,
11
+ "f1-score": 1.0,
12
+ "support": 4607.0
13
+ },
14
+ "entertainment_recreation": {
15
+ "precision": 1.0,
16
+ "recall": 1.0,
17
+ "f1-score": 1.0,
18
+ "support": 4776.0
19
+ },
20
+ "financial_services": {
21
+ "precision": 1.0,
22
+ "recall": 1.0,
23
+ "f1-score": 1.0,
24
+ "support": 4651.0
25
+ },
26
+ "food_dining": {
27
+ "precision": 0.9959201202490874,
28
+ "recall": 0.9984930032292788,
29
+ "f1-score": 0.997204902171576,
30
+ "support": 4645.0
31
+ },
32
+ "government_legal": {
33
+ "precision": 0.9893985287754219,
34
+ "recall": 0.9952121871599565,
35
+ "f1-score": 0.9922968427904958,
36
+ "support": 4595.0
37
+ },
38
+ "healthcare_medical": {
39
+ "precision": 0.9845045831514623,
40
+ "recall": 0.9920826918847592,
41
+ "f1-score": 0.9882791105268923,
42
+ "support": 4547.0
43
+ },
44
+ "income": {
45
+ "precision": 0.995475113122172,
46
+ "recall": 0.9974093264248705,
47
+ "f1-score": 0.99644128113879,
48
+ "support": 4632.0
49
+ },
50
+ "shopping_retail": {
51
+ "precision": 0.9903105042942083,
52
+ "recall": 0.9725346020761245,
53
+ "f1-score": 0.981342062193126,
54
+ "support": 4624.0
55
+ },
56
+ "transportation": {
57
+ "precision": 0.9985216473072862,
58
+ "recall": 0.9949494949494949,
59
+ "f1-score": 0.996732370612417,
60
+ "support": 4752.0
61
+ },
62
+ "utilities_services": {
63
+ "precision": 0.994768962510898,
64
+ "recall": 0.998468606431853,
65
+ "f1-score": 0.9966153510208539,
66
+ "support": 4571.0
67
+ },
68
+ "accuracy": 0.9949353448275862,
69
+ "macro avg": {
70
+ "precision": 0.9948899459410535,
71
+ "recall": 0.9949149912156339,
72
+ "f1-score": 0.9948911920454151,
73
+ "support": 46400.0
74
+ },
75
+ "weighted avg": {
76
+ "precision": 0.9949391773566729,
77
+ "recall": 0.9949353448275862,
78
+ "f1-score": 0.9949260470529634,
79
+ "support": 46400.0
80
+ }
81
+ },
82
+ "confusion_matrix": [
83
+ [
84
+ 4607,
85
+ 0,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0,
93
+ 0
94
+ ],
95
+ [
96
+ 0,
97
+ 4776,
98
+ 0,
99
+ 0,
100
+ 0,
101
+ 0,
102
+ 0,
103
+ 0,
104
+ 0,
105
+ 0
106
+ ],
107
+ [
108
+ 0,
109
+ 0,
110
+ 4651,
111
+ 0,
112
+ 0,
113
+ 0,
114
+ 0,
115
+ 0,
116
+ 0,
117
+ 0
118
+ ],
119
+ [
120
+ 0,
121
+ 0,
122
+ 0,
123
+ 4638,
124
+ 0,
125
+ 0,
126
+ 0,
127
+ 7,
128
+ 0,
129
+ 0
130
+ ],
131
+ [
132
+ 0,
133
+ 0,
134
+ 0,
135
+ 0,
136
+ 4573,
137
+ 0,
138
+ 21,
139
+ 1,
140
+ 0,
141
+ 0
142
+ ],
143
+ [
144
+ 0,
145
+ 0,
146
+ 0,
147
+ 0,
148
+ 0,
149
+ 4511,
150
+ 0,
151
+ 36,
152
+ 0,
153
+ 0
154
+ ],
155
+ [
156
+ 0,
157
+ 0,
158
+ 0,
159
+ 0,
160
+ 12,
161
+ 0,
162
+ 4620,
163
+ 0,
164
+ 0,
165
+ 0
166
+ ],
167
+ [
168
+ 0,
169
+ 0,
170
+ 0,
171
+ 19,
172
+ 37,
173
+ 71,
174
+ 0,
175
+ 4497,
176
+ 0,
177
+ 0
178
+ ],
179
+ [
180
+ 0,
181
+ 0,
182
+ 0,
183
+ 0,
184
+ 0,
185
+ 0,
186
+ 0,
187
+ 0,
188
+ 4728,
189
+ 24
190
+ ],
191
+ [
192
+ 0,
193
+ 0,
194
+ 0,
195
+ 0,
196
+ 0,
197
+ 0,
198
+ 0,
199
+ 0,
200
+ 7,
201
+ 4564
202
+ ]
203
+ ],
204
+ "samples": 46400,
205
+ "confidence_threshold": 0.7,
206
+ "confidence_coverage": 0.983728448275862,
207
+ "high_confidence_accuracy": 0.9995180194983021
208
+ },
209
+ "Canada": {
210
+ "model_name": "tfidf-test-Canada",
211
+ "accuracy": 0.9936672051696285,
212
+ "macro_f1": 0.9936446459609755,
213
+ "weighted_f1": 0.9936629177325705,
214
+ "report": {
215
+ "charity_donations": {
216
+ "precision": 1.0,
217
+ "recall": 1.0,
218
+ "f1-score": 1.0,
219
+ "support": 4663.0
220
+ },
221
+ "entertainment_recreation": {
222
+ "precision": 1.0,
223
+ "recall": 1.0,
224
+ "f1-score": 1.0,
225
+ "support": 4606.0
226
+ },
227
+ "financial_services": {
228
+ "precision": 1.0,
229
+ "recall": 1.0,
230
+ "f1-score": 1.0,
231
+ "support": 4634.0
232
+ },
233
+ "food_dining": {
234
+ "precision": 0.9976866456361725,
235
+ "recall": 0.9953839697859841,
236
+ "f1-score": 0.9965339775233694,
237
+ "support": 4766.0
238
+ },
239
+ "government_legal": {
240
+ "precision": 0.9914921465968587,
241
+ "recall": 0.9934426229508196,
242
+ "f1-score": 0.9924664264657713,
243
+ "support": 4575.0
244
+ },
245
+ "healthcare_medical": {
246
+ "precision": 0.9790239726027398,
247
+ "recall": 0.9872652708827974,
248
+ "f1-score": 0.9831273508866201,
249
+ "support": 4633.0
250
+ },
251
+ "income": {
252
+ "precision": 0.9952090592334495,
253
+ "recall": 0.9971634300676413,
254
+ "f1-score": 0.9961852861035423,
255
+ "support": 4583.0
256
+ },
257
+ "shopping_retail": {
258
+ "precision": 0.9805336832895888,
259
+ "recall": 0.9707665656128194,
260
+ "f1-score": 0.9756256800870512,
261
+ "support": 4618.0
262
+ },
263
+ "transportation": {
264
+ "precision": 0.9970631424375918,
265
+ "recall": 0.9956011730205279,
266
+ "f1-score": 0.9963316214233309,
267
+ "support": 4774.0
268
+ },
269
+ "utilities_services": {
270
+ "precision": 0.995414847161572,
271
+ "recall": 0.996938552372622,
272
+ "f1-score": 0.9961761171200699,
273
+ "support": 4573.0
274
+ },
275
+ "accuracy": 0.9936672051696285,
276
+ "macro avg": {
277
+ "precision": 0.9936423496957971,
278
+ "recall": 0.9936561584693212,
279
+ "f1-score": 0.9936446459609755,
280
+ "support": 46425.0
281
+ },
282
+ "weighted avg": {
283
+ "precision": 0.9936678167973348,
284
+ "recall": 0.9936672051696285,
285
+ "f1-score": 0.9936629177325705,
286
+ "support": 46425.0
287
+ }
288
+ },
289
+ "confusion_matrix": [
290
+ [
291
+ 4663,
292
+ 0,
293
+ 0,
294
+ 0,
295
+ 0,
296
+ 0,
297
+ 0,
298
+ 0,
299
+ 0,
300
+ 0
301
+ ],
302
+ [
303
+ 0,
304
+ 4606,
305
+ 0,
306
+ 0,
307
+ 0,
308
+ 0,
309
+ 0,
310
+ 0,
311
+ 0,
312
+ 0
313
+ ],
314
+ [
315
+ 0,
316
+ 0,
317
+ 4634,
318
+ 0,
319
+ 0,
320
+ 0,
321
+ 0,
322
+ 0,
323
+ 0,
324
+ 0
325
+ ],
326
+ [
327
+ 0,
328
+ 0,
329
+ 0,
330
+ 4744,
331
+ 0,
332
+ 0,
333
+ 0,
334
+ 22,
335
+ 0,
336
+ 0
337
+ ],
338
+ [
339
+ 0,
340
+ 0,
341
+ 0,
342
+ 0,
343
+ 4545,
344
+ 0,
345
+ 22,
346
+ 8,
347
+ 0,
348
+ 0
349
+ ],
350
+ [
351
+ 0,
352
+ 0,
353
+ 0,
354
+ 0,
355
+ 0,
356
+ 4574,
357
+ 0,
358
+ 59,
359
+ 0,
360
+ 0
361
+ ],
362
+ [
363
+ 0,
364
+ 0,
365
+ 0,
366
+ 0,
367
+ 13,
368
+ 0,
369
+ 4570,
370
+ 0,
371
+ 0,
372
+ 0
373
+ ],
374
+ [
375
+ 0,
376
+ 0,
377
+ 0,
378
+ 11,
379
+ 26,
380
+ 98,
381
+ 0,
382
+ 4483,
383
+ 0,
384
+ 0
385
+ ],
386
+ [
387
+ 0,
388
+ 0,
389
+ 0,
390
+ 0,
391
+ 0,
392
+ 0,
393
+ 0,
394
+ 0,
395
+ 4753,
396
+ 21
397
+ ],
398
+ [
399
+ 0,
400
+ 0,
401
+ 0,
402
+ 0,
403
+ 0,
404
+ 0,
405
+ 0,
406
+ 0,
407
+ 14,
408
+ 4559
409
+ ]
410
+ ],
411
+ "samples": 46425,
412
+ "confidence_threshold": 0.7,
413
+ "confidence_coverage": 0.9820140010770059,
414
+ "high_confidence_accuracy": 0.9996051765738101
415
+ },
416
+ "India": {
417
+ "model_name": "tfidf-test-India",
418
+ "accuracy": 0.9891758415179814,
419
+ "macro_f1": 0.9903764138960277,
420
+ "weighted_f1": 0.9891960532390336,
421
+ "report": {
422
+ "charity_donations": {
423
+ "precision": 1.0,
424
+ "recall": 1.0,
425
+ "f1-score": 1.0,
426
+ "support": 3743.0
427
+ },
428
+ "entertainment_recreation": {
429
+ "precision": 0.9987646695491044,
430
+ "recall": 0.990404246631278,
431
+ "f1-score": 0.9945668887749872,
432
+ "support": 4898.0
433
+ },
434
+ "financial_services": {
435
+ "precision": 0.9661000294782353,
436
+ "recall": 0.9995933306222041,
437
+ "f1-score": 0.9825613351321641,
438
+ "support": 9836.0
439
+ },
440
+ "food_dining": {
441
+ "precision": 0.9954788327168105,
442
+ "recall": 0.9799716771191584,
443
+ "f1-score": 0.9876643898460598,
444
+ "support": 4943.0
445
+ },
446
+ "government_legal": {
447
+ "precision": 0.9899794450154162,
448
+ "recall": 0.9912528942629277,
449
+ "f1-score": 0.9906157603805116,
450
+ "support": 3887.0
451
+ },
452
+ "healthcare_medical": {
453
+ "precision": 0.9856581077955123,
454
+ "recall": 0.9820235077206729,
455
+ "f1-score": 0.9838374509351189,
456
+ "support": 4339.0
457
+ },
458
+ "income": {
459
+ "precision": 0.993570854540584,
460
+ "recall": 0.9970430107526882,
461
+ "f1-score": 0.9953039044679994,
462
+ "support": 3720.0
463
+ },
464
+ "shopping_retail": {
465
+ "precision": 0.9941539819104346,
466
+ "recall": 0.9815944238727946,
467
+ "f1-score": 0.9878342832091188,
468
+ "support": 9182.0
469
+ },
470
+ "transportation": {
471
+ "precision": 0.9970918155380141,
472
+ "recall": 0.9830022527134958,
473
+ "f1-score": 0.989996906259668,
474
+ "support": 4883.0
475
+ },
476
+ "utilities_services": {
477
+ "precision": 0.9958997722095672,
478
+ "recall": 0.9869074492099322,
479
+ "f1-score": 0.9913832199546485,
480
+ "support": 4430.0
481
+ },
482
+ "accuracy": 0.9891758415179814,
483
+ "macro avg": {
484
+ "precision": 0.9916697508753678,
485
+ "recall": 0.9891792792905152,
486
+ "f1-score": 0.9903764138960277,
487
+ "support": 53861.0
488
+ },
489
+ "weighted avg": {
490
+ "precision": 0.9893619225593913,
491
+ "recall": 0.9891758415179814,
492
+ "f1-score": 0.9891960532390336,
493
+ "support": 53861.0
494
+ }
495
+ },
496
+ "confusion_matrix": [
497
+ [
498
+ 3743,
499
+ 0,
500
+ 0,
501
+ 0,
502
+ 0,
503
+ 0,
504
+ 0,
505
+ 0,
506
+ 0,
507
+ 0
508
+ ],
509
+ [
510
+ 0,
511
+ 4851,
512
+ 44,
513
+ 1,
514
+ 0,
515
+ 1,
516
+ 0,
517
+ 1,
518
+ 0,
519
+ 0
520
+ ],
521
+ [
522
+ 0,
523
+ 1,
524
+ 9832,
525
+ 1,
526
+ 0,
527
+ 0,
528
+ 2,
529
+ 0,
530
+ 0,
531
+ 0
532
+ ],
533
+ [
534
+ 0,
535
+ 2,
536
+ 82,
537
+ 4844,
538
+ 0,
539
+ 0,
540
+ 0,
541
+ 14,
542
+ 1,
543
+ 0
544
+ ],
545
+ [
546
+ 0,
547
+ 1,
548
+ 2,
549
+ 1,
550
+ 3853,
551
+ 0,
552
+ 22,
553
+ 7,
554
+ 1,
555
+ 0
556
+ ],
557
+ [
558
+ 0,
559
+ 0,
560
+ 47,
561
+ 0,
562
+ 0,
563
+ 4261,
564
+ 0,
565
+ 31,
566
+ 0,
567
+ 0
568
+ ],
569
+ [
570
+ 0,
571
+ 0,
572
+ 0,
573
+ 0,
574
+ 11,
575
+ 0,
576
+ 3709,
577
+ 0,
578
+ 0,
579
+ 0
580
+ ],
581
+ [
582
+ 0,
583
+ 0,
584
+ 60,
585
+ 19,
586
+ 28,
587
+ 61,
588
+ 0,
589
+ 9013,
590
+ 0,
591
+ 1
592
+ ],
593
+ [
594
+ 0,
595
+ 1,
596
+ 65,
597
+ 0,
598
+ 0,
599
+ 0,
600
+ 0,
601
+ 0,
602
+ 4800,
603
+ 17
604
+ ],
605
+ [
606
+ 0,
607
+ 1,
608
+ 45,
609
+ 0,
610
+ 0,
611
+ 0,
612
+ 0,
613
+ 0,
614
+ 12,
615
+ 4372
616
+ ]
617
+ ],
618
+ "samples": 53861,
619
+ "confidence_threshold": 0.7,
620
+ "confidence_coverage": 0.9753810735040196,
621
+ "high_confidence_accuracy": 0.9996193014181022
622
+ },
623
+ "UK": {
624
+ "model_name": "tfidf-test-UK",
625
+ "accuracy": 0.9940697128717095,
626
+ "macro_f1": 0.9940612354515542,
627
+ "weighted_f1": 0.9940660583800917,
628
+ "report": {
629
+ "charity_donations": {
630
+ "precision": 1.0,
631
+ "recall": 1.0,
632
+ "f1-score": 1.0,
633
+ "support": 4560.0
634
+ },
635
+ "entertainment_recreation": {
636
+ "precision": 1.0,
637
+ "recall": 1.0,
638
+ "f1-score": 1.0,
639
+ "support": 4662.0
640
+ },
641
+ "financial_services": {
642
+ "precision": 1.0,
643
+ "recall": 1.0,
644
+ "f1-score": 1.0,
645
+ "support": 4741.0
646
+ },
647
+ "food_dining": {
648
+ "precision": 0.9978840457046128,
649
+ "recall": 0.995987328405491,
650
+ "f1-score": 0.996934784906458,
651
+ "support": 4735.0
652
+ },
653
+ "government_legal": {
654
+ "precision": 0.9921161825726141,
655
+ "recall": 0.9943855271366189,
656
+ "f1-score": 0.993249558624987,
657
+ "support": 4809.0
658
+ },
659
+ "healthcare_medical": {
660
+ "precision": 0.980806142034549,
661
+ "recall": 0.988819608686304,
662
+ "f1-score": 0.984796573875803,
663
+ "support": 4651.0
664
+ },
665
+ "income": {
666
+ "precision": 0.9959393032699295,
667
+ "recall": 0.9970047068891742,
668
+ "f1-score": 0.9964717203036458,
669
+ "support": 4674.0
670
+ },
671
+ "shopping_retail": {
672
+ "precision": 0.9828781967923711,
673
+ "recall": 0.9733848465335909,
674
+ "f1-score": 0.9781084870052842,
675
+ "support": 4659.0
676
+ },
677
+ "transportation": {
678
+ "precision": 0.9966044142614601,
679
+ "recall": 0.9944938585345193,
680
+ "f1-score": 0.9955480178079288,
681
+ "support": 4722.0
682
+ },
683
+ "utilities_services": {
684
+ "precision": 0.9944385026737967,
685
+ "recall": 0.9965702036441586,
686
+ "f1-score": 0.9955032119914347,
687
+ "support": 4665.0
688
+ },
689
+ "accuracy": 0.9940697128717095,
690
+ "macro avg": {
691
+ "precision": 0.9940666787309332,
692
+ "recall": 0.9940646079829858,
693
+ "f1-score": 0.9940612354515542,
694
+ "support": 46878.0
695
+ },
696
+ "weighted avg": {
697
+ "precision": 0.994071174675987,
698
+ "recall": 0.9940697128717095,
699
+ "f1-score": 0.9940660583800917,
700
+ "support": 46878.0
701
+ }
702
+ },
703
+ "confusion_matrix": [
704
+ [
705
+ 4560,
706
+ 0,
707
+ 0,
708
+ 0,
709
+ 0,
710
+ 0,
711
+ 0,
712
+ 0,
713
+ 0,
714
+ 0
715
+ ],
716
+ [
717
+ 0,
718
+ 4662,
719
+ 0,
720
+ 0,
721
+ 0,
722
+ 0,
723
+ 0,
724
+ 0,
725
+ 0,
726
+ 0
727
+ ],
728
+ [
729
+ 0,
730
+ 0,
731
+ 4741,
732
+ 0,
733
+ 0,
734
+ 0,
735
+ 0,
736
+ 0,
737
+ 0,
738
+ 0
739
+ ],
740
+ [
741
+ 0,
742
+ 0,
743
+ 0,
744
+ 4716,
745
+ 0,
746
+ 0,
747
+ 0,
748
+ 19,
749
+ 0,
750
+ 0
751
+ ],
752
+ [
753
+ 0,
754
+ 0,
755
+ 0,
756
+ 0,
757
+ 4782,
758
+ 0,
759
+ 19,
760
+ 8,
761
+ 0,
762
+ 0
763
+ ],
764
+ [
765
+ 0,
766
+ 0,
767
+ 0,
768
+ 0,
769
+ 0,
770
+ 4599,
771
+ 0,
772
+ 52,
773
+ 0,
774
+ 0
775
+ ],
776
+ [
777
+ 0,
778
+ 0,
779
+ 0,
780
+ 0,
781
+ 14,
782
+ 0,
783
+ 4660,
784
+ 0,
785
+ 0,
786
+ 0
787
+ ],
788
+ [
789
+ 0,
790
+ 0,
791
+ 0,
792
+ 10,
793
+ 24,
794
+ 90,
795
+ 0,
796
+ 4535,
797
+ 0,
798
+ 0
799
+ ],
800
+ [
801
+ 0,
802
+ 0,
803
+ 0,
804
+ 0,
805
+ 0,
806
+ 0,
807
+ 0,
808
+ 0,
809
+ 4696,
810
+ 26
811
+ ],
812
+ [
813
+ 0,
814
+ 0,
815
+ 0,
816
+ 0,
817
+ 0,
818
+ 0,
819
+ 0,
820
+ 0,
821
+ 16,
822
+ 4649
823
+ ]
824
+ ],
825
+ "samples": 46878,
826
+ "confidence_threshold": 0.7,
827
+ "confidence_coverage": 0.9814838517001578,
828
+ "high_confidence_accuracy": 0.9995218430775918
829
+ },
830
+ "USA": {
831
+ "model_name": "tfidf-test-USA",
832
+ "accuracy": 0.9936759233367033,
833
+ "macro_f1": 0.9936282588322631,
834
+ "weighted_f1": 0.9936595529700609,
835
+ "report": {
836
+ "charity_donations": {
837
+ "precision": 0.999781325169473,
838
+ "recall": 1.0,
839
+ "f1-score": 0.9998906506287589,
840
+ "support": 4572.0
841
+ },
842
+ "entertainment_recreation": {
843
+ "precision": 1.0,
844
+ "recall": 1.0,
845
+ "f1-score": 1.0,
846
+ "support": 4776.0
847
+ },
848
+ "financial_services": {
849
+ "precision": 1.0,
850
+ "recall": 0.9997854997854998,
851
+ "f1-score": 0.9998927383889306,
852
+ "support": 4662.0
853
+ },
854
+ "food_dining": {
855
+ "precision": 0.9934140641597621,
856
+ "recall": 0.9965899403239556,
857
+ "f1-score": 0.9949994680285137,
858
+ "support": 4692.0
859
+ },
860
+ "government_legal": {
861
+ "precision": 0.9905956112852664,
862
+ "recall": 0.9941275167785235,
863
+ "f1-score": 0.9923584214382917,
864
+ "support": 4768.0
865
+ },
866
+ "healthcare_medical": {
867
+ "precision": 0.9775060056780956,
868
+ "recall": 0.9907038512616202,
869
+ "f1-score": 0.984060679344839,
870
+ "support": 4518.0
871
+ },
872
+ "income": {
873
+ "precision": 0.9957541899441341,
874
+ "recall": 0.9986553115194979,
875
+ "f1-score": 0.9972026407071725,
876
+ "support": 4462.0
877
+ },
878
+ "shopping_retail": {
879
+ "precision": 0.9852227613586237,
880
+ "recall": 0.9629230437594309,
881
+ "f1-score": 0.973945274174207,
882
+ "support": 4639.0
883
+ },
884
+ "transportation": {
885
+ "precision": 0.9987408184679958,
886
+ "recall": 0.9953984522066514,
887
+ "f1-score": 0.9970668342761366,
888
+ "support": 4781.0
889
+ },
890
+ "utilities_services": {
891
+ "precision": 0.9952524816573155,
892
+ "recall": 0.9984845204589738,
893
+ "f1-score": 0.996865881335783,
894
+ "support": 4619.0
895
+ },
896
+ "accuracy": 0.9936759233367033,
897
+ "macro avg": {
898
+ "precision": 0.9936267257720667,
899
+ "recall": 0.9936668136094152,
900
+ "f1-score": 0.9936282588322631,
901
+ "support": 46489.0
902
+ },
903
+ "weighted avg": {
904
+ "precision": 0.9936799171270243,
905
+ "recall": 0.9936759233367033,
906
+ "f1-score": 0.9936595529700609,
907
+ "support": 46489.0
908
+ }
909
+ },
910
+ "confusion_matrix": [
911
+ [
912
+ 4572,
913
+ 0,
914
+ 0,
915
+ 0,
916
+ 0,
917
+ 0,
918
+ 0,
919
+ 0,
920
+ 0,
921
+ 0
922
+ ],
923
+ [
924
+ 0,
925
+ 4776,
926
+ 0,
927
+ 0,
928
+ 0,
929
+ 0,
930
+ 0,
931
+ 0,
932
+ 0,
933
+ 0
934
+ ],
935
+ [
936
+ 0,
937
+ 0,
938
+ 4661,
939
+ 0,
940
+ 0,
941
+ 0,
942
+ 1,
943
+ 0,
944
+ 0,
945
+ 0
946
+ ],
947
+ [
948
+ 1,
949
+ 0,
950
+ 0,
951
+ 4676,
952
+ 0,
953
+ 0,
954
+ 0,
955
+ 15,
956
+ 0,
957
+ 0
958
+ ],
959
+ [
960
+ 0,
961
+ 0,
962
+ 0,
963
+ 0,
964
+ 4740,
965
+ 0,
966
+ 18,
967
+ 10,
968
+ 0,
969
+ 0
970
+ ],
971
+ [
972
+ 0,
973
+ 0,
974
+ 0,
975
+ 0,
976
+ 0,
977
+ 4476,
978
+ 0,
979
+ 42,
980
+ 0,
981
+ 0
982
+ ],
983
+ [
984
+ 0,
985
+ 0,
986
+ 0,
987
+ 0,
988
+ 6,
989
+ 0,
990
+ 4456,
991
+ 0,
992
+ 0,
993
+ 0
994
+ ],
995
+ [
996
+ 0,
997
+ 0,
998
+ 0,
999
+ 30,
1000
+ 39,
1001
+ 103,
1002
+ 0,
1003
+ 4467,
1004
+ 0,
1005
+ 0
1006
+ ],
1007
+ [
1008
+ 0,
1009
+ 0,
1010
+ 0,
1011
+ 0,
1012
+ 0,
1013
+ 0,
1014
+ 0,
1015
+ 0,
1016
+ 4759,
1017
+ 22
1018
+ ],
1019
+ [
1020
+ 0,
1021
+ 0,
1022
+ 0,
1023
+ 1,
1024
+ 0,
1025
+ 0,
1026
+ 0,
1027
+ 0,
1028
+ 6,
1029
+ 4612
1030
+ ]
1031
+ ],
1032
+ "samples": 46489,
1033
+ "confidence_threshold": 0.7,
1034
+ "confidence_coverage": 0.9823399083654197,
1035
+ "high_confidence_accuracy": 0.999496365069633
1036
+ }
1037
+ }
reports/dataset_quality.json ADDED
@@ -0,0 +1,30 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "rows": 1687124,
3
+ "columns": [
4
+ "text",
5
+ "label",
6
+ "source",
7
+ "source_label",
8
+ "country",
9
+ "currency",
10
+ "language",
11
+ "record_id"
12
+ ],
13
+ "missing_by_column": {
14
+ "text": 0,
15
+ "label": 0,
16
+ "source": 0,
17
+ "source_label": 0,
18
+ "country": 0,
19
+ "currency": 0,
20
+ "language": 0,
21
+ "record_id": 0
22
+ },
23
+ "duplicate_text_rows": 173174,
24
+ "text_length": {
25
+ "mean": 34.24548165991356,
26
+ "median": 25.0,
27
+ "p95": 142.0,
28
+ "max": 206
29
+ }
30
+ }
reports/dataset_summary.json ADDED
@@ -0,0 +1,29 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "rows": 1687124,
3
+ "classes": 10,
4
+ "fingerprint": "d108095d65fb9b6895133560fe3c85d4633ed878c29a4eb8059b20a43d482205",
5
+ "class_counts": {
6
+ "financial_services": 205592,
7
+ "shopping_retail": 202030,
8
+ "transportation": 166379,
9
+ "food_dining": 166367,
10
+ "entertainment_recreation": 165250,
11
+ "utilities_services": 159787,
12
+ "healthcare_medical": 159555,
13
+ "government_legal": 156292,
14
+ "charity_donations": 153635,
15
+ "income": 152237
16
+ },
17
+ "country_counts": {
18
+ "India": 439581,
19
+ "Canada": 312426,
20
+ "UK": 312032,
21
+ "Australia": 311926,
22
+ "USA": 311159
23
+ },
24
+ "source_counts": {
25
+ "global-transaction-categorization": 1560218,
26
+ "finee-india": 126175,
27
+ "synthetic-indian-transactions": 731
28
+ }
29
+ }
reports/duplicate_candidates.json ADDED
@@ -0,0 +1,5007 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "indexed_rows": 500000,
3
+ "candidate_pairs": 50000,
4
+ "threshold": 0.92,
5
+ "examples": [
6
+ [
7
+ 6,
8
+ 76322,
9
+ 0.9569108853181859
10
+ ],
11
+ [
12
+ 6,
13
+ 86380,
14
+ 0.9569108853181859
15
+ ],
16
+ [
17
+ 6,
18
+ 141830,
19
+ 0.9569108853181859
20
+ ],
21
+ [
22
+ 6,
23
+ 181897,
24
+ 0.9347752208046333
25
+ ],
26
+ [
27
+ 6,
28
+ 278711,
29
+ 0.9347752208046333
30
+ ],
31
+ [
32
+ 6,
33
+ 350716,
34
+ 0.9347752208046333
35
+ ],
36
+ [
37
+ 16,
38
+ 184762,
39
+ 0.9650001837511328
40
+ ],
41
+ [
42
+ 16,
43
+ 339032,
44
+ 0.9650001837511328
45
+ ],
46
+ [
47
+ 21,
48
+ 3271,
49
+ 0.9262436959739049
50
+ ],
51
+ [
52
+ 21,
53
+ 218369,
54
+ 0.936725321039221
55
+ ],
56
+ [
57
+ 21,
58
+ 371967,
59
+ 0.9999999999999999
60
+ ],
61
+ [
62
+ 21,
63
+ 400319,
64
+ 0.936725321039221
65
+ ],
66
+ [
67
+ 28,
68
+ 98362,
69
+ 0.9389588021495819
70
+ ],
71
+ [
72
+ 28,
73
+ 298660,
74
+ 0.9289891339367131
75
+ ],
76
+ [
77
+ 28,
78
+ 476342,
79
+ 0.9389588021495819
80
+ ],
81
+ [
82
+ 35,
83
+ 14912,
84
+ 0.951872264755927
85
+ ],
86
+ [
87
+ 35,
88
+ 108016,
89
+ 0.9999999999999989
90
+ ],
91
+ [
92
+ 35,
93
+ 335553,
94
+ 0.951872264755927
95
+ ],
96
+ [
97
+ 35,
98
+ 377591,
99
+ 0.9999999999999989
100
+ ],
101
+ [
102
+ 38,
103
+ 237506,
104
+ 0.9999999999999998
105
+ ],
106
+ [
107
+ 38,
108
+ 368052,
109
+ 0.9712863181672184
110
+ ],
111
+ [
112
+ 38,
113
+ 396256,
114
+ 0.9712863181672184
115
+ ],
116
+ [
117
+ 44,
118
+ 398670,
119
+ 0.9401259484529804
120
+ ],
121
+ [
122
+ 45,
123
+ 387418,
124
+ 0.959349676767061
125
+ ],
126
+ [
127
+ 47,
128
+ 101044,
129
+ 0.9315705062893224
130
+ ],
131
+ [
132
+ 47,
133
+ 188756,
134
+ 0.9423088325571768
135
+ ],
136
+ [
137
+ 47,
138
+ 191886,
139
+ 0.9315705062893224
140
+ ],
141
+ [
142
+ 47,
143
+ 213683,
144
+ 0.9315705062893224
145
+ ],
146
+ [
147
+ 47,
148
+ 296404,
149
+ 0.9315705062893224
150
+ ],
151
+ [
152
+ 55,
153
+ 78943,
154
+ 0.9420008951536991
155
+ ],
156
+ [
157
+ 55,
158
+ 97404,
159
+ 0.9999999999999999
160
+ ],
161
+ [
162
+ 55,
163
+ 269567,
164
+ 0.9420008951536991
165
+ ],
166
+ [
167
+ 60,
168
+ 485579,
169
+ 0.9999999999999998
170
+ ],
171
+ [
172
+ 61,
173
+ 2521,
174
+ 0.9501958979288543
175
+ ],
176
+ [
177
+ 61,
178
+ 45468,
179
+ 0.9353646250845972
180
+ ],
181
+ [
182
+ 61,
183
+ 169281,
184
+ 0.9816380260642956
185
+ ],
186
+ [
187
+ 61,
188
+ 178860,
189
+ 0.9285727502135307
190
+ ],
191
+ [
192
+ 61,
193
+ 180395,
194
+ 0.9302897703638828
195
+ ],
196
+ [
197
+ 61,
198
+ 245482,
199
+ 0.9285727502135307
200
+ ],
201
+ [
202
+ 61,
203
+ 292173,
204
+ 0.9999999999999997
205
+ ],
206
+ [
207
+ 64,
208
+ 18000,
209
+ 0.9211242271270801
210
+ ],
211
+ [
212
+ 64,
213
+ 107429,
214
+ 0.9709331610547435
215
+ ],
216
+ [
217
+ 64,
218
+ 144855,
219
+ 0.9709331610547435
220
+ ],
221
+ [
222
+ 64,
223
+ 470868,
224
+ 0.9709331610547435
225
+ ],
226
+ [
227
+ 65,
228
+ 96881,
229
+ 0.9461886165935008
230
+ ],
231
+ [
232
+ 65,
233
+ 97071,
234
+ 0.982034533516726
235
+ ],
236
+ [
237
+ 65,
238
+ 269063,
239
+ 0.9461886165935008
240
+ ],
241
+ [
242
+ 65,
243
+ 299724,
244
+ 0.982034533516726
245
+ ],
246
+ [
247
+ 65,
248
+ 427574,
249
+ 0.982034533516726
250
+ ],
251
+ [
252
+ 67,
253
+ 239594,
254
+ 0.9675543965445252
255
+ ],
256
+ [
257
+ 68,
258
+ 208341,
259
+ 0.9291315246568975
260
+ ],
261
+ [
262
+ 68,
263
+ 265603,
264
+ 0.9547947797756278
265
+ ],
266
+ [
267
+ 68,
268
+ 271566,
269
+ 0.9393082560897947
270
+ ],
271
+ [
272
+ 68,
273
+ 325420,
274
+ 0.9547947797756278
275
+ ],
276
+ [
277
+ 68,
278
+ 386041,
279
+ 0.9291315246568975
280
+ ],
281
+ [
282
+ 68,
283
+ 394281,
284
+ 0.9291315246568975
285
+ ],
286
+ [
287
+ 85,
288
+ 15574,
289
+ 0.955606848307565
290
+ ],
291
+ [
292
+ 85,
293
+ 132222,
294
+ 0.9378361403839358
295
+ ],
296
+ [
297
+ 85,
298
+ 252334,
299
+ 0.9378361403839358
300
+ ],
301
+ [
302
+ 85,
303
+ 342443,
304
+ 0.9709364228510461
305
+ ],
306
+ [
307
+ 85,
308
+ 459929,
309
+ 0.955606848307565
310
+ ],
311
+ [
312
+ 85,
313
+ 492103,
314
+ 0.9483289107978262
315
+ ],
316
+ [
317
+ 86,
318
+ 258658,
319
+ 0.972594852006915
320
+ ],
321
+ [
322
+ 86,
323
+ 485515,
324
+ 0.9597970127086867
325
+ ],
326
+ [
327
+ 87,
328
+ 30688,
329
+ 0.9345280931094708
330
+ ],
331
+ [
332
+ 87,
333
+ 262076,
334
+ 0.9345280931094708
335
+ ],
336
+ [
337
+ 87,
338
+ 332743,
339
+ 0.9345280931094708
340
+ ],
341
+ [
342
+ 87,
343
+ 336277,
344
+ 0.9541336173677055
345
+ ],
346
+ [
347
+ 87,
348
+ 340161,
349
+ 0.9345280931094708
350
+ ],
351
+ [
352
+ 90,
353
+ 112565,
354
+ 0.9844313119682937
355
+ ],
356
+ [
357
+ 90,
358
+ 324352,
359
+ 0.9345598284581905
360
+ ],
361
+ [
362
+ 90,
363
+ 412858,
364
+ 0.9730466739805138
365
+ ],
366
+ [
367
+ 90,
368
+ 438636,
369
+ 0.9730466739805138
370
+ ],
371
+ [
372
+ 90,
373
+ 440270,
374
+ 0.9426782410272069
375
+ ],
376
+ [
377
+ 90,
378
+ 455316,
379
+ 0.9730466739805138
380
+ ],
381
+ [
382
+ 95,
383
+ 101560,
384
+ 0.9686725784499903
385
+ ],
386
+ [
387
+ 95,
388
+ 115454,
389
+ 0.9686725784499903
390
+ ],
391
+ [
392
+ 95,
393
+ 126970,
394
+ 0.9999999999999992
395
+ ],
396
+ [
397
+ 95,
398
+ 280217,
399
+ 0.9568172420013635
400
+ ],
401
+ [
402
+ 95,
403
+ 315270,
404
+ 0.924279461372782
405
+ ],
406
+ [
407
+ 95,
408
+ 417312,
409
+ 0.9686725784499903
410
+ ],
411
+ [
412
+ 95,
413
+ 418646,
414
+ 0.924279461372782
415
+ ],
416
+ [
417
+ 97,
418
+ 393520,
419
+ 0.9279344671977388
420
+ ],
421
+ [
422
+ 100,
423
+ 395230,
424
+ 0.9756079505503606
425
+ ],
426
+ [
427
+ 101,
428
+ 35219,
429
+ 0.9304936211377764
430
+ ],
431
+ [
432
+ 101,
433
+ 49545,
434
+ 0.9999999999999996
435
+ ],
436
+ [
437
+ 101,
438
+ 99155,
439
+ 0.9999999999999996
440
+ ],
441
+ [
442
+ 101,
443
+ 272953,
444
+ 0.9587193822924753
445
+ ],
446
+ [
447
+ 101,
448
+ 282138,
449
+ 0.9745131421273576
450
+ ],
451
+ [
452
+ 101,
453
+ 485687,
454
+ 0.9999999999999996
455
+ ],
456
+ [
457
+ 106,
458
+ 410925,
459
+ 0.927165838627213
460
+ ],
461
+ [
462
+ 109,
463
+ 95649,
464
+ 0.9999999999999998
465
+ ],
466
+ [
467
+ 109,
468
+ 208756,
469
+ 0.979946790439496
470
+ ],
471
+ [
472
+ 110,
473
+ 197776,
474
+ 0.9372346892324137
475
+ ],
476
+ [
477
+ 119,
478
+ 318626,
479
+ 0.9375698144775327
480
+ ],
481
+ [
482
+ 120,
483
+ 31967,
484
+ 0.954061716855963
485
+ ],
486
+ [
487
+ 120,
488
+ 157154,
489
+ 0.9442296433598757
490
+ ],
491
+ [
492
+ 120,
493
+ 427497,
494
+ 0.969546138847939
495
+ ],
496
+ [
497
+ 120,
498
+ 430320,
499
+ 0.954061716855963
500
+ ],
501
+ [
502
+ 120,
503
+ 437673,
504
+ 0.9442296433598757
505
+ ],
506
+ [
507
+ 129,
508
+ 99743,
509
+ 0.9658054056208742
510
+ ],
511
+ [
512
+ 129,
513
+ 256530,
514
+ 0.9237015491175189
515
+ ],
516
+ [
517
+ 129,
518
+ 308044,
519
+ 0.9999999999999989
520
+ ],
521
+ [
522
+ 129,
523
+ 477542,
524
+ 0.9237015491175189
525
+ ],
526
+ [
527
+ 134,
528
+ 85313,
529
+ 0.9600549917921264
530
+ ],
531
+ [
532
+ 134,
533
+ 214655,
534
+ 0.9600549917921264
535
+ ],
536
+ [
537
+ 134,
538
+ 266378,
539
+ 0.9600549917921264
540
+ ],
541
+ [
542
+ 134,
543
+ 308757,
544
+ 0.9600549917921264
545
+ ],
546
+ [
547
+ 134,
548
+ 331964,
549
+ 0.942769725939123
550
+ ],
551
+ [
552
+ 134,
553
+ 461503,
554
+ 0.9600549917921264
555
+ ],
556
+ [
557
+ 138,
558
+ 180673,
559
+ 0.9438658361984463
560
+ ],
561
+ [
562
+ 138,
563
+ 315592,
564
+ 0.9438658361984463
565
+ ],
566
+ [
567
+ 138,
568
+ 379394,
569
+ 0.9581129630523821
570
+ ],
571
+ [
572
+ 138,
573
+ 382140,
574
+ 0.9438658361984463
575
+ ],
576
+ [
577
+ 138,
578
+ 387816,
579
+ 0.9438658361984463
580
+ ],
581
+ [
582
+ 138,
583
+ 422505,
584
+ 0.9438658361984463
585
+ ],
586
+ [
587
+ 155,
588
+ 116127,
589
+ 0.9648497026281571
590
+ ],
591
+ [
592
+ 155,
593
+ 166606,
594
+ 0.9417889601854382
595
+ ],
596
+ [
597
+ 155,
598
+ 175496,
599
+ 0.9239023531405356
600
+ ],
601
+ [
602
+ 155,
603
+ 360456,
604
+ 0.9342745020503482
605
+ ],
606
+ [
607
+ 165,
608
+ 78890,
609
+ 0.9222009762895308
610
+ ],
611
+ [
612
+ 165,
613
+ 186006,
614
+ 0.9274988562252541
615
+ ],
616
+ [
617
+ 165,
618
+ 289382,
619
+ 0.9561384543371363
620
+ ],
621
+ [
622
+ 165,
623
+ 402477,
624
+ 0.9561384543371363
625
+ ],
626
+ [
627
+ 165,
628
+ 449746,
629
+ 0.9561384543371363
630
+ ],
631
+ [
632
+ 166,
633
+ 239266,
634
+ 0.9439591623058534
635
+ ],
636
+ [
637
+ 168,
638
+ 87336,
639
+ 0.9315327111366013
640
+ ],
641
+ [
642
+ 169,
643
+ 128770,
644
+ 0.9671273368761197
645
+ ],
646
+ [
647
+ 171,
648
+ 34484,
649
+ 0.9657842764832104
650
+ ],
651
+ [
652
+ 171,
653
+ 253297,
654
+ 0.9576987895723466
655
+ ],
656
+ [
657
+ 171,
658
+ 335593,
659
+ 0.9693655606260183
660
+ ],
661
+ [
662
+ 182,
663
+ 125256,
664
+ 0.9270761491928805
665
+ ],
666
+ [
667
+ 182,
668
+ 260210,
669
+ 0.9421311777363055
670
+ ],
671
+ [
672
+ 182,
673
+ 439684,
674
+ 0.9270761491928805
675
+ ],
676
+ [
677
+ 196,
678
+ 87550,
679
+ 0.9585282699440338
680
+ ],
681
+ [
682
+ 196,
683
+ 101887,
684
+ 0.9585282699440338
685
+ ],
686
+ [
687
+ 196,
688
+ 344020,
689
+ 0.9754739436965019
690
+ ],
691
+ [
692
+ 203,
693
+ 108707,
694
+ 0.9470304378074317
695
+ ],
696
+ [
697
+ 211,
698
+ 8358,
699
+ 0.9347659715229313
700
+ ],
701
+ [
702
+ 213,
703
+ 124754,
704
+ 0.9229454993416641
705
+ ],
706
+ [
707
+ 214,
708
+ 71775,
709
+ 0.9999999999999999
710
+ ],
711
+ [
712
+ 214,
713
+ 169568,
714
+ 0.9215695635881596
715
+ ],
716
+ [
717
+ 214,
718
+ 263343,
719
+ 0.9613118553069526
720
+ ],
721
+ [
722
+ 214,
723
+ 363326,
724
+ 0.9215695635881596
725
+ ],
726
+ [
727
+ 214,
728
+ 379229,
729
+ 0.9613118553069526
730
+ ],
731
+ [
732
+ 214,
733
+ 461478,
734
+ 0.9613118553069526
735
+ ],
736
+ [
737
+ 214,
738
+ 470744,
739
+ 0.9613118553069526
740
+ ],
741
+ [
742
+ 216,
743
+ 106402,
744
+ 0.9274399994002813
745
+ ],
746
+ [
747
+ 216,
748
+ 310026,
749
+ 0.9274399994002813
750
+ ],
751
+ [
752
+ 221,
753
+ 44473,
754
+ 0.9276028452483042
755
+ ],
756
+ [
757
+ 221,
758
+ 448127,
759
+ 0.9579814811547946
760
+ ],
761
+ [
762
+ 227,
763
+ 166992,
764
+ 0.9210392169047823
765
+ ],
766
+ [
767
+ 227,
768
+ 243350,
769
+ 0.9721598318624787
770
+ ],
771
+ [
772
+ 227,
773
+ 442128,
774
+ 0.9478840779955781
775
+ ],
776
+ [
777
+ 227,
778
+ 445117,
779
+ 0.9501551212683111
780
+ ],
781
+ [
782
+ 230,
783
+ 4116,
784
+ 0.9391578350846503
785
+ ],
786
+ [
787
+ 230,
788
+ 122153,
789
+ 0.9381433824258836
790
+ ],
791
+ [
792
+ 230,
793
+ 206537,
794
+ 0.9301407267899473
795
+ ],
796
+ [
797
+ 230,
798
+ 225861,
799
+ 0.9202493679913375
800
+ ],
801
+ [
802
+ 230,
803
+ 262033,
804
+ 0.9828442545057101
805
+ ],
806
+ [
807
+ 230,
808
+ 302508,
809
+ 0.9301407267899473
810
+ ],
811
+ [
812
+ 230,
813
+ 370328,
814
+ 0.9202493679913375
815
+ ],
816
+ [
817
+ 230,
818
+ 393386,
819
+ 0.9301407267899473
820
+ ],
821
+ [
822
+ 230,
823
+ 446519,
824
+ 0.9301407267899473
825
+ ],
826
+ [
827
+ 236,
828
+ 178827,
829
+ 0.9342471220080196
830
+ ],
831
+ [
832
+ 236,
833
+ 289596,
834
+ 0.9342471220080196
835
+ ],
836
+ [
837
+ 242,
838
+ 106046,
839
+ 0.9391563889957711
840
+ ],
841
+ [
842
+ 242,
843
+ 241194,
844
+ 0.9391563889957711
845
+ ],
846
+ [
847
+ 242,
848
+ 298331,
849
+ 0.9391563889957711
850
+ ],
851
+ [
852
+ 242,
853
+ 434507,
854
+ 0.9685039745126858
855
+ ],
856
+ [
857
+ 242,
858
+ 475533,
859
+ 0.9539300445246598
860
+ ],
861
+ [
862
+ 249,
863
+ 231320,
864
+ 0.9511351053090109
865
+ ],
866
+ [
867
+ 249,
868
+ 254359,
869
+ 0.9511351053090109
870
+ ],
871
+ [
872
+ 249,
873
+ 261005,
874
+ 0.9299747248857237
875
+ ],
876
+ [
877
+ 249,
878
+ 314708,
879
+ 0.9299747248857237
880
+ ],
881
+ [
882
+ 253,
883
+ 216651,
884
+ 0.9742409825994149
885
+ ],
886
+ [
887
+ 275,
888
+ 177444,
889
+ 0.9339894834085539
890
+ ],
891
+ [
892
+ 275,
893
+ 196373,
894
+ 0.9339894834085539
895
+ ],
896
+ [
897
+ 275,
898
+ 197094,
899
+ 0.9815271922019322
900
+ ],
901
+ [
902
+ 275,
903
+ 379877,
904
+ 0.9815271922019322
905
+ ],
906
+ [
907
+ 276,
908
+ 102027,
909
+ 0.935339923204448
910
+ ],
911
+ [
912
+ 276,
913
+ 197989,
914
+ 0.9683932679380731
915
+ ],
916
+ [
917
+ 276,
918
+ 318919,
919
+ 0.9683932679380731
920
+ ],
921
+ [
922
+ 276,
923
+ 319055,
924
+ 0.9280073361654412
925
+ ],
926
+ [
927
+ 276,
928
+ 451922,
929
+ 0.9683932679380731
930
+ ],
931
+ [
932
+ 277,
933
+ 221898,
934
+ 0.9381356996968246
935
+ ],
936
+ [
937
+ 289,
938
+ 231749,
939
+ 0.955932219107113
940
+ ],
941
+ [
942
+ 289,
943
+ 288929,
944
+ 0.955932219107113
945
+ ],
946
+ [
947
+ 289,
948
+ 315709,
949
+ 0.955932219107113
950
+ ],
951
+ [
952
+ 289,
953
+ 471654,
954
+ 0.9288647974664532
955
+ ],
956
+ [
957
+ 296,
958
+ 254251,
959
+ 0.9296227377026482
960
+ ],
961
+ [
962
+ 301,
963
+ 13913,
964
+ 0.923946979302372
965
+ ],
966
+ [
967
+ 301,
968
+ 60298,
969
+ 0.9372926149231819
970
+ ],
971
+ [
972
+ 301,
973
+ 215911,
974
+ 0.9372926149231819
975
+ ],
976
+ [
977
+ 301,
978
+ 223657,
979
+ 0.9372926149231819
980
+ ],
981
+ [
982
+ 301,
983
+ 247994,
984
+ 0.9999999999999997
985
+ ],
986
+ [
987
+ 301,
988
+ 353929,
989
+ 0.9576058361634048
990
+ ],
991
+ [
992
+ 301,
993
+ 362416,
994
+ 0.9372926149231819
995
+ ],
996
+ [
997
+ 304,
998
+ 8974,
999
+ 0.9999999999999999
1000
+ ],
1001
+ [
1002
+ 304,
1003
+ 26122,
1004
+ 0.9749817798486549
1005
+ ],
1006
+ [
1007
+ 304,
1008
+ 82553,
1009
+ 0.934346433803283
1010
+ ],
1011
+ [
1012
+ 304,
1013
+ 86792,
1014
+ 0.9999999999999999
1015
+ ],
1016
+ [
1017
+ 304,
1018
+ 192632,
1019
+ 0.943788188023349
1020
+ ],
1021
+ [
1022
+ 304,
1023
+ 284028,
1024
+ 0.9295168026508286
1025
+ ],
1026
+ [
1027
+ 304,
1028
+ 307760,
1029
+ 0.934346433803283
1030
+ ],
1031
+ [
1032
+ 304,
1033
+ 455630,
1034
+ 0.9254717143675608
1035
+ ],
1036
+ [
1037
+ 307,
1038
+ 6305,
1039
+ 0.959750026942162
1040
+ ],
1041
+ [
1042
+ 307,
1043
+ 37455,
1044
+ 0.964302982128182
1045
+ ],
1046
+ [
1047
+ 307,
1048
+ 156089,
1049
+ 0.9227495696258928
1050
+ ],
1051
+ [
1052
+ 307,
1053
+ 245056,
1054
+ 0.9408904740151943
1055
+ ],
1056
+ [
1057
+ 307,
1058
+ 262316,
1059
+ 0.9408904740151943
1060
+ ],
1061
+ [
1062
+ 307,
1063
+ 315982,
1064
+ 0.9494252795262221
1065
+ ],
1066
+ [
1067
+ 307,
1068
+ 385067,
1069
+ 0.9365146411204803
1070
+ ],
1071
+ [
1072
+ 307,
1073
+ 423691,
1074
+ 0.9227495696258928
1075
+ ],
1076
+ [
1077
+ 336,
1078
+ 40075,
1079
+ 0.9697262212841831
1080
+ ],
1081
+ [
1082
+ 336,
1083
+ 200066,
1084
+ 0.9837746609460948
1085
+ ],
1086
+ [
1087
+ 336,
1088
+ 314619,
1089
+ 0.9744844303719529
1090
+ ],
1091
+ [
1092
+ 336,
1093
+ 412444,
1094
+ 0.9625184074487257
1095
+ ],
1096
+ [
1097
+ 337,
1098
+ 23635,
1099
+ 0.9814183293351668
1100
+ ],
1101
+ [
1102
+ 337,
1103
+ 61059,
1104
+ 0.9354801348194209
1105
+ ],
1106
+ [
1107
+ 337,
1108
+ 325248,
1109
+ 0.9263099046034194
1110
+ ],
1111
+ [
1112
+ 337,
1113
+ 389010,
1114
+ 0.988655713760737
1115
+ ],
1116
+ [
1117
+ 337,
1118
+ 397165,
1119
+ 0.9814183293351668
1120
+ ],
1121
+ [
1122
+ 337,
1123
+ 409824,
1124
+ 0.974448339709839
1125
+ ],
1126
+ [
1127
+ 337,
1128
+ 466109,
1129
+ 0.9603372995675837
1130
+ ],
1131
+ [
1132
+ 337,
1133
+ 484841,
1134
+ 0.9814183293351668
1135
+ ],
1136
+ [
1137
+ 337,
1138
+ 491364,
1139
+ 0.9433150677917049
1140
+ ],
1141
+ [
1142
+ 338,
1143
+ 81446,
1144
+ 0.9999999999999998
1145
+ ],
1146
+ [
1147
+ 338,
1148
+ 291958,
1149
+ 0.9360424888772756
1150
+ ],
1151
+ [
1152
+ 338,
1153
+ 373896,
1154
+ 0.9672511562232529
1155
+ ],
1156
+ [
1157
+ 338,
1158
+ 397359,
1159
+ 0.9360424888772756
1160
+ ],
1161
+ [
1162
+ 339,
1163
+ 493555,
1164
+ 0.9461859091389725
1165
+ ],
1166
+ [
1167
+ 341,
1168
+ 37937,
1169
+ 0.9265819842726992
1170
+ ],
1171
+ [
1172
+ 341,
1173
+ 43390,
1174
+ 0.9265819842726992
1175
+ ],
1176
+ [
1177
+ 341,
1178
+ 204600,
1179
+ 0.9558501083264379
1180
+ ],
1181
+ [
1182
+ 341,
1183
+ 235615,
1184
+ 0.9558501083264379
1185
+ ],
1186
+ [
1187
+ 341,
1188
+ 290523,
1189
+ 0.9265819842726992
1190
+ ],
1191
+ [
1192
+ 347,
1193
+ 383973,
1194
+ 0.922261098117815
1195
+ ],
1196
+ [
1197
+ 347,
1198
+ 424048,
1199
+ 0.9529369273315175
1200
+ ],
1201
+ [
1202
+ 347,
1203
+ 443894,
1204
+ 0.9529369273315175
1205
+ ],
1206
+ [
1207
+ 359,
1208
+ 31752,
1209
+ 0.9744347662821284
1210
+ ],
1211
+ [
1212
+ 359,
1213
+ 108007,
1214
+ 0.9608696911310923
1215
+ ],
1216
+ [
1217
+ 359,
1218
+ 111462,
1219
+ 0.9426903120298387
1220
+ ],
1221
+ [
1222
+ 359,
1223
+ 131871,
1224
+ 0.9314665316867591
1225
+ ],
1226
+ [
1227
+ 359,
1228
+ 223848,
1229
+ 0.9744347662821284
1230
+ ],
1231
+ [
1232
+ 359,
1233
+ 436716,
1234
+ 0.9450260166200225
1235
+ ],
1236
+ [
1237
+ 365,
1238
+ 29685,
1239
+ 0.9471376223104829
1240
+ ],
1241
+ [
1242
+ 365,
1243
+ 444646,
1244
+ 0.9359436833197462
1245
+ ],
1246
+ [
1247
+ 365,
1248
+ 450622,
1249
+ 0.932713153035473
1250
+ ],
1251
+ [
1252
+ 370,
1253
+ 35338,
1254
+ 0.9433632355521044
1255
+ ],
1256
+ [
1257
+ 370,
1258
+ 36312,
1259
+ 0.9682380739355729
1260
+ ],
1261
+ [
1262
+ 370,
1263
+ 51282,
1264
+ 0.9725646416198909
1265
+ ],
1266
+ [
1267
+ 370,
1268
+ 139058,
1269
+ 0.9434973363808481
1270
+ ],
1271
+ [
1272
+ 370,
1273
+ 161975,
1274
+ 0.9725646416198909
1275
+ ],
1276
+ [
1277
+ 370,
1278
+ 256346,
1279
+ 0.9411088157331723
1280
+ ],
1281
+ [
1282
+ 370,
1283
+ 374273,
1284
+ 0.9354209157361805
1285
+ ],
1286
+ [
1287
+ 370,
1288
+ 462900,
1289
+ 0.9514869003257086
1290
+ ],
1291
+ [
1292
+ 370,
1293
+ 488240,
1294
+ 0.9725646416198909
1295
+ ],
1296
+ [
1297
+ 379,
1298
+ 172113,
1299
+ 0.9392279608467081
1300
+ ],
1301
+ [
1302
+ 381,
1303
+ 23242,
1304
+ 0.9539342244893797
1305
+ ],
1306
+ [
1307
+ 381,
1308
+ 421600,
1309
+ 0.9799064264471429
1310
+ ],
1311
+ [
1312
+ 381,
1313
+ 454876,
1314
+ 0.921391027392549
1315
+ ],
1316
+ [
1317
+ 387,
1318
+ 1560,
1319
+ 0.9739641319824425
1320
+ ],
1321
+ [
1322
+ 387,
1323
+ 43810,
1324
+ 0.9739641319824425
1325
+ ],
1326
+ [
1327
+ 387,
1328
+ 84718,
1329
+ 0.93672786788419
1330
+ ],
1331
+ [
1332
+ 387,
1333
+ 248019,
1334
+ 0.93672786788419
1335
+ ],
1336
+ [
1337
+ 387,
1338
+ 294216,
1339
+ 0.9233133253058652
1340
+ ],
1341
+ [
1342
+ 387,
1343
+ 486838,
1344
+ 0.9739641319824425
1345
+ ],
1346
+ [
1347
+ 396,
1348
+ 276327,
1349
+ 0.9536532232765201
1350
+ ],
1351
+ [
1352
+ 399,
1353
+ 173979,
1354
+ 0.9534648569313304
1355
+ ],
1356
+ [
1357
+ 410,
1358
+ 70752,
1359
+ 0.9841383553349488
1360
+ ],
1361
+ [
1362
+ 412,
1363
+ 164658,
1364
+ 0.9253074980948955
1365
+ ],
1366
+ [
1367
+ 412,
1368
+ 411712,
1369
+ 0.9506427776933281
1370
+ ],
1371
+ [
1372
+ 419,
1373
+ 173247,
1374
+ 0.9239976493371709
1375
+ ],
1376
+ [
1377
+ 420,
1378
+ 204282,
1379
+ 0.9338483988744172
1380
+ ],
1381
+ [
1382
+ 420,
1383
+ 391056,
1384
+ 0.9338483988744172
1385
+ ],
1386
+ [
1387
+ 422,
1388
+ 324168,
1389
+ 0.9378359131439705
1390
+ ],
1391
+ [
1392
+ 425,
1393
+ 188656,
1394
+ 0.9516210468404523
1395
+ ],
1396
+ [
1397
+ 425,
1398
+ 258927,
1399
+ 0.9516210468404523
1400
+ ],
1401
+ [
1402
+ 425,
1403
+ 373678,
1404
+ 0.9607973608153708
1405
+ ],
1406
+ [
1407
+ 425,
1408
+ 442589,
1409
+ 0.9668875168200395
1410
+ ],
1411
+ [
1412
+ 440,
1413
+ 252530,
1414
+ 0.9429249104115698
1415
+ ],
1416
+ [
1417
+ 440,
1418
+ 356703,
1419
+ 0.9620483029386051
1420
+ ],
1421
+ [
1422
+ 440,
1423
+ 426090,
1424
+ 0.9765770438989231
1425
+ ],
1426
+ [
1427
+ 440,
1428
+ 482848,
1429
+ 0.9765770438989231
1430
+ ],
1431
+ [
1432
+ 442,
1433
+ 116405,
1434
+ 0.9733301211345148
1435
+ ],
1436
+ [
1437
+ 442,
1438
+ 124436,
1439
+ 0.9522671722762315
1440
+ ],
1441
+ [
1442
+ 442,
1443
+ 197088,
1444
+ 0.9344495173394642
1445
+ ],
1446
+ [
1447
+ 442,
1448
+ 477243,
1449
+ 0.9616168095416695
1450
+ ],
1451
+ [
1452
+ 445,
1453
+ 2449,
1454
+ 0.927434301728441
1455
+ ],
1456
+ [
1457
+ 445,
1458
+ 13503,
1459
+ 0.9853721829075641
1460
+ ],
1461
+ [
1462
+ 445,
1463
+ 301647,
1464
+ 0.9765617389455221
1465
+ ],
1466
+ [
1467
+ 445,
1468
+ 395365,
1469
+ 0.927434301728441
1470
+ ],
1471
+ [
1472
+ 445,
1473
+ 457390,
1474
+ 0.9853721829075641
1475
+ ],
1476
+ [
1477
+ 450,
1478
+ 37307,
1479
+ 0.9426316151183709
1480
+ ],
1481
+ [
1482
+ 450,
1483
+ 65947,
1484
+ 0.9999999999999999
1485
+ ],
1486
+ [
1487
+ 450,
1488
+ 129279,
1489
+ 0.9426316151183709
1490
+ ],
1491
+ [
1492
+ 450,
1493
+ 143628,
1494
+ 0.953388728003114
1495
+ ],
1496
+ [
1497
+ 450,
1498
+ 423929,
1499
+ 0.9999999999999999
1500
+ ],
1501
+ [
1502
+ 450,
1503
+ 440800,
1504
+ 0.9426316151183709
1505
+ ],
1506
+ [
1507
+ 450,
1508
+ 490236,
1509
+ 0.9747235065991032
1510
+ ],
1511
+ [
1512
+ 451,
1513
+ 41307,
1514
+ 0.9214497364774421
1515
+ ],
1516
+ [
1517
+ 451,
1518
+ 430332,
1519
+ 0.940827566703477
1520
+ ],
1521
+ [
1522
+ 464,
1523
+ 6364,
1524
+ 0.9401148786198855
1525
+ ],
1526
+ [
1527
+ 464,
1528
+ 18125,
1529
+ 0.9540065520923324
1530
+ ],
1531
+ [
1532
+ 464,
1533
+ 198130,
1534
+ 0.9632350440858664
1535
+ ],
1536
+ [
1537
+ 464,
1538
+ 246444,
1539
+ 0.983484301762338
1540
+ ],
1541
+ [
1542
+ 464,
1543
+ 316067,
1544
+ 0.9632350440858664
1545
+ ],
1546
+ [
1547
+ 464,
1548
+ 332792,
1549
+ 0.9632350440858664
1550
+ ],
1551
+ [
1552
+ 464,
1553
+ 369807,
1554
+ 0.9401148786198855
1555
+ ],
1556
+ [
1557
+ 466,
1558
+ 181947,
1559
+ 0.9566318521885717
1560
+ ],
1561
+ [
1562
+ 467,
1563
+ 30000,
1564
+ 0.952259575684497
1565
+ ],
1566
+ [
1567
+ 467,
1568
+ 43201,
1569
+ 0.9683616054914709
1570
+ ],
1571
+ [
1572
+ 467,
1573
+ 203110,
1574
+ 0.952259575684497
1575
+ ],
1576
+ [
1577
+ 467,
1578
+ 267434,
1579
+ 0.952259575684497
1580
+ ],
1581
+ [
1582
+ 467,
1583
+ 303473,
1584
+ 0.9683616054914709
1585
+ ],
1586
+ [
1587
+ 467,
1588
+ 330250,
1589
+ 0.952259575684497
1590
+ ],
1591
+ [
1592
+ 467,
1593
+ 424450,
1594
+ 0.9683616054914709
1595
+ ],
1596
+ [
1597
+ 468,
1598
+ 42252,
1599
+ 0.9480975242956816
1600
+ ],
1601
+ [
1602
+ 468,
1603
+ 287128,
1604
+ 0.9480975242956816
1605
+ ],
1606
+ [
1607
+ 468,
1608
+ 417942,
1609
+ 0.9480975242956816
1610
+ ],
1611
+ [
1612
+ 469,
1613
+ 92158,
1614
+ 0.9635943753974651
1615
+ ],
1616
+ [
1617
+ 469,
1618
+ 163685,
1619
+ 0.9999999999999994
1620
+ ],
1621
+ [
1622
+ 469,
1623
+ 201385,
1624
+ 0.9635943753974651
1625
+ ],
1626
+ [
1627
+ 469,
1628
+ 216989,
1629
+ 0.9752301282853649
1630
+ ],
1631
+ [
1632
+ 469,
1633
+ 270939,
1634
+ 0.9479071631728484
1635
+ ],
1636
+ [
1637
+ 469,
1638
+ 345453,
1639
+ 0.9752301282853649
1640
+ ],
1641
+ [
1642
+ 469,
1643
+ 360464,
1644
+ 0.9752301282853649
1645
+ ],
1646
+ [
1647
+ 469,
1648
+ 375206,
1649
+ 0.9999999999999994
1650
+ ],
1651
+ [
1652
+ 469,
1653
+ 417146,
1654
+ 0.945842138629035
1655
+ ],
1656
+ [
1657
+ 471,
1658
+ 30784,
1659
+ 0.9321329916389158
1660
+ ],
1661
+ [
1662
+ 471,
1663
+ 73182,
1664
+ 0.9321329916389158
1665
+ ],
1666
+ [
1667
+ 471,
1668
+ 73353,
1669
+ 0.9493627191368135
1670
+ ],
1671
+ [
1672
+ 471,
1673
+ 132072,
1674
+ 0.9695281113893225
1675
+ ],
1676
+ [
1677
+ 471,
1678
+ 165801,
1679
+ 0.9365215571811126
1680
+ ],
1681
+ [
1682
+ 471,
1683
+ 255849,
1684
+ 0.9212387657096248
1685
+ ],
1686
+ [
1687
+ 471,
1688
+ 373278,
1689
+ 0.9695281113893225
1690
+ ],
1691
+ [
1692
+ 471,
1693
+ 389424,
1694
+ 0.9883696144509153
1695
+ ],
1696
+ [
1697
+ 471,
1698
+ 436170,
1699
+ 0.9321329916389158
1700
+ ],
1701
+ [
1702
+ 473,
1703
+ 4306,
1704
+ 0.9430655637318617
1705
+ ],
1706
+ [
1707
+ 473,
1708
+ 130408,
1709
+ 0.9503163808522959
1710
+ ],
1711
+ [
1712
+ 473,
1713
+ 251338,
1714
+ 0.9853070921808301
1715
+ ],
1716
+ [
1717
+ 473,
1718
+ 293610,
1719
+ 0.9503163808522959
1720
+ ],
1721
+ [
1722
+ 473,
1723
+ 385150,
1724
+ 0.9615104609524749
1725
+ ],
1726
+ [
1727
+ 473,
1728
+ 386213,
1729
+ 0.9615104609524749
1730
+ ],
1731
+ [
1732
+ 473,
1733
+ 471818,
1734
+ 0.9853070921808301
1735
+ ],
1736
+ [
1737
+ 480,
1738
+ 91640,
1739
+ 0.9210418216838656
1740
+ ],
1741
+ [
1742
+ 480,
1743
+ 449657,
1744
+ 0.9210418216838656
1745
+ ],
1746
+ [
1747
+ 480,
1748
+ 472778,
1749
+ 0.9210418216838656
1750
+ ],
1751
+ [
1752
+ 480,
1753
+ 487911,
1754
+ 0.9387942229449984
1755
+ ],
1756
+ [
1757
+ 484,
1758
+ 65723,
1759
+ 0.9520062350594526
1760
+ ],
1761
+ [
1762
+ 484,
1763
+ 111314,
1764
+ 0.9412941079863979
1765
+ ],
1766
+ [
1767
+ 484,
1768
+ 212902,
1769
+ 0.9449333465530395
1770
+ ],
1771
+ [
1772
+ 484,
1773
+ 299743,
1774
+ 0.9999999999999989
1775
+ ],
1776
+ [
1777
+ 484,
1778
+ 303506,
1779
+ 0.9449333465530395
1780
+ ],
1781
+ [
1782
+ 496,
1783
+ 56812,
1784
+ 0.9300335600887769
1785
+ ],
1786
+ [
1787
+ 498,
1788
+ 52539,
1789
+ 0.972596088714569
1790
+ ],
1791
+ [
1792
+ 498,
1793
+ 145695,
1794
+ 0.9999999999999998
1795
+ ],
1796
+ [
1797
+ 498,
1798
+ 172451,
1799
+ 0.972596088714569
1800
+ ],
1801
+ [
1802
+ 498,
1803
+ 325555,
1804
+ 0.9687416755126035
1805
+ ],
1806
+ [
1807
+ 498,
1808
+ 344015,
1809
+ 0.9612150141854572
1810
+ ],
1811
+ [
1812
+ 498,
1813
+ 455498,
1814
+ 0.9245389138787695
1815
+ ],
1816
+ [
1817
+ 498,
1818
+ 475881,
1819
+ 0.9412647540471755
1820
+ ],
1821
+ [
1822
+ 498,
1823
+ 488309,
1824
+ 0.950953721198859
1825
+ ],
1826
+ [
1827
+ 502,
1828
+ 121115,
1829
+ 0.9562840420651029
1830
+ ],
1831
+ [
1832
+ 502,
1833
+ 293075,
1834
+ 0.9562840420651029
1835
+ ],
1836
+ [
1837
+ 502,
1838
+ 484052,
1839
+ 0.9584700957570479
1840
+ ],
1841
+ [
1842
+ 517,
1843
+ 444503,
1844
+ 0.9476943177960863
1845
+ ],
1846
+ [
1847
+ 521,
1848
+ 1571,
1849
+ 0.9718188506381806
1850
+ ],
1851
+ [
1852
+ 521,
1853
+ 33925,
1854
+ 0.9262914492612428
1855
+ ],
1856
+ [
1857
+ 521,
1858
+ 60837,
1859
+ 0.9999999999999998
1860
+ ],
1861
+ [
1862
+ 521,
1863
+ 125046,
1864
+ 0.9999999999999998
1865
+ ],
1866
+ [
1867
+ 521,
1868
+ 132661,
1869
+ 0.9999999999999998
1870
+ ],
1871
+ [
1872
+ 521,
1873
+ 183652,
1874
+ 0.927710245210915
1875
+ ],
1876
+ [
1877
+ 521,
1878
+ 221838,
1879
+ 0.9355017319762571
1880
+ ],
1881
+ [
1882
+ 521,
1883
+ 268131,
1884
+ 0.9718188506381806
1885
+ ],
1886
+ [
1887
+ 521,
1888
+ 309600,
1889
+ 0.9718188506381806
1890
+ ],
1891
+ [
1892
+ 521,
1893
+ 480360,
1894
+ 0.956133453017757
1895
+ ],
1896
+ [
1897
+ 525,
1898
+ 133312,
1899
+ 0.9614432649194411
1900
+ ],
1901
+ [
1902
+ 525,
1903
+ 272813,
1904
+ 0.9614432649194411
1905
+ ],
1906
+ [
1907
+ 525,
1908
+ 308469,
1909
+ 0.9614432649194411
1910
+ ],
1911
+ [
1912
+ 525,
1913
+ 494429,
1914
+ 0.9614432649194411
1915
+ ],
1916
+ [
1917
+ 527,
1918
+ 118170,
1919
+ 0.9466729097965849
1920
+ ],
1921
+ [
1922
+ 529,
1923
+ 122417,
1924
+ 0.9416990873705339
1925
+ ],
1926
+ [
1927
+ 529,
1928
+ 201529,
1929
+ 0.9416990873705339
1930
+ ],
1931
+ [
1932
+ 529,
1933
+ 263176,
1934
+ 0.9616232856001878
1935
+ ],
1936
+ [
1937
+ 529,
1938
+ 287259,
1939
+ 0.9616232856001878
1940
+ ],
1941
+ [
1942
+ 529,
1943
+ 358003,
1944
+ 0.9616232856001878
1945
+ ],
1946
+ [
1947
+ 532,
1948
+ 81070,
1949
+ 0.95493603878535
1950
+ ],
1951
+ [
1952
+ 538,
1953
+ 271905,
1954
+ 0.9374364540448189
1955
+ ],
1956
+ [
1957
+ 538,
1958
+ 414004,
1959
+ 0.9374364540448189
1960
+ ],
1961
+ [
1962
+ 541,
1963
+ 446323,
1964
+ 0.9281237198269422
1965
+ ],
1966
+ [
1967
+ 550,
1968
+ 403686,
1969
+ 0.9999999999999999
1970
+ ],
1971
+ [
1972
+ 550,
1973
+ 439602,
1974
+ 0.9999999999999999
1975
+ ],
1976
+ [
1977
+ 559,
1978
+ 147517,
1979
+ 0.931799534763926
1980
+ ],
1981
+ [
1982
+ 559,
1983
+ 190664,
1984
+ 0.931799534763926
1985
+ ],
1986
+ [
1987
+ 559,
1988
+ 403031,
1989
+ 0.931799534763926
1990
+ ],
1991
+ [
1992
+ 563,
1993
+ 51995,
1994
+ 0.9750896972199731
1995
+ ],
1996
+ [
1997
+ 563,
1998
+ 131453,
1999
+ 0.9604983972679649
2000
+ ],
2001
+ [
2002
+ 563,
2003
+ 137522,
2004
+ 0.9346440237254808
2005
+ ],
2006
+ [
2007
+ 563,
2008
+ 208269,
2009
+ 0.9346440237254808
2010
+ ],
2011
+ [
2012
+ 563,
2013
+ 218759,
2014
+ 0.9604983972679649
2015
+ ],
2016
+ [
2017
+ 563,
2018
+ 219410,
2019
+ 0.9346440237254808
2020
+ ],
2021
+ [
2022
+ 563,
2023
+ 429715,
2024
+ 0.9604983972679649
2025
+ ],
2026
+ [
2027
+ 579,
2028
+ 130159,
2029
+ 0.9232119965779378
2030
+ ],
2031
+ [
2032
+ 579,
2033
+ 138317,
2034
+ 0.9232119965779378
2035
+ ],
2036
+ [
2037
+ 579,
2038
+ 316761,
2039
+ 0.9341039350812795
2040
+ ],
2041
+ [
2042
+ 579,
2043
+ 440179,
2044
+ 0.9232119965779378
2045
+ ],
2046
+ [
2047
+ 580,
2048
+ 10766,
2049
+ 0.9999999999999997
2050
+ ],
2051
+ [
2052
+ 580,
2053
+ 35321,
2054
+ 0.9585120705250577
2055
+ ],
2056
+ [
2057
+ 580,
2058
+ 38712,
2059
+ 0.9491437313410578
2060
+ ],
2061
+ [
2062
+ 580,
2063
+ 179988,
2064
+ 0.9812303884020132
2065
+ ],
2066
+ [
2067
+ 580,
2068
+ 327215,
2069
+ 0.9664686594072341
2070
+ ],
2071
+ [
2072
+ 580,
2073
+ 344739,
2074
+ 0.9776871245819292
2075
+ ],
2076
+ [
2077
+ 580,
2078
+ 401174,
2079
+ 0.9237676423002295
2080
+ ],
2081
+ [
2082
+ 580,
2083
+ 412080,
2084
+ 0.9999999999999997
2085
+ ],
2086
+ [
2087
+ 580,
2088
+ 416726,
2089
+ 0.9607947710720334
2090
+ ],
2091
+ [
2092
+ 580,
2093
+ 455428,
2094
+ 0.9999999999999997
2095
+ ],
2096
+ [
2097
+ 591,
2098
+ 13382,
2099
+ 0.9829904683714399
2100
+ ],
2101
+ [
2102
+ 591,
2103
+ 33917,
2104
+ 0.9284172302096665
2105
+ ],
2106
+ [
2107
+ 591,
2108
+ 56949,
2109
+ 0.970437847169916
2110
+ ],
2111
+ [
2112
+ 591,
2113
+ 79459,
2114
+ 0.9676926738553399
2115
+ ],
2116
+ [
2117
+ 591,
2118
+ 232215,
2119
+ 0.9284172302096665
2120
+ ],
2121
+ [
2122
+ 591,
2123
+ 234545,
2124
+ 0.9999999999999997
2125
+ ],
2126
+ [
2127
+ 591,
2128
+ 401138,
2129
+ 0.970437847169916
2130
+ ],
2131
+ [
2132
+ 591,
2133
+ 459063,
2134
+ 0.970437847169916
2135
+ ],
2136
+ [
2137
+ 599,
2138
+ 57644,
2139
+ 0.9669178035093177
2140
+ ],
2141
+ [
2142
+ 599,
2143
+ 120554,
2144
+ 0.9536537664494986
2145
+ ],
2146
+ [
2147
+ 599,
2148
+ 229128,
2149
+ 0.921807554857117
2150
+ ],
2151
+ [
2152
+ 599,
2153
+ 278502,
2154
+ 0.9669178035093177
2155
+ ],
2156
+ [
2157
+ 599,
2158
+ 315597,
2159
+ 0.921807554857117
2160
+ ],
2161
+ [
2162
+ 599,
2163
+ 351032,
2164
+ 0.9293080445807788
2165
+ ],
2166
+ [
2167
+ 601,
2168
+ 123015,
2169
+ 0.9314634777196564
2170
+ ],
2171
+ [
2172
+ 601,
2173
+ 143186,
2174
+ 0.9436154290295105
2175
+ ],
2176
+ [
2177
+ 601,
2178
+ 251276,
2179
+ 0.9436154290295105
2180
+ ],
2181
+ [
2182
+ 609,
2183
+ 37014,
2184
+ 0.9579507188545295
2185
+ ],
2186
+ [
2187
+ 609,
2188
+ 41302,
2189
+ 0.9332447571340635
2190
+ ],
2191
+ [
2192
+ 609,
2193
+ 42813,
2194
+ 0.9579507188545295
2195
+ ],
2196
+ [
2197
+ 609,
2198
+ 263436,
2199
+ 0.9579507188545295
2200
+ ],
2201
+ [
2202
+ 621,
2203
+ 41449,
2204
+ 0.9597018880383841
2205
+ ],
2206
+ [
2207
+ 621,
2208
+ 467418,
2209
+ 0.9379121569853
2210
+ ],
2211
+ [
2212
+ 627,
2213
+ 127598,
2214
+ 0.9573577760070149
2215
+ ],
2216
+ [
2217
+ 627,
2218
+ 158266,
2219
+ 0.9252747502844678
2220
+ ],
2221
+ [
2222
+ 627,
2223
+ 182826,
2224
+ 0.9690977765542894
2225
+ ],
2226
+ [
2227
+ 627,
2228
+ 390949,
2229
+ 0.9344737189686023
2230
+ ],
2231
+ [
2232
+ 635,
2233
+ 102571,
2234
+ 0.9465765395580519
2235
+ ],
2236
+ [
2237
+ 635,
2238
+ 284977,
2239
+ 0.9215203688782141
2240
+ ],
2241
+ [
2242
+ 635,
2243
+ 394569,
2244
+ 0.9633432947461982
2245
+ ],
2246
+ [
2247
+ 635,
2248
+ 483282,
2249
+ 0.9765441882590518
2250
+ ],
2251
+ [
2252
+ 636,
2253
+ 211792,
2254
+ 0.9611750771367156
2255
+ ],
2256
+ [
2257
+ 636,
2258
+ 336296,
2259
+ 0.9611750771367156
2260
+ ],
2261
+ [
2262
+ 636,
2263
+ 383655,
2264
+ 0.9361675316997123
2265
+ ],
2266
+ [
2267
+ 636,
2268
+ 385407,
2269
+ 0.9611750771367156
2270
+ ],
2271
+ [
2272
+ 642,
2273
+ 162407,
2274
+ 0.967121184003425
2275
+ ],
2276
+ [
2277
+ 642,
2278
+ 203589,
2279
+ 0.9264488095836839
2280
+ ],
2281
+ [
2282
+ 642,
2283
+ 290908,
2284
+ 0.9264488095836839
2285
+ ],
2286
+ [
2287
+ 642,
2288
+ 303154,
2289
+ 0.9399536792211226
2290
+ ],
2291
+ [
2292
+ 645,
2293
+ 54416,
2294
+ 0.946586451406337
2295
+ ],
2296
+ [
2297
+ 645,
2298
+ 288625,
2299
+ 0.946586451406337
2300
+ ],
2301
+ [
2302
+ 645,
2303
+ 464283,
2304
+ 0.9220982797445217
2305
+ ],
2306
+ [
2307
+ 645,
2308
+ 497165,
2309
+ 0.9220982797445217
2310
+ ],
2311
+ [
2312
+ 649,
2313
+ 401697,
2314
+ 0.9539775965848976
2315
+ ],
2316
+ [
2317
+ 659,
2318
+ 39851,
2319
+ 0.9317601391400236
2320
+ ],
2321
+ [
2322
+ 660,
2323
+ 113791,
2324
+ 0.9518077901425306
2325
+ ],
2326
+ [
2327
+ 660,
2328
+ 146041,
2329
+ 0.9581165061802294
2330
+ ],
2331
+ [
2332
+ 660,
2333
+ 193211,
2334
+ 0.9400179635916767
2335
+ ],
2336
+ [
2337
+ 660,
2338
+ 220967,
2339
+ 0.9518077901425306
2340
+ ],
2341
+ [
2342
+ 660,
2343
+ 242046,
2344
+ 0.9518077901425306
2345
+ ],
2346
+ [
2347
+ 660,
2348
+ 339482,
2349
+ 0.9672478953023959
2350
+ ],
2351
+ [
2352
+ 660,
2353
+ 425057,
2354
+ 0.957344377972823
2355
+ ],
2356
+ [
2357
+ 667,
2358
+ 118594,
2359
+ 0.9261845126316526
2360
+ ],
2361
+ [
2362
+ 667,
2363
+ 172007,
2364
+ 0.9560887130877913
2365
+ ],
2366
+ [
2367
+ 667,
2368
+ 189891,
2369
+ 0.9261845126316526
2370
+ ],
2371
+ [
2372
+ 667,
2373
+ 235834,
2374
+ 0.9261845126316526
2375
+ ],
2376
+ [
2377
+ 667,
2378
+ 281881,
2379
+ 0.9261845126316526
2380
+ ],
2381
+ [
2382
+ 676,
2383
+ 193373,
2384
+ 0.9615405575044855
2385
+ ],
2386
+ [
2387
+ 676,
2388
+ 200314,
2389
+ 0.9615405575044855
2390
+ ],
2391
+ [
2392
+ 676,
2393
+ 258922,
2394
+ 0.9479193278068535
2395
+ ],
2396
+ [
2397
+ 676,
2398
+ 262619,
2399
+ 0.938077428223345
2400
+ ],
2401
+ [
2402
+ 676,
2403
+ 298705,
2404
+ 0.9615405575044855
2405
+ ],
2406
+ [
2407
+ 679,
2408
+ 31144,
2409
+ 0.9281435857788473
2410
+ ],
2411
+ [
2412
+ 684,
2413
+ 43579,
2414
+ 0.9531612588795209
2415
+ ],
2416
+ [
2417
+ 684,
2418
+ 153288,
2419
+ 0.9427846369721121
2420
+ ],
2421
+ [
2422
+ 684,
2423
+ 171769,
2424
+ 0.9531612588795209
2425
+ ],
2426
+ [
2427
+ 685,
2428
+ 61182,
2429
+ 0.9271722561726765
2430
+ ],
2431
+ [
2432
+ 685,
2433
+ 155543,
2434
+ 0.9826988641525762
2435
+ ],
2436
+ [
2437
+ 685,
2438
+ 332673,
2439
+ 0.9271722561726765
2440
+ ],
2441
+ [
2442
+ 685,
2443
+ 336822,
2444
+ 0.9271722561726765
2445
+ ],
2446
+ [
2447
+ 685,
2448
+ 400887,
2449
+ 0.9271722561726765
2450
+ ],
2451
+ [
2452
+ 685,
2453
+ 449968,
2454
+ 0.9416328586741354
2455
+ ],
2456
+ [
2457
+ 690,
2458
+ 40213,
2459
+ 0.9718823311291929
2460
+ ],
2461
+ [
2462
+ 690,
2463
+ 91377,
2464
+ 0.930207956147908
2465
+ ],
2466
+ [
2467
+ 690,
2468
+ 137106,
2469
+ 0.9526321594006405
2470
+ ],
2471
+ [
2472
+ 690,
2473
+ 260596,
2474
+ 0.9718823311291929
2475
+ ],
2476
+ [
2477
+ 690,
2478
+ 326301,
2479
+ 0.9526321594006405
2480
+ ],
2481
+ [
2482
+ 690,
2483
+ 382968,
2484
+ 0.9388141665779639
2485
+ ],
2486
+ [
2487
+ 690,
2488
+ 405473,
2489
+ 0.9526321594006405
2490
+ ],
2491
+ [
2492
+ 690,
2493
+ 431831,
2494
+ 0.9718823311291929
2495
+ ],
2496
+ [
2497
+ 690,
2498
+ 450335,
2499
+ 0.981301567084865
2500
+ ],
2501
+ [
2502
+ 691,
2503
+ 174357,
2504
+ 0.9670956862613862
2505
+ ],
2506
+ [
2507
+ 691,
2508
+ 214065,
2509
+ 0.9670956862613862
2510
+ ],
2511
+ [
2512
+ 691,
2513
+ 237890,
2514
+ 0.9427641750678423
2515
+ ],
2516
+ [
2517
+ 691,
2518
+ 242159,
2519
+ 0.9427641750678423
2520
+ ],
2521
+ [
2522
+ 691,
2523
+ 339953,
2524
+ 0.9670956862613862
2525
+ ],
2526
+ [
2527
+ 691,
2528
+ 412344,
2529
+ 0.9427641750678423
2530
+ ],
2531
+ [
2532
+ 692,
2533
+ 136805,
2534
+ 0.9255160274697998
2535
+ ],
2536
+ [
2537
+ 692,
2538
+ 286070,
2539
+ 0.922636522225936
2540
+ ],
2541
+ [
2542
+ 702,
2543
+ 271620,
2544
+ 0.9477385285411166
2545
+ ],
2546
+ [
2547
+ 705,
2548
+ 36433,
2549
+ 0.9590703963164325
2550
+ ],
2551
+ [
2552
+ 705,
2553
+ 116959,
2554
+ 0.9590703963164325
2555
+ ],
2556
+ [
2557
+ 705,
2558
+ 144321,
2559
+ 0.9669072472893679
2560
+ ],
2561
+ [
2562
+ 705,
2563
+ 221551,
2564
+ 0.9720894776671123
2565
+ ],
2566
+ [
2567
+ 705,
2568
+ 289824,
2569
+ 0.9425626856804971
2570
+ ],
2571
+ [
2572
+ 705,
2573
+ 359921,
2574
+ 0.9265972320533125
2575
+ ],
2576
+ [
2577
+ 705,
2578
+ 488101,
2579
+ 0.9590703963164325
2580
+ ],
2581
+ [
2582
+ 709,
2583
+ 275057,
2584
+ 0.9548670091104149
2585
+ ],
2586
+ [
2587
+ 715,
2588
+ 4180,
2589
+ 0.9675523005880733
2590
+ ],
2591
+ [
2592
+ 715,
2593
+ 16769,
2594
+ 0.95497317840821
2595
+ ],
2596
+ [
2597
+ 715,
2598
+ 169811,
2599
+ 0.9675523005880733
2600
+ ],
2601
+ [
2602
+ 715,
2603
+ 209171,
2604
+ 0.9268210015095936
2605
+ ],
2606
+ [
2607
+ 715,
2608
+ 249732,
2609
+ 0.9675523005880733
2610
+ ],
2611
+ [
2612
+ 715,
2613
+ 285825,
2614
+ 0.9477150907855315
2615
+ ],
2616
+ [
2617
+ 715,
2618
+ 290224,
2619
+ 0.9251100138635445
2620
+ ],
2621
+ [
2622
+ 715,
2623
+ 316702,
2624
+ 0.9477150907855315
2625
+ ],
2626
+ [
2627
+ 715,
2628
+ 343281,
2629
+ 0.9251100138635445
2630
+ ],
2631
+ [
2632
+ 715,
2633
+ 384938,
2634
+ 0.9251100138635445
2635
+ ],
2636
+ [
2637
+ 715,
2638
+ 386791,
2639
+ 0.9383068771351176
2640
+ ],
2641
+ [
2642
+ 716,
2643
+ 35219,
2644
+ 0.9782048905678163
2645
+ ],
2646
+ [
2647
+ 716,
2648
+ 272953,
2649
+ 0.9214461878082616
2650
+ ],
2651
+ [
2652
+ 716,
2653
+ 282138,
2654
+ 0.9366259161623623
2655
+ ],
2656
+ [
2657
+ 741,
2658
+ 227714,
2659
+ 0.9319773284311021
2660
+ ],
2661
+ [
2662
+ 744,
2663
+ 253295,
2664
+ 0.9711979134593742
2665
+ ],
2666
+ [
2667
+ 744,
2668
+ 331028,
2669
+ 0.928464712430442
2670
+ ],
2671
+ [
2672
+ 749,
2673
+ 214585,
2674
+ 0.9284852584350799
2675
+ ],
2676
+ [
2677
+ 749,
2678
+ 277800,
2679
+ 0.9536893182875994
2680
+ ],
2681
+ [
2682
+ 754,
2683
+ 4442,
2684
+ 0.9841895110594914
2685
+ ],
2686
+ [
2687
+ 754,
2688
+ 82019,
2689
+ 0.9634467400241903
2690
+ ],
2691
+ [
2692
+ 754,
2693
+ 221023,
2694
+ 0.9456285312006552
2695
+ ],
2696
+ [
2697
+ 754,
2698
+ 363817,
2699
+ 0.9634467400241903
2700
+ ],
2701
+ [
2702
+ 754,
2703
+ 408976,
2704
+ 0.9411443475057498
2705
+ ],
2706
+ [
2707
+ 758,
2708
+ 5382,
2709
+ 0.9538184762665407
2710
+ ],
2711
+ [
2712
+ 758,
2713
+ 96830,
2714
+ 0.9444373493460652
2715
+ ],
2716
+ [
2717
+ 758,
2718
+ 119392,
2719
+ 0.9538184762665407
2720
+ ],
2721
+ [
2722
+ 758,
2723
+ 150068,
2724
+ 0.9798538685312934
2725
+ ],
2726
+ [
2727
+ 758,
2728
+ 307654,
2729
+ 0.9626024516470488
2730
+ ],
2731
+ [
2732
+ 758,
2733
+ 338027,
2734
+ 0.969744415183304
2735
+ ],
2736
+ [
2737
+ 758,
2738
+ 391586,
2739
+ 0.9354034257254474
2740
+ ],
2741
+ [
2742
+ 759,
2743
+ 8925,
2744
+ 0.9365921631452621
2745
+ ],
2746
+ [
2747
+ 759,
2748
+ 126638,
2749
+ 0.9525547644161662
2750
+ ],
2751
+ [
2752
+ 759,
2753
+ 197344,
2754
+ 0.9228111607945921
2755
+ ],
2756
+ [
2757
+ 759,
2758
+ 428924,
2759
+ 0.9525547644161662
2760
+ ],
2761
+ [
2762
+ 760,
2763
+ 46828,
2764
+ 0.979150776968018
2765
+ ],
2766
+ [
2767
+ 760,
2768
+ 106786,
2769
+ 0.9353864956162284
2770
+ ],
2771
+ [
2772
+ 760,
2773
+ 195490,
2774
+ 0.9228470300493653
2775
+ ],
2776
+ [
2777
+ 768,
2778
+ 374481,
2779
+ 0.9340060335421336
2780
+ ],
2781
+ [
2782
+ 768,
2783
+ 437462,
2784
+ 0.9745119736364606
2785
+ ],
2786
+ [
2787
+ 774,
2788
+ 273565,
2789
+ 0.9248798134211723
2790
+ ],
2791
+ [
2792
+ 778,
2793
+ 42086,
2794
+ 0.9597679950190163
2795
+ ],
2796
+ [
2797
+ 778,
2798
+ 56337,
2799
+ 0.9639186845651324
2800
+ ],
2801
+ [
2802
+ 778,
2803
+ 104121,
2804
+ 0.9635764795879889
2805
+ ],
2806
+ [
2807
+ 778,
2808
+ 119982,
2809
+ 0.9358391579808851
2810
+ ],
2811
+ [
2812
+ 778,
2813
+ 147477,
2814
+ 0.9233465414533832
2815
+ ],
2816
+ [
2817
+ 778,
2818
+ 203844,
2819
+ 0.9221781859121966
2820
+ ],
2821
+ [
2822
+ 778,
2823
+ 208090,
2824
+ 0.9639186845651324
2825
+ ],
2826
+ [
2827
+ 778,
2828
+ 282804,
2829
+ 0.9525596395142646
2830
+ ],
2831
+ [
2832
+ 778,
2833
+ 386918,
2834
+ 0.9221781859121966
2835
+ ],
2836
+ [
2837
+ 778,
2838
+ 416000,
2839
+ 0.9639186845651324
2840
+ ],
2841
+ [
2842
+ 778,
2843
+ 424221,
2844
+ 0.9639186845651324
2845
+ ],
2846
+ [
2847
+ 778,
2848
+ 473151,
2849
+ 0.9525596395142646
2850
+ ],
2851
+ [
2852
+ 778,
2853
+ 481957,
2854
+ 0.9999999999999999
2855
+ ],
2856
+ [
2857
+ 779,
2858
+ 331920,
2859
+ 0.9318294679196344
2860
+ ],
2861
+ [
2862
+ 779,
2863
+ 468579,
2864
+ 0.9318294679196344
2865
+ ],
2866
+ [
2867
+ 786,
2868
+ 2107,
2869
+ 0.975199230502239
2870
+ ],
2871
+ [
2872
+ 786,
2873
+ 7078,
2874
+ 0.975199230502239
2875
+ ],
2876
+ [
2877
+ 786,
2878
+ 58275,
2879
+ 0.975199230502239
2880
+ ],
2881
+ [
2882
+ 786,
2883
+ 84705,
2884
+ 0.9338270169066532
2885
+ ],
2886
+ [
2887
+ 786,
2888
+ 87325,
2889
+ 0.9635497703754616
2890
+ ],
2891
+ [
2892
+ 786,
2893
+ 407263,
2894
+ 0.9560096865392987
2895
+ ],
2896
+ [
2897
+ 793,
2898
+ 194449,
2899
+ 0.9501865849873344
2900
+ ],
2901
+ [
2902
+ 793,
2903
+ 213668,
2904
+ 0.9753788918674515
2905
+ ],
2906
+ [
2907
+ 793,
2908
+ 216090,
2909
+ 0.9753788918674515
2910
+ ],
2911
+ [
2912
+ 793,
2913
+ 220995,
2914
+ 0.9753788918674515
2915
+ ],
2916
+ [
2917
+ 793,
2918
+ 439124,
2919
+ 0.9307480577182374
2920
+ ],
2921
+ [
2922
+ 794,
2923
+ 382518,
2924
+ 0.9999999999999999
2925
+ ],
2926
+ [
2927
+ 794,
2928
+ 476502,
2929
+ 0.9999999999999999
2930
+ ],
2931
+ [
2932
+ 799,
2933
+ 132574,
2934
+ 0.9707457244211244
2935
+ ],
2936
+ [
2937
+ 799,
2938
+ 141462,
2939
+ 0.9212418127490455
2940
+ ],
2941
+ [
2942
+ 799,
2943
+ 156191,
2944
+ 0.9707457244211244
2945
+ ],
2946
+ [
2947
+ 799,
2948
+ 354987,
2949
+ 0.9328237736494385
2950
+ ],
2951
+ [
2952
+ 799,
2953
+ 441806,
2954
+ 0.9328237736494385
2955
+ ],
2956
+ [
2957
+ 799,
2958
+ 481229,
2959
+ 0.9328237736494385
2960
+ ],
2961
+ [
2962
+ 801,
2963
+ 21545,
2964
+ 0.932084477941573
2965
+ ],
2966
+ [
2967
+ 801,
2968
+ 177392,
2969
+ 0.9754559648668539
2970
+ ],
2971
+ [
2972
+ 802,
2973
+ 440766,
2974
+ 0.9538339105858277
2975
+ ],
2976
+ [
2977
+ 806,
2978
+ 417688,
2979
+ 0.9272600050807871
2980
+ ],
2981
+ [
2982
+ 817,
2983
+ 190351,
2984
+ 0.9831451484986194
2985
+ ],
2986
+ [
2987
+ 817,
2988
+ 248819,
2989
+ 0.9531048048967303
2990
+ ],
2991
+ [
2992
+ 817,
2993
+ 313650,
2994
+ 0.9685775677776245
2995
+ ],
2996
+ [
2997
+ 817,
2998
+ 316989,
2999
+ 0.9611125926276082
3000
+ ],
3001
+ [
3002
+ 817,
3003
+ 427630,
3004
+ 0.9471299886858444
3005
+ ],
3006
+ [
3007
+ 826,
3008
+ 220685,
3009
+ 0.9217346765686623
3010
+ ],
3011
+ [
3012
+ 826,
3013
+ 339390,
3014
+ 0.9217346765686623
3015
+ ],
3016
+ [
3017
+ 830,
3018
+ 45606,
3019
+ 0.9999999999999997
3020
+ ],
3021
+ [
3022
+ 830,
3023
+ 111168,
3024
+ 0.9760856635319659
3025
+ ],
3026
+ [
3027
+ 830,
3028
+ 120173,
3029
+ 0.9999999999999997
3030
+ ],
3031
+ [
3032
+ 830,
3033
+ 226427,
3034
+ 0.9999999999999997
3035
+ ],
3036
+ [
3037
+ 830,
3038
+ 408153,
3039
+ 0.9360492907548799
3040
+ ],
3041
+ [
3042
+ 830,
3043
+ 415848,
3044
+ 0.9511626571488083
3045
+ ],
3046
+ [
3047
+ 830,
3048
+ 446382,
3049
+ 0.9829165738050829
3050
+ ],
3051
+ [
3052
+ 830,
3053
+ 498596,
3054
+ 0.9760856635319659
3055
+ ],
3056
+ [
3057
+ 832,
3058
+ 347222,
3059
+ 0.96899355288454
3060
+ ],
3061
+ [
3062
+ 833,
3063
+ 315466,
3064
+ 0.9238346922867497
3065
+ ],
3066
+ [
3067
+ 841,
3068
+ 20455,
3069
+ 0.9752222771581552
3070
+ ],
3071
+ [
3072
+ 841,
3073
+ 374182,
3074
+ 0.9752222771581552
3075
+ ],
3076
+ [
3077
+ 843,
3078
+ 10094,
3079
+ 0.9351656969120601
3080
+ ],
3081
+ [
3082
+ 843,
3083
+ 20453,
3084
+ 0.9284889832239328
3085
+ ],
3086
+ [
3087
+ 843,
3088
+ 81776,
3089
+ 0.9704684023117883
3090
+ ],
3091
+ [
3092
+ 843,
3093
+ 97946,
3094
+ 0.9591080681735997
3095
+ ],
3096
+ [
3097
+ 843,
3098
+ 98677,
3099
+ 0.9284889832239328
3100
+ ],
3101
+ [
3102
+ 843,
3103
+ 119150,
3104
+ 0.9284889832239328
3105
+ ],
3106
+ [
3107
+ 843,
3108
+ 230988,
3109
+ 0.9284889832239328
3110
+ ],
3111
+ [
3112
+ 843,
3113
+ 270682,
3114
+ 0.9284889832239328
3115
+ ],
3116
+ [
3117
+ 848,
3118
+ 316047,
3119
+ 0.9740311305796308
3120
+ ],
3121
+ [
3122
+ 850,
3123
+ 43324,
3124
+ 0.9806902404157589
3125
+ ],
3126
+ [
3127
+ 850,
3128
+ 346955,
3129
+ 0.9226532268506542
3130
+ ],
3131
+ [
3132
+ 851,
3133
+ 118714,
3134
+ 0.9378222630191263
3135
+ ],
3136
+ [
3137
+ 851,
3138
+ 283881,
3139
+ 0.9378222630191263
3140
+ ],
3141
+ [
3142
+ 851,
3143
+ 375529,
3144
+ 0.9378222630191263
3145
+ ],
3146
+ [
3147
+ 851,
3148
+ 413113,
3149
+ 0.9378222630191263
3150
+ ],
3151
+ [
3152
+ 859,
3153
+ 88150,
3154
+ 0.9571870790809635
3155
+ ],
3156
+ [
3157
+ 861,
3158
+ 421939,
3159
+ 0.9832937772512729
3160
+ ],
3161
+ [
3162
+ 862,
3163
+ 55218,
3164
+ 0.9217593567637231
3165
+ ],
3166
+ [
3167
+ 862,
3168
+ 275605,
3169
+ 0.9453162936251254
3170
+ ],
3171
+ [
3172
+ 863,
3173
+ 132253,
3174
+ 0.9999999999999999
3175
+ ],
3176
+ [
3177
+ 863,
3178
+ 135014,
3179
+ 0.9999999999999999
3180
+ ],
3181
+ [
3182
+ 863,
3183
+ 164703,
3184
+ 0.9999999999999999
3185
+ ],
3186
+ [
3187
+ 863,
3188
+ 186753,
3189
+ 0.9408574911731076
3190
+ ],
3191
+ [
3192
+ 863,
3193
+ 254690,
3194
+ 0.9202858840243239
3195
+ ],
3196
+ [
3197
+ 863,
3198
+ 339046,
3199
+ 0.9750549027133834
3200
+ ],
3201
+ [
3202
+ 863,
3203
+ 360696,
3204
+ 0.9221427423557218
3205
+ ],
3206
+ [
3207
+ 863,
3208
+ 427895,
3209
+ 0.9317336431712655
3210
+ ],
3211
+ [
3212
+ 863,
3213
+ 434189,
3214
+ 0.9531235023726135
3215
+ ],
3216
+ [
3217
+ 872,
3218
+ 175152,
3219
+ 0.9377657506297079
3220
+ ],
3221
+ [
3222
+ 872,
3223
+ 325197,
3224
+ 0.9377657506297079
3225
+ ],
3226
+ [
3227
+ 875,
3228
+ 303246,
3229
+ 0.9310387965233773
3230
+ ],
3231
+ [
3232
+ 880,
3233
+ 120314,
3234
+ 0.9821778928713346
3235
+ ],
3236
+ [
3237
+ 880,
3238
+ 209610,
3239
+ 0.951815800829736
3240
+ ],
3241
+ [
3242
+ 880,
3243
+ 224727,
3244
+ 0.951815800829736
3245
+ ],
3246
+ [
3247
+ 880,
3248
+ 244835,
3249
+ 0.951815800829736
3250
+ ],
3251
+ [
3252
+ 880,
3253
+ 369939,
3254
+ 0.9821778928713346
3255
+ ],
3256
+ [
3257
+ 883,
3258
+ 79240,
3259
+ 0.9880267741824649
3260
+ ],
3261
+ [
3262
+ 883,
3263
+ 80091,
3264
+ 0.9686208152571203
3265
+ ],
3266
+ [
3267
+ 883,
3268
+ 134349,
3269
+ 0.9405556527105055
3270
+ ],
3271
+ [
3272
+ 883,
3273
+ 170889,
3274
+ 0.9686208152571203
3275
+ ],
3276
+ [
3277
+ 883,
3278
+ 279391,
3279
+ 0.9289395066187627
3280
+ ],
3281
+ [
3282
+ 883,
3283
+ 321038,
3284
+ 0.9405556527105055
3285
+ ],
3286
+ [
3287
+ 883,
3288
+ 358853,
3289
+ 0.9216188652943913
3290
+ ],
3291
+ [
3292
+ 886,
3293
+ 26458,
3294
+ 0.9425142207977993
3295
+ ],
3296
+ [
3297
+ 886,
3298
+ 151243,
3299
+ 0.9425142207977993
3300
+ ],
3301
+ [
3302
+ 886,
3303
+ 153731,
3304
+ 0.9823833297275822
3305
+ ],
3306
+ [
3307
+ 886,
3308
+ 241254,
3309
+ 0.9717463385279338
3310
+ ],
3311
+ [
3312
+ 913,
3313
+ 154438,
3314
+ 0.9407286676089468
3315
+ ],
3316
+ [
3317
+ 913,
3318
+ 190011,
3319
+ 0.921318657511892
3320
+ ],
3321
+ [
3322
+ 913,
3323
+ 224559,
3324
+ 0.9407286676089468
3325
+ ],
3326
+ [
3327
+ 913,
3328
+ 386409,
3329
+ 0.9407286676089468
3330
+ ],
3331
+ [
3332
+ 913,
3333
+ 465758,
3334
+ 0.9559910784834212
3335
+ ],
3336
+ [
3337
+ 914,
3338
+ 20609,
3339
+ 0.9236094991807737
3340
+ ],
3341
+ [
3342
+ 914,
3343
+ 346223,
3344
+ 0.9408898978835074
3345
+ ],
3346
+ [
3347
+ 917,
3348
+ 101477,
3349
+ 0.9671680617040895
3350
+ ],
3351
+ [
3352
+ 917,
3353
+ 120286,
3354
+ 0.9671680617040895
3355
+ ],
3356
+ [
3357
+ 917,
3358
+ 147029,
3359
+ 0.9671680617040895
3360
+ ],
3361
+ [
3362
+ 917,
3363
+ 203698,
3364
+ 0.9214302609634097
3365
+ ],
3366
+ [
3367
+ 917,
3368
+ 303592,
3369
+ 0.9456013885783862
3370
+ ],
3371
+ [
3372
+ 917,
3373
+ 426981,
3374
+ 0.9671680617040895
3375
+ ],
3376
+ [
3377
+ 917,
3378
+ 498478,
3379
+ 0.9223804121434862
3380
+ ],
3381
+ [
3382
+ 918,
3383
+ 37671,
3384
+ 0.9652940608262063
3385
+ ],
3386
+ [
3387
+ 918,
3388
+ 299565,
3389
+ 0.9226271970771072
3390
+ ],
3391
+ [
3392
+ 918,
3393
+ 373887,
3394
+ 0.9462772216719754
3395
+ ],
3396
+ [
3397
+ 918,
3398
+ 404046,
3399
+ 0.9367625826737989
3400
+ ],
3401
+ [
3402
+ 921,
3403
+ 82982,
3404
+ 0.9211319917243936
3405
+ ],
3406
+ [
3407
+ 921,
3408
+ 296488,
3409
+ 0.9211319917243936
3410
+ ],
3411
+ [
3412
+ 921,
3413
+ 364295,
3414
+ 0.973406645308448
3415
+ ],
3416
+ [
3417
+ 921,
3418
+ 483627,
3419
+ 0.9459212066748571
3420
+ ],
3421
+ [
3422
+ 921,
3423
+ 489172,
3424
+ 0.9301516587223541
3425
+ ],
3426
+ [
3427
+ 923,
3428
+ 38848,
3429
+ 0.9999999999999999
3430
+ ],
3431
+ [
3432
+ 923,
3433
+ 83275,
3434
+ 0.9999999999999999
3435
+ ],
3436
+ [
3437
+ 923,
3438
+ 114717,
3439
+ 0.931507329104981
3440
+ ],
3441
+ [
3442
+ 923,
3443
+ 251823,
3444
+ 0.9999999999999999
3445
+ ],
3446
+ [
3447
+ 925,
3448
+ 44827,
3449
+ 0.9620241170936883
3450
+ ],
3451
+ [
3452
+ 925,
3453
+ 159567,
3454
+ 0.9431882031575993
3455
+ ],
3456
+ [
3457
+ 925,
3458
+ 160767,
3459
+ 0.9431882031575993
3460
+ ],
3461
+ [
3462
+ 925,
3463
+ 283644,
3464
+ 0.9431882031575993
3465
+ ],
3466
+ [
3467
+ 925,
3468
+ 323405,
3469
+ 0.9620241170936883
3470
+ ],
3471
+ [
3472
+ 925,
3473
+ 332064,
3474
+ 0.9620241170936883
3475
+ ],
3476
+ [
3477
+ 925,
3478
+ 411957,
3479
+ 0.9390355524154176
3480
+ ],
3481
+ [
3482
+ 928,
3483
+ 182732,
3484
+ 0.9500943904635375
3485
+ ],
3486
+ [
3487
+ 928,
3488
+ 237823,
3489
+ 0.9500943904635375
3490
+ ],
3491
+ [
3492
+ 928,
3493
+ 393671,
3494
+ 0.9202579874607955
3495
+ ],
3496
+ [
3497
+ 928,
3498
+ 425489,
3499
+ 0.9453906029308814
3500
+ ],
3501
+ [
3502
+ 928,
3503
+ 434897,
3504
+ 0.9768376503938748
3505
+ ],
3506
+ [
3507
+ 930,
3508
+ 152318,
3509
+ 0.9409099590132962
3510
+ ],
3511
+ [
3512
+ 930,
3513
+ 222527,
3514
+ 0.9409099590132962
3515
+ ],
3516
+ [
3517
+ 930,
3518
+ 340954,
3519
+ 0.9409099590132962
3520
+ ],
3521
+ [
3522
+ 930,
3523
+ 482392,
3524
+ 0.9456136326607361
3525
+ ],
3526
+ [
3527
+ 930,
3528
+ 482417,
3529
+ 0.9210092435711507
3530
+ ],
3531
+ [
3532
+ 934,
3533
+ 106098,
3534
+ 0.9220776326549248
3535
+ ],
3536
+ [
3537
+ 934,
3538
+ 197545,
3539
+ 0.9220776326549248
3540
+ ],
3541
+ [
3542
+ 934,
3543
+ 209118,
3544
+ 0.9782928840614608
3545
+ ],
3546
+ [
3547
+ 934,
3548
+ 344957,
3549
+ 0.9782928840614608
3550
+ ],
3551
+ [
3552
+ 934,
3553
+ 420565,
3554
+ 0.9220776326549248
3555
+ ],
3556
+ [
3557
+ 939,
3558
+ 117862,
3559
+ 0.942414271126192
3560
+ ],
3561
+ [
3562
+ 939,
3563
+ 330936,
3564
+ 0.9532342884326673
3565
+ ],
3566
+ [
3567
+ 939,
3568
+ 359576,
3569
+ 0.9735844294622664
3570
+ ],
3571
+ [
3572
+ 939,
3573
+ 399461,
3574
+ 0.9735844294622664
3575
+ ],
3576
+ [
3577
+ 939,
3578
+ 489163,
3579
+ 0.9376908841451245
3580
+ ],
3581
+ [
3582
+ 954,
3583
+ 107033,
3584
+ 0.9268568375524907
3585
+ ],
3586
+ [
3587
+ 956,
3588
+ 76322,
3589
+ 0.9285385507221632
3590
+ ],
3591
+ [
3592
+ 956,
3593
+ 86380,
3594
+ 0.9285385507221632
3595
+ ],
3596
+ [
3597
+ 956,
3598
+ 141830,
3599
+ 0.9285385507221632
3600
+ ],
3601
+ [
3602
+ 956,
3603
+ 181897,
3604
+ 0.9505265296386283
3605
+ ],
3606
+ [
3607
+ 956,
3608
+ 278711,
3609
+ 0.9505265296386283
3610
+ ],
3611
+ [
3612
+ 956,
3613
+ 338683,
3614
+ 0.924566068550966
3615
+ ],
3616
+ [
3617
+ 956,
3618
+ 350716,
3619
+ 0.9505265296386283
3620
+ ],
3621
+ [
3622
+ 961,
3623
+ 70100,
3624
+ 0.9568046680836708
3625
+ ],
3626
+ [
3627
+ 961,
3628
+ 94679,
3629
+ 0.9285653422367454
3630
+ ],
3631
+ [
3632
+ 961,
3633
+ 117211,
3634
+ 0.93695364951065
3635
+ ],
3636
+ [
3637
+ 961,
3638
+ 163034,
3639
+ 0.9568046680836708
3640
+ ],
3641
+ [
3642
+ 961,
3643
+ 203609,
3644
+ 0.9999999999999998
3645
+ ],
3646
+ [
3647
+ 961,
3648
+ 264810,
3649
+ 0.9568046680836708
3650
+ ],
3651
+ [
3652
+ 961,
3653
+ 321256,
3654
+ 0.9659908076225997
3655
+ ],
3656
+ [
3657
+ 961,
3658
+ 353408,
3659
+ 0.9728679784092943
3660
+ ],
3661
+ [
3662
+ 961,
3663
+ 467005,
3664
+ 0.9251219502457495
3665
+ ],
3666
+ [
3667
+ 961,
3668
+ 483170,
3669
+ 0.9285653422367454
3670
+ ],
3671
+ [
3672
+ 963,
3673
+ 383151,
3674
+ 0.932574620525862
3675
+ ],
3676
+ [
3677
+ 970,
3678
+ 351992,
3679
+ 0.9406397762016372
3680
+ ],
3681
+ [
3682
+ 970,
3683
+ 438101,
3684
+ 0.979795361581881
3685
+ ],
3686
+ [
3687
+ 971,
3688
+ 407723,
3689
+ 0.9408411651450355
3690
+ ],
3691
+ [
3692
+ 971,
3693
+ 448627,
3694
+ 0.973478613124993
3695
+ ],
3696
+ [
3697
+ 971,
3698
+ 492605,
3699
+ 0.926150075916318
3700
+ ],
3701
+ [
3702
+ 972,
3703
+ 183587,
3704
+ 0.9445784628264969
3705
+ ],
3706
+ [
3707
+ 974,
3708
+ 36884,
3709
+ 0.9746868313585277
3710
+ ],
3711
+ [
3712
+ 974,
3713
+ 58057,
3714
+ 0.9454468255789927
3715
+ ],
3716
+ [
3717
+ 974,
3718
+ 61100,
3719
+ 0.9476889803599398
3720
+ ],
3721
+ [
3722
+ 974,
3723
+ 115474,
3724
+ 0.9476889803599398
3725
+ ],
3726
+ [
3727
+ 974,
3728
+ 208656,
3729
+ 0.9505244006059891
3730
+ ],
3731
+ [
3732
+ 974,
3733
+ 269439,
3734
+ 0.9746868313585277
3735
+ ],
3736
+ [
3737
+ 974,
3738
+ 329181,
3739
+ 0.9551278856626921
3740
+ ],
3741
+ [
3742
+ 974,
3743
+ 456837,
3744
+ 0.9746868313585277
3745
+ ],
3746
+ [
3747
+ 974,
3748
+ 470259,
3749
+ 0.9476889803599398
3750
+ ],
3751
+ [
3752
+ 974,
3753
+ 481138,
3754
+ 0.9476889803599398
3755
+ ],
3756
+ [
3757
+ 977,
3758
+ 145978,
3759
+ 0.9486129652996147
3760
+ ],
3761
+ [
3762
+ 977,
3763
+ 275218,
3764
+ 0.9729294007204736
3765
+ ],
3766
+ [
3767
+ 983,
3768
+ 2117,
3769
+ 0.925640468356616
3770
+ ],
3771
+ [
3772
+ 983,
3773
+ 248291,
3774
+ 0.925640468356616
3775
+ ],
3776
+ [
3777
+ 983,
3778
+ 468597,
3779
+ 0.925640468356616
3780
+ ],
3781
+ [
3782
+ 990,
3783
+ 154932,
3784
+ 0.9408664125572189
3785
+ ],
3786
+ [
3787
+ 990,
3788
+ 405842,
3789
+ 0.9408664125572189
3790
+ ],
3791
+ [
3792
+ 990,
3793
+ 475471,
3794
+ 0.9999999999999996
3795
+ ],
3796
+ [
3797
+ 1008,
3798
+ 123335,
3799
+ 0.9345850764524805
3800
+ ],
3801
+ [
3802
+ 1008,
3803
+ 280335,
3804
+ 0.9345850764524805
3805
+ ],
3806
+ [
3807
+ 1008,
3808
+ 302449,
3809
+ 0.9454869883759904
3810
+ ],
3811
+ [
3812
+ 1008,
3813
+ 389530,
3814
+ 0.9625754913339561
3815
+ ],
3816
+ [
3817
+ 1008,
3818
+ 426237,
3819
+ 0.9454869883759904
3820
+ ],
3821
+ [
3822
+ 1008,
3823
+ 461643,
3824
+ 0.9211373671201963
3825
+ ],
3826
+ [
3827
+ 1008,
3828
+ 464308,
3829
+ 0.9770942624784199
3830
+ ],
3831
+ [
3832
+ 1018,
3833
+ 339738,
3834
+ 0.9601954090495614
3835
+ ],
3836
+ [
3837
+ 1021,
3838
+ 18210,
3839
+ 0.9544223464152446
3840
+ ],
3841
+ [
3842
+ 1021,
3843
+ 67861,
3844
+ 0.9362242768542957
3845
+ ],
3846
+ [
3847
+ 1021,
3848
+ 192424,
3849
+ 0.9351756631149284
3850
+ ],
3851
+ [
3852
+ 1021,
3853
+ 260311,
3854
+ 0.9507431117415815
3855
+ ],
3856
+ [
3857
+ 1021,
3858
+ 325731,
3859
+ 0.9544223464152446
3860
+ ],
3861
+ [
3862
+ 1021,
3863
+ 340347,
3864
+ 0.9272267170482568
3865
+ ],
3866
+ [
3867
+ 1021,
3868
+ 358434,
3869
+ 0.9688481476639093
3870
+ ],
3871
+ [
3872
+ 1026,
3873
+ 498337,
3874
+ 0.9493916120010238
3875
+ ],
3876
+ [
3877
+ 1030,
3878
+ 53667,
3879
+ 0.9313339569683532
3880
+ ],
3881
+ [
3882
+ 1030,
3883
+ 107464,
3884
+ 0.9630730283634069
3885
+ ],
3886
+ [
3887
+ 1030,
3888
+ 147079,
3889
+ 0.9302442122345036
3890
+ ],
3891
+ [
3892
+ 1030,
3893
+ 362340,
3894
+ 0.940364946624247
3895
+ ],
3896
+ [
3897
+ 1030,
3898
+ 433636,
3899
+ 0.9557159566773024
3900
+ ],
3901
+ [
3902
+ 1030,
3903
+ 480276,
3904
+ 0.9585719151338468
3905
+ ],
3906
+ [
3907
+ 1038,
3908
+ 243009,
3909
+ 0.9268414883511933
3910
+ ],
3911
+ [
3912
+ 1038,
3913
+ 362311,
3914
+ 0.9268414883511933
3915
+ ],
3916
+ [
3917
+ 1055,
3918
+ 49366,
3919
+ 0.984029717160225
3920
+ ],
3921
+ [
3922
+ 1055,
3923
+ 125124,
3924
+ 0.9630889816656262
3925
+ ],
3926
+ [
3927
+ 1055,
3928
+ 247616,
3929
+ 0.9748799641029338
3930
+ ],
3931
+ [
3932
+ 1055,
3933
+ 387987,
3934
+ 0.9630889816656262
3935
+ ],
3936
+ [
3937
+ 1055,
3938
+ 389074,
3939
+ 0.9748799641029338
3940
+ ],
3941
+ [
3942
+ 1055,
3943
+ 482585,
3944
+ 0.970192150388383
3945
+ ],
3946
+ [
3947
+ 1064,
3948
+ 62556,
3949
+ 0.9563398404027249
3950
+ ],
3951
+ [
3952
+ 1064,
3953
+ 109928,
3954
+ 0.9417351279846391
3955
+ ],
3956
+ [
3957
+ 1064,
3958
+ 193111,
3959
+ 0.9714791825120637
3960
+ ],
3961
+ [
3962
+ 1064,
3963
+ 262964,
3964
+ 0.9563398404027249
3965
+ ],
3966
+ [
3967
+ 1064,
3968
+ 287596,
3969
+ 0.9753909201040344
3970
+ ],
3971
+ [
3972
+ 1064,
3973
+ 346331,
3974
+ 0.9563398404027249
3975
+ ],
3976
+ [
3977
+ 1064,
3978
+ 373237,
3979
+ 0.9753909201040344
3980
+ ],
3981
+ [
3982
+ 1064,
3983
+ 450880,
3984
+ 0.9753909201040344
3985
+ ],
3986
+ [
3987
+ 1064,
3988
+ 473359,
3989
+ 0.9753909201040344
3990
+ ],
3991
+ [
3992
+ 1071,
3993
+ 266489,
3994
+ 0.9590009141408342
3995
+ ],
3996
+ [
3997
+ 1071,
3998
+ 359124,
3999
+ 0.9590009141408342
4000
+ ],
4001
+ [
4002
+ 1071,
4003
+ 363318,
4004
+ 0.9573956566277878
4005
+ ],
4006
+ [
4007
+ 1074,
4008
+ 59018,
4009
+ 0.9570052786593055
4010
+ ],
4011
+ [
4012
+ 1074,
4013
+ 196140,
4014
+ 0.9570052786593055
4015
+ ],
4016
+ [
4017
+ 1075,
4018
+ 333757,
4019
+ 0.9652761920762298
4020
+ ],
4021
+ [
4022
+ 1075,
4023
+ 401066,
4024
+ 0.9493245585540933
4025
+ ],
4026
+ [
4027
+ 1078,
4028
+ 48192,
4029
+ 0.937271272919452
4030
+ ],
4031
+ [
4032
+ 1078,
4033
+ 120695,
4034
+ 0.937271272919452
4035
+ ],
4036
+ [
4037
+ 1078,
4038
+ 262135,
4039
+ 0.9620993870522285
4040
+ ],
4041
+ [
4042
+ 1078,
4043
+ 305355,
4044
+ 0.9226535019111841
4045
+ ],
4046
+ [
4047
+ 1080,
4048
+ 162184,
4049
+ 0.9752772965646528
4050
+ ],
4051
+ [
4052
+ 1080,
4053
+ 358205,
4054
+ 0.9682637994216406
4055
+ ],
4056
+ [
4057
+ 1080,
4058
+ 379924,
4059
+ 0.9752772965646528
4060
+ ],
4061
+ [
4062
+ 1082,
4063
+ 6107,
4064
+ 0.9999999999999998
4065
+ ],
4066
+ [
4067
+ 1085,
4068
+ 337776,
4069
+ 0.9307301959310547
4070
+ ],
4071
+ [
4072
+ 1088,
4073
+ 378862,
4074
+ 0.9697073705382425
4075
+ ],
4076
+ [
4077
+ 1091,
4078
+ 166561,
4079
+ 0.9284361437400898
4080
+ ],
4081
+ [
4082
+ 1091,
4083
+ 318951,
4084
+ 0.9284361437400898
4085
+ ],
4086
+ [
4087
+ 1091,
4088
+ 415859,
4089
+ 0.9235515581070799
4090
+ ],
4091
+ [
4092
+ 1094,
4093
+ 45587,
4094
+ 0.9329175885048491
4095
+ ],
4096
+ [
4097
+ 1094,
4098
+ 88859,
4099
+ 0.9329175885048491
4100
+ ],
4101
+ [
4102
+ 1094,
4103
+ 205952,
4104
+ 0.9329175885048491
4105
+ ],
4106
+ [
4107
+ 1099,
4108
+ 133802,
4109
+ 0.9243451778118542
4110
+ ],
4111
+ [
4112
+ 1101,
4113
+ 23969,
4114
+ 0.9217785474444462
4115
+ ],
4116
+ [
4117
+ 1101,
4118
+ 125484,
4119
+ 0.9676025212984053
4120
+ ],
4121
+ [
4122
+ 1101,
4123
+ 178313,
4124
+ 0.9999999999999994
4125
+ ],
4126
+ [
4127
+ 1101,
4128
+ 210967,
4129
+ 0.9999999999999994
4130
+ ],
4131
+ [
4132
+ 1101,
4133
+ 230106,
4134
+ 0.9676025212984053
4135
+ ],
4136
+ [
4137
+ 1101,
4138
+ 265712,
4139
+ 0.946386895489197
4140
+ ],
4141
+ [
4142
+ 1101,
4143
+ 493334,
4144
+ 0.9676025212984053
4145
+ ],
4146
+ [
4147
+ 1103,
4148
+ 53762,
4149
+ 0.9440541458903817
4150
+ ],
4151
+ [
4152
+ 1103,
4153
+ 56435,
4154
+ 0.9797070326419334
4155
+ ],
4156
+ [
4157
+ 1103,
4158
+ 92196,
4159
+ 0.9999999999999998
4160
+ ],
4161
+ [
4162
+ 1103,
4163
+ 150175,
4164
+ 0.96233706043569
4165
+ ],
4166
+ [
4167
+ 1103,
4168
+ 214790,
4169
+ 0.9534952243426053
4170
+ ],
4171
+ [
4172
+ 1103,
4173
+ 270938,
4174
+ 0.9999999999999998
4175
+ ],
4176
+ [
4177
+ 1103,
4178
+ 273244,
4179
+ 0.9681998494312849
4180
+ ],
4181
+ [
4182
+ 1103,
4183
+ 328992,
4184
+ 0.9999999999999998
4185
+ ],
4186
+ [
4187
+ 1103,
4188
+ 369469,
4189
+ 0.9681998494312849
4190
+ ],
4191
+ [
4192
+ 1103,
4193
+ 374775,
4194
+ 0.9534952243426053
4195
+ ],
4196
+ [
4197
+ 1103,
4198
+ 496433,
4199
+ 0.9681998494312849
4200
+ ],
4201
+ [
4202
+ 1105,
4203
+ 38133,
4204
+ 0.9268927631669386
4205
+ ],
4206
+ [
4207
+ 1105,
4208
+ 258564,
4209
+ 0.9268927631669386
4210
+ ],
4211
+ [
4212
+ 1105,
4213
+ 337928,
4214
+ 0.9517737004443392
4215
+ ],
4216
+ [
4217
+ 1109,
4218
+ 198455,
4219
+ 0.9203231673796425
4220
+ ],
4221
+ [
4222
+ 1111,
4223
+ 162493,
4224
+ 0.942337219689891
4225
+ ],
4226
+ [
4227
+ 1112,
4228
+ 117165,
4229
+ 0.9260219443327345
4230
+ ],
4231
+ [
4232
+ 1121,
4233
+ 20298,
4234
+ 0.9564452776536262
4235
+ ],
4236
+ [
4237
+ 1121,
4238
+ 224141,
4239
+ 0.9332126868042823
4240
+ ],
4241
+ [
4242
+ 1121,
4243
+ 303775,
4244
+ 0.9498848744398697
4245
+ ],
4246
+ [
4247
+ 1121,
4248
+ 474139,
4249
+ 0.9244000714367864
4250
+ ],
4251
+ [
4252
+ 1124,
4253
+ 382178,
4254
+ 0.9484513861392051
4255
+ ],
4256
+ [
4257
+ 1124,
4258
+ 443233,
4259
+ 0.9484513861392051
4260
+ ],
4261
+ [
4262
+ 1124,
4263
+ 473332,
4264
+ 0.9484513861392051
4265
+ ],
4266
+ [
4267
+ 1127,
4268
+ 211527,
4269
+ 0.9237636562248488
4270
+ ],
4271
+ [
4272
+ 1127,
4273
+ 217782,
4274
+ 0.9847457279476864
4275
+ ],
4276
+ [
4277
+ 1127,
4278
+ 304499,
4279
+ 0.961103226982439
4280
+ ],
4281
+ [
4282
+ 1127,
4283
+ 350742,
4284
+ 0.9499780901970125
4285
+ ],
4286
+ [
4287
+ 1127,
4288
+ 397968,
4289
+ 0.9499780901970125
4290
+ ],
4291
+ [
4292
+ 1127,
4293
+ 409526,
4294
+ 0.961103226982439
4295
+ ],
4296
+ [
4297
+ 1127,
4298
+ 410774,
4299
+ 0.961103226982439
4300
+ ],
4301
+ [
4302
+ 1131,
4303
+ 21127,
4304
+ 0.9411887657920203
4305
+ ],
4306
+ [
4307
+ 1131,
4308
+ 299050,
4309
+ 0.9626067634804859
4310
+ ],
4311
+ [
4312
+ 1131,
4313
+ 337765,
4314
+ 0.9411887657920203
4315
+ ],
4316
+ [
4317
+ 1131,
4318
+ 441909,
4319
+ 0.9411887657920203
4320
+ ],
4321
+ [
4322
+ 1133,
4323
+ 29069,
4324
+ 0.966727826205192
4325
+ ],
4326
+ [
4327
+ 1133,
4328
+ 32820,
4329
+ 0.9429374895531828
4330
+ ],
4331
+ [
4332
+ 1133,
4333
+ 89070,
4334
+ 0.9429374895531828
4335
+ ],
4336
+ [
4337
+ 1133,
4338
+ 130613,
4339
+ 0.9712456103709378
4340
+ ],
4341
+ [
4342
+ 1133,
4343
+ 377534,
4344
+ 0.9999999999999993
4345
+ ],
4346
+ [
4347
+ 1133,
4348
+ 384037,
4349
+ 0.9409062577392334
4350
+ ],
4351
+ [
4352
+ 1133,
4353
+ 388082,
4354
+ 0.9249029212324722
4355
+ ],
4356
+ [
4357
+ 1133,
4358
+ 429046,
4359
+ 0.9824701054831212
4360
+ ],
4361
+ [
4362
+ 1133,
4363
+ 453689,
4364
+ 0.9712456103709378
4365
+ ],
4366
+ [
4367
+ 1134,
4368
+ 265494,
4369
+ 0.9246055408370056
4370
+ ],
4371
+ [
4372
+ 1135,
4373
+ 22322,
4374
+ 0.9862496563891352
4375
+ ],
4376
+ [
4377
+ 1135,
4378
+ 35392,
4379
+ 0.9783299484331893
4380
+ ],
4381
+ [
4382
+ 1135,
4383
+ 170645,
4384
+ 0.9680793451217327
4385
+ ],
4386
+ [
4387
+ 1135,
4388
+ 174990,
4389
+ 0.9549514159999897
4390
+ ],
4391
+ [
4392
+ 1135,
4393
+ 213272,
4394
+ 0.9783299484331893
4395
+ ],
4396
+ [
4397
+ 1135,
4398
+ 434800,
4399
+ 0.9680793451217327
4400
+ ],
4401
+ [
4402
+ 1135,
4403
+ 443451,
4404
+ 0.9783299484331893
4405
+ ],
4406
+ [
4407
+ 1135,
4408
+ 496127,
4409
+ 0.9246595800300451
4410
+ ],
4411
+ [
4412
+ 1144,
4413
+ 201182,
4414
+ 0.9312135454250817
4415
+ ],
4416
+ [
4417
+ 1144,
4418
+ 366230,
4419
+ 0.9638201843080754
4420
+ ],
4421
+ [
4422
+ 1152,
4423
+ 83254,
4424
+ 0.9467633072805361
4425
+ ],
4426
+ [
4427
+ 1157,
4428
+ 487920,
4429
+ 0.9657267926754519
4430
+ ],
4431
+ [
4432
+ 1159,
4433
+ 175537,
4434
+ 0.9421407950543267
4435
+ ],
4436
+ [
4437
+ 1159,
4438
+ 268451,
4439
+ 0.9373971857839963
4440
+ ],
4441
+ [
4442
+ 1159,
4443
+ 337186,
4444
+ 0.9202543839044159
4445
+ ],
4446
+ [
4447
+ 1159,
4448
+ 340544,
4449
+ 0.9202543839044159
4450
+ ],
4451
+ [
4452
+ 1159,
4453
+ 361162,
4454
+ 0.9831088453636716
4455
+ ],
4456
+ [
4457
+ 1159,
4458
+ 363063,
4459
+ 0.9443346142115641
4460
+ ],
4461
+ [
4462
+ 1159,
4463
+ 368395,
4464
+ 0.9202543839044159
4465
+ ],
4466
+ [
4467
+ 1165,
4468
+ 312741,
4469
+ 0.939879888330517
4470
+ ],
4471
+ [
4472
+ 1172,
4473
+ 39479,
4474
+ 0.9458492359524295
4475
+ ],
4476
+ [
4477
+ 1172,
4478
+ 143690,
4479
+ 0.922092196580223
4480
+ ],
4481
+ [
4482
+ 1172,
4483
+ 275690,
4484
+ 0.9458492359524295
4485
+ ],
4486
+ [
4487
+ 1172,
4488
+ 329222,
4489
+ 0.922092196580223
4490
+ ],
4491
+ [
4492
+ 1173,
4493
+ 275951,
4494
+ 0.9730639211141395
4495
+ ],
4496
+ [
4497
+ 1173,
4498
+ 332158,
4499
+ 0.9346005473244346
4500
+ ],
4501
+ [
4502
+ 1173,
4503
+ 371020,
4504
+ 0.9837466992248414
4505
+ ],
4506
+ [
4507
+ 1173,
4508
+ 417357,
4509
+ 0.9730639211141395
4510
+ ],
4511
+ [
4512
+ 1177,
4513
+ 22342,
4514
+ 0.962774419677722
4515
+ ],
4516
+ [
4517
+ 1177,
4518
+ 158422,
4519
+ 0.9383795747198187
4520
+ ],
4521
+ [
4522
+ 1177,
4523
+ 231491,
4524
+ 0.9383795747198187
4525
+ ],
4526
+ [
4527
+ 1177,
4528
+ 319465,
4529
+ 0.9338288950718607
4530
+ ],
4531
+ [
4532
+ 1177,
4533
+ 323984,
4534
+ 0.962774419677722
4535
+ ],
4536
+ [
4537
+ 1177,
4538
+ 355277,
4539
+ 0.962774419677722
4540
+ ],
4541
+ [
4542
+ 1177,
4543
+ 377726,
4544
+ 0.9383795747198187
4545
+ ],
4546
+ [
4547
+ 1177,
4548
+ 382028,
4549
+ 0.9472632902496475
4550
+ ],
4551
+ [
4552
+ 1177,
4553
+ 407861,
4554
+ 0.9383795747198187
4555
+ ],
4556
+ [
4557
+ 1186,
4558
+ 22296,
4559
+ 0.9291853572770413
4560
+ ],
4561
+ [
4562
+ 1186,
4563
+ 52206,
4564
+ 0.9291853572770413
4565
+ ],
4566
+ [
4567
+ 1186,
4568
+ 64678,
4569
+ 0.9291853572770413
4570
+ ],
4571
+ [
4572
+ 1186,
4573
+ 160116,
4574
+ 0.9819908099429802
4575
+ ],
4576
+ [
4577
+ 1186,
4578
+ 200147,
4579
+ 0.9355752679727879
4580
+ ],
4581
+ [
4582
+ 1186,
4583
+ 338906,
4584
+ 0.9291853572770413
4585
+ ],
4586
+ [
4587
+ 1186,
4588
+ 380089,
4589
+ 0.9254931485999628
4590
+ ],
4591
+ [
4592
+ 1186,
4593
+ 459910,
4594
+ 0.95149307341076
4595
+ ],
4596
+ [
4597
+ 1192,
4598
+ 278572,
4599
+ 0.9468145312102751
4600
+ ],
4601
+ [
4602
+ 1192,
4603
+ 421056,
4604
+ 0.9468145312102751
4605
+ ],
4606
+ [
4607
+ 1196,
4608
+ 1624,
4609
+ 0.9999999999999998
4610
+ ],
4611
+ [
4612
+ 1196,
4613
+ 42649,
4614
+ 0.9657489685859476
4615
+ ],
4616
+ [
4617
+ 1196,
4618
+ 55232,
4619
+ 0.9657489685859476
4620
+ ],
4621
+ [
4622
+ 1196,
4623
+ 135661,
4624
+ 0.9302238427812799
4625
+ ],
4626
+ [
4627
+ 1196,
4628
+ 334727,
4629
+ 0.9776741849124975
4630
+ ],
4631
+ [
4632
+ 1196,
4633
+ 490871,
4634
+ 0.9657489685859476
4635
+ ],
4636
+ [
4637
+ 1200,
4638
+ 35287,
4639
+ 0.9382462221148598
4640
+ ],
4641
+ [
4642
+ 1200,
4643
+ 239634,
4644
+ 0.9382462221148598
4645
+ ],
4646
+ [
4647
+ 1203,
4648
+ 77391,
4649
+ 0.9528979132032578
4650
+ ],
4651
+ [
4652
+ 1203,
4653
+ 149346,
4654
+ 0.9255389106296006
4655
+ ],
4656
+ [
4657
+ 1203,
4658
+ 317423,
4659
+ 0.9255389106296006
4660
+ ],
4661
+ [
4662
+ 1203,
4663
+ 333320,
4664
+ 0.9255389106296006
4665
+ ],
4666
+ [
4667
+ 1203,
4668
+ 452232,
4669
+ 0.9745114027457565
4670
+ ],
4671
+ [
4672
+ 1207,
4673
+ 31057,
4674
+ 0.9401256762312529
4675
+ ],
4676
+ [
4677
+ 1207,
4678
+ 168971,
4679
+ 0.9305392051910837
4680
+ ],
4681
+ [
4682
+ 1207,
4683
+ 238589,
4684
+ 0.9202430131421394
4685
+ ],
4686
+ [
4687
+ 1207,
4688
+ 298128,
4689
+ 0.9584049059582207
4690
+ ],
4691
+ [
4692
+ 1207,
4693
+ 308512,
4694
+ 0.9401256762312529
4695
+ ],
4696
+ [
4697
+ 1207,
4698
+ 411927,
4699
+ 0.9481732915657314
4700
+ ],
4701
+ [
4702
+ 1207,
4703
+ 485469,
4704
+ 0.9401256762312529
4705
+ ],
4706
+ [
4707
+ 1219,
4708
+ 136217,
4709
+ 0.9759505779801322
4710
+ ],
4711
+ [
4712
+ 1219,
4713
+ 308956,
4714
+ 0.9573046030570814
4715
+ ],
4716
+ [
4717
+ 1223,
4718
+ 17790,
4719
+ 0.9748507447140344
4720
+ ],
4721
+ [
4722
+ 1223,
4723
+ 112837,
4724
+ 0.9363061038288816
4725
+ ],
4726
+ [
4727
+ 1223,
4728
+ 302423,
4729
+ 0.9329560426015548
4730
+ ],
4731
+ [
4732
+ 1223,
4733
+ 419447,
4734
+ 0.9748507447140344
4735
+ ],
4736
+ [
4737
+ 1223,
4738
+ 488428,
4739
+ 0.9554098540811915
4740
+ ],
4741
+ [
4742
+ 1225,
4743
+ 49245,
4744
+ 0.9522255921724462
4745
+ ],
4746
+ [
4747
+ 1225,
4748
+ 350982,
4749
+ 0.970482539122734
4750
+ ],
4751
+ [
4752
+ 1226,
4753
+ 37332,
4754
+ 0.9643727814584843
4755
+ ],
4756
+ [
4757
+ 1226,
4758
+ 244931,
4759
+ 0.9643727814584843
4760
+ ],
4761
+ [
4762
+ 1226,
4763
+ 311378,
4764
+ 0.9796836484081123
4765
+ ],
4766
+ [
4767
+ 1226,
4768
+ 373612,
4769
+ 0.9643727814584843
4770
+ ],
4771
+ [
4772
+ 1226,
4773
+ 408217,
4774
+ 0.9403889915267051
4775
+ ],
4776
+ [
4777
+ 1239,
4778
+ 94864,
4779
+ 0.9657646840543253
4780
+ ],
4781
+ [
4782
+ 1239,
4783
+ 238038,
4784
+ 0.9657646840543253
4785
+ ],
4786
+ [
4787
+ 1239,
4788
+ 464893,
4789
+ 0.9672681280685052
4790
+ ],
4791
+ [
4792
+ 1239,
4793
+ 470295,
4794
+ 0.9306108063826621
4795
+ ],
4796
+ [
4797
+ 1242,
4798
+ 35666,
4799
+ 0.9552589313707468
4800
+ ],
4801
+ [
4802
+ 1242,
4803
+ 56811,
4804
+ 0.9805071778778974
4805
+ ],
4806
+ [
4807
+ 1242,
4808
+ 88269,
4809
+ 0.9552589313707468
4810
+ ],
4811
+ [
4812
+ 1242,
4813
+ 98569,
4814
+ 0.9552589313707468
4815
+ ],
4816
+ [
4817
+ 1242,
4818
+ 238667,
4819
+ 0.969432632193669
4820
+ ],
4821
+ [
4822
+ 1242,
4823
+ 272642,
4824
+ 0.9999999999999991
4825
+ ],
4826
+ [
4827
+ 1242,
4828
+ 368484,
4829
+ 0.969432632193669
4830
+ ],
4831
+ [
4832
+ 1244,
4833
+ 267770,
4834
+ 0.954036358997866
4835
+ ],
4836
+ [
4837
+ 1244,
4838
+ 358686,
4839
+ 0.954036358997866
4840
+ ],
4841
+ [
4842
+ 1255,
4843
+ 237782,
4844
+ 0.9238579027621379
4845
+ ],
4846
+ [
4847
+ 1257,
4848
+ 72345,
4849
+ 0.9355567178323783
4850
+ ],
4851
+ [
4852
+ 1264,
4853
+ 114676,
4854
+ 0.9637134349956072
4855
+ ],
4856
+ [
4857
+ 1264,
4858
+ 130538,
4859
+ 0.9477970536868301
4860
+ ],
4861
+ [
4862
+ 1264,
4863
+ 218499,
4864
+ 0.9477970536868301
4865
+ ],
4866
+ [
4867
+ 1264,
4868
+ 248379,
4869
+ 0.9238569875937084
4870
+ ],
4871
+ [
4872
+ 1264,
4873
+ 285033,
4874
+ 0.9477970536868301
4875
+ ],
4876
+ [
4877
+ 1264,
4878
+ 432070,
4879
+ 0.9238569875937084
4880
+ ],
4881
+ [
4882
+ 1264,
4883
+ 437708,
4884
+ 0.9238569875937084
4885
+ ],
4886
+ [
4887
+ 1264,
4888
+ 458591,
4889
+ 0.9477970536868301
4890
+ ],
4891
+ [
4892
+ 1267,
4893
+ 156659,
4894
+ 0.960033127485912
4895
+ ],
4896
+ [
4897
+ 1267,
4898
+ 271459,
4899
+ 0.9438800398880226
4900
+ ],
4901
+ [
4902
+ 1267,
4903
+ 311220,
4904
+ 0.960033127485912
4905
+ ],
4906
+ [
4907
+ 1267,
4908
+ 352981,
4909
+ 0.9534660062520778
4910
+ ],
4911
+ [
4912
+ 1267,
4913
+ 400705,
4914
+ 0.9534660062520778
4915
+ ],
4916
+ [
4917
+ 1267,
4918
+ 420282,
4919
+ 0.9224201102986915
4920
+ ],
4921
+ [
4922
+ 1267,
4923
+ 444993,
4924
+ 0.968460731962802
4925
+ ],
4926
+ [
4927
+ 1270,
4928
+ 468855,
4929
+ 0.9305167630126341
4930
+ ],
4931
+ [
4932
+ 1273,
4933
+ 66516,
4934
+ 0.9999999999999998
4935
+ ],
4936
+ [
4937
+ 1273,
4938
+ 205386,
4939
+ 0.9999999999999998
4940
+ ],
4941
+ [
4942
+ 1273,
4943
+ 257943,
4944
+ 0.9999999999999998
4945
+ ],
4946
+ [
4947
+ 1273,
4948
+ 259504,
4949
+ 0.9264752329017355
4950
+ ],
4951
+ [
4952
+ 1273,
4953
+ 466706,
4954
+ 0.9999999999999998
4955
+ ],
4956
+ [
4957
+ 1273,
4958
+ 474148,
4959
+ 0.9577011123194441
4960
+ ],
4961
+ [
4962
+ 1286,
4963
+ 293802,
4964
+ 0.9237615369729684
4965
+ ],
4966
+ [
4967
+ 1286,
4968
+ 316250,
4969
+ 0.9999999999999998
4970
+ ],
4971
+ [
4972
+ 1286,
4973
+ 392697,
4974
+ 0.9237615369729684
4975
+ ],
4976
+ [
4977
+ 1286,
4978
+ 499299,
4979
+ 0.9237615369729684
4980
+ ],
4981
+ [
4982
+ 1288,
4983
+ 13654,
4984
+ 0.9400326649339904
4985
+ ],
4986
+ [
4987
+ 1288,
4988
+ 46005,
4989
+ 0.9765226427953282
4990
+ ],
4991
+ [
4992
+ 1288,
4993
+ 144221,
4994
+ 0.9400326649339904
4995
+ ],
4996
+ [
4997
+ 1288,
4998
+ 324125,
4999
+ 0.9400326649339904
5000
+ ],
5001
+ [
5002
+ 1291,
5003
+ 211290,
5004
+ 0.9552201766176319
5005
+ ]
5006
+ ]
5007
+ }
reports/figures/dataset_class_distribution.png ADDED
reports/figures/text_length_distribution.png ADDED
reports/figures/tfidf-test_confusion_matrix.png ADDED

Git LFS Details

  • SHA256: da0c6185b122f39f2d7056f32f1fcb0aa1f5b9267bd26e14503c4666ae9c1fad
  • Pointer size: 131 Bytes
  • Size of remote file: 224 kB
reports/figures/tfidf-test_per_class_metrics.png ADDED

Git LFS Details

  • SHA256: 1d752e38b18c577c3c2e3673ef12a4f07e9d17c345764643bd9c17199e5f6758
  • Pointer size: 131 Bytes
  • Size of remote file: 117 kB
reports/figures/tfidf-validation_confusion_matrix.png ADDED

Git LFS Details

  • SHA256: 063b84e0c2958f4c60c6dc5f864b7f005609c2fb9b7b8c9a2a023bb5825c14a2
  • Pointer size: 131 Bytes
  • Size of remote file: 229 kB
reports/figures/tfidf-validation_per_class_metrics.png ADDED

Git LFS Details

  • SHA256: 5e976d5ecb1c77130444a11cd729108c4e7815f49e26f961e37ede8925d1e0af
  • Pointer size: 131 Bytes
  • Size of remote file: 119 kB
reports/model_comparison.json ADDED
@@ -0,0 +1,629 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "models": [
3
+ {
4
+ "model_name": "tfidf",
5
+ "accuracy": 0.9922900091368398,
6
+ "macro_f1": 0.9924938150790148,
7
+ "weighted_f1": 0.9922797304638178,
8
+ "report": {
9
+ "charity_donations": {
10
+ "precision": 1.0,
11
+ "recall": 1.0,
12
+ "f1-score": 1.0,
13
+ "support": 22160.0
14
+ },
15
+ "entertainment_recreation": {
16
+ "precision": 0.9997045540877052,
17
+ "recall": 0.9978094194961665,
18
+ "f1-score": 0.9987560877906854,
19
+ "support": 23738.0
20
+ },
21
+ "financial_services": {
22
+ "precision": 0.9894933943618017,
23
+ "recall": 0.9998948806895827,
24
+ "f1-score": 0.9946669455191886,
25
+ "support": 28539.0
26
+ },
27
+ "food_dining": {
28
+ "precision": 0.9956344708894765,
29
+ "recall": 0.993840610072907,
30
+ "f1-score": 0.9947367317410724,
31
+ "support": 23866.0
32
+ },
33
+ "government_legal": {
34
+ "precision": 0.9895684097421203,
35
+ "recall": 0.9904552787237856,
36
+ "f1-score": 0.990011645614978,
37
+ "support": 22316.0
38
+ },
39
+ "healthcare_medical": {
40
+ "precision": 0.9781765910179901,
41
+ "recall": 0.9873373197326768,
42
+ "f1-score": 0.9827356075359401,
43
+ "support": 22744.0
44
+ },
45
+ "income": {
46
+ "precision": 0.99191313434192,
47
+ "recall": 0.9977151213270575,
48
+ "f1-score": 0.9948056682006653,
49
+ "support": 21883.0
50
+ },
51
+ "shopping_retail": {
52
+ "precision": 0.9869506249770903,
53
+ "recall": 0.9699207492795389,
54
+ "f1-score": 0.9783615850002725,
55
+ "support": 27760.0
56
+ },
57
+ "transportation": {
58
+ "precision": 0.998010245120867,
59
+ "recall": 0.9929657554441683,
60
+ "f1-score": 0.9954816097293189,
61
+ "support": 23741.0
62
+ },
63
+ "utilities_services": {
64
+ "precision": 0.9948188784395681,
65
+ "recall": 0.9959462993636126,
66
+ "f1-score": 0.9953822696580266,
67
+ "support": 22942.0
68
+ },
69
+ "accuracy": 0.9922900091368398,
70
+ "macro avg": {
71
+ "precision": 0.992427030297854,
72
+ "recall": 0.9925885434129496,
73
+ "f1-score": 0.9924938150790148,
74
+ "support": 239689.0
75
+ },
76
+ "weighted avg": {
77
+ "precision": 0.9923003908962669,
78
+ "recall": 0.9922900091368398,
79
+ "f1-score": 0.9922797304638178,
80
+ "support": 239689.0
81
+ }
82
+ },
83
+ "confusion_matrix": [
84
+ [
85
+ 22160,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0,
93
+ 0,
94
+ 0
95
+ ],
96
+ [
97
+ 0,
98
+ 23686,
99
+ 48,
100
+ 0,
101
+ 1,
102
+ 1,
103
+ 0,
104
+ 1,
105
+ 1,
106
+ 0
107
+ ],
108
+ [
109
+ 0,
110
+ 1,
111
+ 28536,
112
+ 0,
113
+ 0,
114
+ 0,
115
+ 1,
116
+ 1,
117
+ 0,
118
+ 0
119
+ ],
120
+ [
121
+ 0,
122
+ 2,
123
+ 57,
124
+ 23719,
125
+ 0,
126
+ 0,
127
+ 0,
128
+ 88,
129
+ 0,
130
+ 0
131
+ ],
132
+ [
133
+ 0,
134
+ 1,
135
+ 0,
136
+ 0,
137
+ 22103,
138
+ 0,
139
+ 176,
140
+ 36,
141
+ 0,
142
+ 0
143
+ ],
144
+ [
145
+ 0,
146
+ 0,
147
+ 58,
148
+ 0,
149
+ 0,
150
+ 22456,
151
+ 0,
152
+ 230,
153
+ 0,
154
+ 0
155
+ ],
156
+ [
157
+ 0,
158
+ 0,
159
+ 0,
160
+ 0,
161
+ 50,
162
+ 0,
163
+ 21833,
164
+ 0,
165
+ 0,
166
+ 0
167
+ ],
168
+ [
169
+ 0,
170
+ 0,
171
+ 49,
172
+ 103,
173
+ 181,
174
+ 500,
175
+ 0,
176
+ 26925,
177
+ 2,
178
+ 0
179
+ ],
180
+ [
181
+ 0,
182
+ 2,
183
+ 45,
184
+ 1,
185
+ 0,
186
+ 0,
187
+ 0,
188
+ 0,
189
+ 23574,
190
+ 119
191
+ ],
192
+ [
193
+ 0,
194
+ 1,
195
+ 46,
196
+ 0,
197
+ 1,
198
+ 0,
199
+ 1,
200
+ 0,
201
+ 44,
202
+ 22849
203
+ ]
204
+ ],
205
+ "samples": 239689,
206
+ "confidence_threshold": 0.7,
207
+ "confidence_coverage": 0.9790228170671161,
208
+ "high_confidence_accuracy": 0.999552546013185
209
+ }
210
+ ],
211
+ "selection_metric": "macro_f1",
212
+ "selected": "tfidf",
213
+ "selected_validation": {
214
+ "model_name": "tfidf",
215
+ "accuracy": 0.9922900091368398,
216
+ "macro_f1": 0.9924938150790148,
217
+ "weighted_f1": 0.9922797304638178,
218
+ "report": {
219
+ "charity_donations": {
220
+ "precision": 1.0,
221
+ "recall": 1.0,
222
+ "f1-score": 1.0,
223
+ "support": 22160.0
224
+ },
225
+ "entertainment_recreation": {
226
+ "precision": 0.9997045540877052,
227
+ "recall": 0.9978094194961665,
228
+ "f1-score": 0.9987560877906854,
229
+ "support": 23738.0
230
+ },
231
+ "financial_services": {
232
+ "precision": 0.9894933943618017,
233
+ "recall": 0.9998948806895827,
234
+ "f1-score": 0.9946669455191886,
235
+ "support": 28539.0
236
+ },
237
+ "food_dining": {
238
+ "precision": 0.9956344708894765,
239
+ "recall": 0.993840610072907,
240
+ "f1-score": 0.9947367317410724,
241
+ "support": 23866.0
242
+ },
243
+ "government_legal": {
244
+ "precision": 0.9895684097421203,
245
+ "recall": 0.9904552787237856,
246
+ "f1-score": 0.990011645614978,
247
+ "support": 22316.0
248
+ },
249
+ "healthcare_medical": {
250
+ "precision": 0.9781765910179901,
251
+ "recall": 0.9873373197326768,
252
+ "f1-score": 0.9827356075359401,
253
+ "support": 22744.0
254
+ },
255
+ "income": {
256
+ "precision": 0.99191313434192,
257
+ "recall": 0.9977151213270575,
258
+ "f1-score": 0.9948056682006653,
259
+ "support": 21883.0
260
+ },
261
+ "shopping_retail": {
262
+ "precision": 0.9869506249770903,
263
+ "recall": 0.9699207492795389,
264
+ "f1-score": 0.9783615850002725,
265
+ "support": 27760.0
266
+ },
267
+ "transportation": {
268
+ "precision": 0.998010245120867,
269
+ "recall": 0.9929657554441683,
270
+ "f1-score": 0.9954816097293189,
271
+ "support": 23741.0
272
+ },
273
+ "utilities_services": {
274
+ "precision": 0.9948188784395681,
275
+ "recall": 0.9959462993636126,
276
+ "f1-score": 0.9953822696580266,
277
+ "support": 22942.0
278
+ },
279
+ "accuracy": 0.9922900091368398,
280
+ "macro avg": {
281
+ "precision": 0.992427030297854,
282
+ "recall": 0.9925885434129496,
283
+ "f1-score": 0.9924938150790148,
284
+ "support": 239689.0
285
+ },
286
+ "weighted avg": {
287
+ "precision": 0.9923003908962669,
288
+ "recall": 0.9922900091368398,
289
+ "f1-score": 0.9922797304638178,
290
+ "support": 239689.0
291
+ }
292
+ },
293
+ "confusion_matrix": [
294
+ [
295
+ 22160,
296
+ 0,
297
+ 0,
298
+ 0,
299
+ 0,
300
+ 0,
301
+ 0,
302
+ 0,
303
+ 0,
304
+ 0
305
+ ],
306
+ [
307
+ 0,
308
+ 23686,
309
+ 48,
310
+ 0,
311
+ 1,
312
+ 1,
313
+ 0,
314
+ 1,
315
+ 1,
316
+ 0
317
+ ],
318
+ [
319
+ 0,
320
+ 1,
321
+ 28536,
322
+ 0,
323
+ 0,
324
+ 0,
325
+ 1,
326
+ 1,
327
+ 0,
328
+ 0
329
+ ],
330
+ [
331
+ 0,
332
+ 2,
333
+ 57,
334
+ 23719,
335
+ 0,
336
+ 0,
337
+ 0,
338
+ 88,
339
+ 0,
340
+ 0
341
+ ],
342
+ [
343
+ 0,
344
+ 1,
345
+ 0,
346
+ 0,
347
+ 22103,
348
+ 0,
349
+ 176,
350
+ 36,
351
+ 0,
352
+ 0
353
+ ],
354
+ [
355
+ 0,
356
+ 0,
357
+ 58,
358
+ 0,
359
+ 0,
360
+ 22456,
361
+ 0,
362
+ 230,
363
+ 0,
364
+ 0
365
+ ],
366
+ [
367
+ 0,
368
+ 0,
369
+ 0,
370
+ 0,
371
+ 50,
372
+ 0,
373
+ 21833,
374
+ 0,
375
+ 0,
376
+ 0
377
+ ],
378
+ [
379
+ 0,
380
+ 0,
381
+ 49,
382
+ 103,
383
+ 181,
384
+ 500,
385
+ 0,
386
+ 26925,
387
+ 2,
388
+ 0
389
+ ],
390
+ [
391
+ 0,
392
+ 2,
393
+ 45,
394
+ 1,
395
+ 0,
396
+ 0,
397
+ 0,
398
+ 0,
399
+ 23574,
400
+ 119
401
+ ],
402
+ [
403
+ 0,
404
+ 1,
405
+ 46,
406
+ 0,
407
+ 1,
408
+ 0,
409
+ 1,
410
+ 0,
411
+ 44,
412
+ 22849
413
+ ]
414
+ ],
415
+ "samples": 239689,
416
+ "confidence_threshold": 0.7,
417
+ "confidence_coverage": 0.9790228170671161,
418
+ "high_confidence_accuracy": 0.999552546013185
419
+ },
420
+ "selected_test": {
421
+ "model_name": "tfidf",
422
+ "accuracy": 0.9929848825051134,
423
+ "macro_f1": 0.9932028965957418,
424
+ "weighted_f1": 0.9929778167820927,
425
+ "report": {
426
+ "charity_donations": {
427
+ "precision": 0.9999548451187573,
428
+ "recall": 1.0,
429
+ "f1-score": 0.9999774220496264,
430
+ "support": 22145.0
431
+ },
432
+ "entertainment_recreation": {
433
+ "precision": 0.999746589517253,
434
+ "recall": 0.9980183826629564,
435
+ "f1-score": 0.9988817385800189,
436
+ "support": 23718.0
437
+ },
438
+ "financial_services": {
439
+ "precision": 0.9880473946784922,
440
+ "recall": 0.9998247090169682,
441
+ "f1-score": 0.9939011640064125,
442
+ "support": 28524.0
443
+ },
444
+ "food_dining": {
445
+ "precision": 0.9960777698114799,
446
+ "recall": 0.9931457886548085,
447
+ "f1-score": 0.9946096184620568,
448
+ "support": 23781.0
449
+ },
450
+ "government_legal": {
451
+ "precision": 0.9907501211293661,
452
+ "recall": 0.9937704338605637,
453
+ "f1-score": 0.9922579791340407,
454
+ "support": 22634.0
455
+ },
456
+ "healthcare_medical": {
457
+ "precision": 0.9814401400744145,
458
+ "recall": 0.9882316643159379,
459
+ "f1-score": 0.9848241934421189,
460
+ "support": 22688.0
461
+ },
462
+ "income": {
463
+ "precision": 0.995253164556962,
464
+ "recall": 0.9974627339042182,
465
+ "f1-score": 0.9963567242198638,
466
+ "support": 22071.0
467
+ },
468
+ "shopping_retail": {
469
+ "precision": 0.9878508434881254,
470
+ "recall": 0.973775340884496,
471
+ "f1-score": 0.980762593325946,
472
+ "support": 27722.0
473
+ },
474
+ "transportation": {
475
+ "precision": 0.9976043374101626,
476
+ "recall": 0.9926396788223486,
477
+ "f1-score": 0.9951158159522062,
478
+ "support": 23912.0
479
+ },
480
+ "utilities_services": {
481
+ "precision": 0.9951458433550532,
482
+ "recall": 0.9955376673374748,
483
+ "f1-score": 0.9953417167851285,
484
+ "support": 22858.0
485
+ },
486
+ "accuracy": 0.9929848825051134,
487
+ "macro avg": {
488
+ "precision": 0.9931871049140065,
489
+ "recall": 0.9932406399459772,
490
+ "f1-score": 0.9932028965957418,
491
+ "support": 240053.0
492
+ },
493
+ "weighted avg": {
494
+ "precision": 0.9929953943768574,
495
+ "recall": 0.9929848825051134,
496
+ "f1-score": 0.9929778167820927,
497
+ "support": 240053.0
498
+ }
499
+ },
500
+ "confusion_matrix": [
501
+ [
502
+ 22145,
503
+ 0,
504
+ 0,
505
+ 0,
506
+ 0,
507
+ 0,
508
+ 0,
509
+ 0,
510
+ 0,
511
+ 0
512
+ ],
513
+ [
514
+ 0,
515
+ 23671,
516
+ 44,
517
+ 1,
518
+ 0,
519
+ 1,
520
+ 0,
521
+ 1,
522
+ 0,
523
+ 0
524
+ ],
525
+ [
526
+ 0,
527
+ 1,
528
+ 28519,
529
+ 1,
530
+ 0,
531
+ 0,
532
+ 3,
533
+ 0,
534
+ 0,
535
+ 0
536
+ ],
537
+ [
538
+ 1,
539
+ 2,
540
+ 82,
541
+ 23618,
542
+ 0,
543
+ 0,
544
+ 0,
545
+ 77,
546
+ 1,
547
+ 0
548
+ ],
549
+ [
550
+ 0,
551
+ 1,
552
+ 2,
553
+ 1,
554
+ 22493,
555
+ 0,
556
+ 102,
557
+ 34,
558
+ 1,
559
+ 0
560
+ ],
561
+ [
562
+ 0,
563
+ 0,
564
+ 47,
565
+ 0,
566
+ 0,
567
+ 22421,
568
+ 0,
569
+ 220,
570
+ 0,
571
+ 0
572
+ ],
573
+ [
574
+ 0,
575
+ 0,
576
+ 0,
577
+ 0,
578
+ 56,
579
+ 0,
580
+ 22015,
581
+ 0,
582
+ 0,
583
+ 0
584
+ ],
585
+ [
586
+ 0,
587
+ 0,
588
+ 60,
589
+ 89,
590
+ 154,
591
+ 423,
592
+ 0,
593
+ 26995,
594
+ 0,
595
+ 1
596
+ ],
597
+ [
598
+ 0,
599
+ 1,
600
+ 65,
601
+ 0,
602
+ 0,
603
+ 0,
604
+ 0,
605
+ 0,
606
+ 23736,
607
+ 110
608
+ ],
609
+ [
610
+ 0,
611
+ 1,
612
+ 45,
613
+ 1,
614
+ 0,
615
+ 0,
616
+ 0,
617
+ 0,
618
+ 55,
619
+ 22756
620
+ ]
621
+ ],
622
+ "samples": 240053,
623
+ "confidence_threshold": 0.7,
624
+ "confidence_coverage": 0.9808167363040662,
625
+ "high_confidence_accuracy": 0.9995540416567564
626
+ },
627
+ "final_refit_training_rows": 1359243,
628
+ "accepted": true
629
+ }
reports/model_selection_validation.json ADDED
@@ -0,0 +1,213 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "models": [
3
+ {
4
+ "model_name": "tfidf",
5
+ "accuracy": 0.9922900091368398,
6
+ "macro_f1": 0.9924938150790148,
7
+ "weighted_f1": 0.9922797304638178,
8
+ "report": {
9
+ "charity_donations": {
10
+ "precision": 1.0,
11
+ "recall": 1.0,
12
+ "f1-score": 1.0,
13
+ "support": 22160.0
14
+ },
15
+ "entertainment_recreation": {
16
+ "precision": 0.9997045540877052,
17
+ "recall": 0.9978094194961665,
18
+ "f1-score": 0.9987560877906854,
19
+ "support": 23738.0
20
+ },
21
+ "financial_services": {
22
+ "precision": 0.9894933943618017,
23
+ "recall": 0.9998948806895827,
24
+ "f1-score": 0.9946669455191886,
25
+ "support": 28539.0
26
+ },
27
+ "food_dining": {
28
+ "precision": 0.9956344708894765,
29
+ "recall": 0.993840610072907,
30
+ "f1-score": 0.9947367317410724,
31
+ "support": 23866.0
32
+ },
33
+ "government_legal": {
34
+ "precision": 0.9895684097421203,
35
+ "recall": 0.9904552787237856,
36
+ "f1-score": 0.990011645614978,
37
+ "support": 22316.0
38
+ },
39
+ "healthcare_medical": {
40
+ "precision": 0.9781765910179901,
41
+ "recall": 0.9873373197326768,
42
+ "f1-score": 0.9827356075359401,
43
+ "support": 22744.0
44
+ },
45
+ "income": {
46
+ "precision": 0.99191313434192,
47
+ "recall": 0.9977151213270575,
48
+ "f1-score": 0.9948056682006653,
49
+ "support": 21883.0
50
+ },
51
+ "shopping_retail": {
52
+ "precision": 0.9869506249770903,
53
+ "recall": 0.9699207492795389,
54
+ "f1-score": 0.9783615850002725,
55
+ "support": 27760.0
56
+ },
57
+ "transportation": {
58
+ "precision": 0.998010245120867,
59
+ "recall": 0.9929657554441683,
60
+ "f1-score": 0.9954816097293189,
61
+ "support": 23741.0
62
+ },
63
+ "utilities_services": {
64
+ "precision": 0.9948188784395681,
65
+ "recall": 0.9959462993636126,
66
+ "f1-score": 0.9953822696580266,
67
+ "support": 22942.0
68
+ },
69
+ "accuracy": 0.9922900091368398,
70
+ "macro avg": {
71
+ "precision": 0.992427030297854,
72
+ "recall": 0.9925885434129496,
73
+ "f1-score": 0.9924938150790148,
74
+ "support": 239689.0
75
+ },
76
+ "weighted avg": {
77
+ "precision": 0.9923003908962669,
78
+ "recall": 0.9922900091368398,
79
+ "f1-score": 0.9922797304638178,
80
+ "support": 239689.0
81
+ }
82
+ },
83
+ "confusion_matrix": [
84
+ [
85
+ 22160,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0,
93
+ 0,
94
+ 0
95
+ ],
96
+ [
97
+ 0,
98
+ 23686,
99
+ 48,
100
+ 0,
101
+ 1,
102
+ 1,
103
+ 0,
104
+ 1,
105
+ 1,
106
+ 0
107
+ ],
108
+ [
109
+ 0,
110
+ 1,
111
+ 28536,
112
+ 0,
113
+ 0,
114
+ 0,
115
+ 1,
116
+ 1,
117
+ 0,
118
+ 0
119
+ ],
120
+ [
121
+ 0,
122
+ 2,
123
+ 57,
124
+ 23719,
125
+ 0,
126
+ 0,
127
+ 0,
128
+ 88,
129
+ 0,
130
+ 0
131
+ ],
132
+ [
133
+ 0,
134
+ 1,
135
+ 0,
136
+ 0,
137
+ 22103,
138
+ 0,
139
+ 176,
140
+ 36,
141
+ 0,
142
+ 0
143
+ ],
144
+ [
145
+ 0,
146
+ 0,
147
+ 58,
148
+ 0,
149
+ 0,
150
+ 22456,
151
+ 0,
152
+ 230,
153
+ 0,
154
+ 0
155
+ ],
156
+ [
157
+ 0,
158
+ 0,
159
+ 0,
160
+ 0,
161
+ 50,
162
+ 0,
163
+ 21833,
164
+ 0,
165
+ 0,
166
+ 0
167
+ ],
168
+ [
169
+ 0,
170
+ 0,
171
+ 49,
172
+ 103,
173
+ 181,
174
+ 500,
175
+ 0,
176
+ 26925,
177
+ 2,
178
+ 0
179
+ ],
180
+ [
181
+ 0,
182
+ 2,
183
+ 45,
184
+ 1,
185
+ 0,
186
+ 0,
187
+ 0,
188
+ 0,
189
+ 23574,
190
+ 119
191
+ ],
192
+ [
193
+ 0,
194
+ 1,
195
+ 46,
196
+ 0,
197
+ 1,
198
+ 0,
199
+ 1,
200
+ 0,
201
+ 44,
202
+ 22849
203
+ ]
204
+ ],
205
+ "samples": 239689,
206
+ "confidence_threshold": 0.7,
207
+ "confidence_coverage": 0.9790228170671161,
208
+ "high_confidence_accuracy": 0.999552546013185
209
+ }
210
+ ],
211
+ "selection_metric": "macro_f1",
212
+ "selected": "tfidf"
213
+ }
reports/split_summary.json ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "train_rows": 1119554,
3
+ "validation_rows": 239689,
4
+ "test_rows": 240053,
5
+ "india_holdout_rows": 78158,
6
+ "train_fingerprint": "009309f355bdd64ebde33be3ff81f2d133cd35f5fa2d4491eb467b188ff9c72d",
7
+ "validation_fingerprint": "6954bc4cd404e364f2392fbf0a0184aab6dffc44a2e955575735ee65781f2319",
8
+ "test_fingerprint": "e941e758ce659cbf6c9d791c1b37abf845e3390afb65b034749fcf559eb84638",
9
+ "india_holdout_fingerprint": "fa3416665596c8132d99b65f6e2d96372ee31723139466183bfde329fb95d814"
10
+ }
reports/tfidf_validation.json ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "model_name": "tfidf",
3
+ "accuracy": 0.9922900091368398,
4
+ "macro_f1": 0.9924938150790148,
5
+ "weighted_f1": 0.9922797304638178,
6
+ "report": {
7
+ "charity_donations": {
8
+ "precision": 1.0,
9
+ "recall": 1.0,
10
+ "f1-score": 1.0,
11
+ "support": 22160.0
12
+ },
13
+ "entertainment_recreation": {
14
+ "precision": 0.9997045540877052,
15
+ "recall": 0.9978094194961665,
16
+ "f1-score": 0.9987560877906854,
17
+ "support": 23738.0
18
+ },
19
+ "financial_services": {
20
+ "precision": 0.9894933943618017,
21
+ "recall": 0.9998948806895827,
22
+ "f1-score": 0.9946669455191886,
23
+ "support": 28539.0
24
+ },
25
+ "food_dining": {
26
+ "precision": 0.9956344708894765,
27
+ "recall": 0.993840610072907,
28
+ "f1-score": 0.9947367317410724,
29
+ "support": 23866.0
30
+ },
31
+ "government_legal": {
32
+ "precision": 0.9895684097421203,
33
+ "recall": 0.9904552787237856,
34
+ "f1-score": 0.990011645614978,
35
+ "support": 22316.0
36
+ },
37
+ "healthcare_medical": {
38
+ "precision": 0.9781765910179901,
39
+ "recall": 0.9873373197326768,
40
+ "f1-score": 0.9827356075359401,
41
+ "support": 22744.0
42
+ },
43
+ "income": {
44
+ "precision": 0.99191313434192,
45
+ "recall": 0.9977151213270575,
46
+ "f1-score": 0.9948056682006653,
47
+ "support": 21883.0
48
+ },
49
+ "shopping_retail": {
50
+ "precision": 0.9869506249770903,
51
+ "recall": 0.9699207492795389,
52
+ "f1-score": 0.9783615850002725,
53
+ "support": 27760.0
54
+ },
55
+ "transportation": {
56
+ "precision": 0.998010245120867,
57
+ "recall": 0.9929657554441683,
58
+ "f1-score": 0.9954816097293189,
59
+ "support": 23741.0
60
+ },
61
+ "utilities_services": {
62
+ "precision": 0.9948188784395681,
63
+ "recall": 0.9959462993636126,
64
+ "f1-score": 0.9953822696580266,
65
+ "support": 22942.0
66
+ },
67
+ "accuracy": 0.9922900091368398,
68
+ "macro avg": {
69
+ "precision": 0.992427030297854,
70
+ "recall": 0.9925885434129496,
71
+ "f1-score": 0.9924938150790148,
72
+ "support": 239689.0
73
+ },
74
+ "weighted avg": {
75
+ "precision": 0.9923003908962669,
76
+ "recall": 0.9922900091368398,
77
+ "f1-score": 0.9922797304638178,
78
+ "support": 239689.0
79
+ }
80
+ },
81
+ "confusion_matrix": [
82
+ [
83
+ 22160,
84
+ 0,
85
+ 0,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0
93
+ ],
94
+ [
95
+ 0,
96
+ 23686,
97
+ 48,
98
+ 0,
99
+ 1,
100
+ 1,
101
+ 0,
102
+ 1,
103
+ 1,
104
+ 0
105
+ ],
106
+ [
107
+ 0,
108
+ 1,
109
+ 28536,
110
+ 0,
111
+ 0,
112
+ 0,
113
+ 1,
114
+ 1,
115
+ 0,
116
+ 0
117
+ ],
118
+ [
119
+ 0,
120
+ 2,
121
+ 57,
122
+ 23719,
123
+ 0,
124
+ 0,
125
+ 0,
126
+ 88,
127
+ 0,
128
+ 0
129
+ ],
130
+ [
131
+ 0,
132
+ 1,
133
+ 0,
134
+ 0,
135
+ 22103,
136
+ 0,
137
+ 176,
138
+ 36,
139
+ 0,
140
+ 0
141
+ ],
142
+ [
143
+ 0,
144
+ 0,
145
+ 58,
146
+ 0,
147
+ 0,
148
+ 22456,
149
+ 0,
150
+ 230,
151
+ 0,
152
+ 0
153
+ ],
154
+ [
155
+ 0,
156
+ 0,
157
+ 0,
158
+ 0,
159
+ 50,
160
+ 0,
161
+ 21833,
162
+ 0,
163
+ 0,
164
+ 0
165
+ ],
166
+ [
167
+ 0,
168
+ 0,
169
+ 49,
170
+ 103,
171
+ 181,
172
+ 500,
173
+ 0,
174
+ 26925,
175
+ 2,
176
+ 0
177
+ ],
178
+ [
179
+ 0,
180
+ 2,
181
+ 45,
182
+ 1,
183
+ 0,
184
+ 0,
185
+ 0,
186
+ 0,
187
+ 23574,
188
+ 119
189
+ ],
190
+ [
191
+ 0,
192
+ 1,
193
+ 46,
194
+ 0,
195
+ 1,
196
+ 0,
197
+ 1,
198
+ 0,
199
+ 44,
200
+ 22849
201
+ ]
202
+ ],
203
+ "samples": 239689,
204
+ "confidence_threshold": 0.7,
205
+ "confidence_coverage": 0.9790228170671161,
206
+ "high_confidence_accuracy": 0.999552546013185
207
+ }
reports/tfidf_validation_country_metrics.json ADDED
@@ -0,0 +1,1037 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "Australia": {
3
+ "model_name": "tfidf-validation-Australia",
4
+ "accuracy": 0.993292498813786,
5
+ "macro_f1": 0.9932808795872896,
6
+ "weighted_f1": 0.9932796167283928,
7
+ "report": {
8
+ "charity_donations": {
9
+ "precision": 1.0,
10
+ "recall": 1.0,
11
+ "f1-score": 1.0,
12
+ "support": 4607.0
13
+ },
14
+ "entertainment_recreation": {
15
+ "precision": 1.0,
16
+ "recall": 1.0,
17
+ "f1-score": 1.0,
18
+ "support": 4630.0
19
+ },
20
+ "financial_services": {
21
+ "precision": 1.0,
22
+ "recall": 1.0,
23
+ "f1-score": 1.0,
24
+ "support": 4640.0
25
+ },
26
+ "food_dining": {
27
+ "precision": 0.9947445869245323,
28
+ "recall": 0.9968401095428692,
29
+ "f1-score": 0.9957912457912458,
30
+ "support": 4747.0
31
+ },
32
+ "government_legal": {
33
+ "precision": 0.990331795209844,
34
+ "recall": 0.9901142355008787,
35
+ "f1-score": 0.9902230034054708,
36
+ "support": 4552.0
37
+ },
38
+ "healthcare_medical": {
39
+ "precision": 0.9771708982291445,
40
+ "recall": 0.9922010398613518,
41
+ "f1-score": 0.9846286144254541,
42
+ "support": 4616.0
43
+ },
44
+ "income": {
45
+ "precision": 0.9913959991395999,
46
+ "recall": 0.9967560553633218,
47
+ "f1-score": 0.9940688018979834,
48
+ "support": 4624.0
49
+ },
50
+ "shopping_retail": {
51
+ "precision": 0.9877327491785323,
52
+ "recall": 0.9655246252676659,
53
+ "f1-score": 0.9765024363833243,
54
+ "support": 4670.0
55
+ },
56
+ "transportation": {
57
+ "precision": 0.9980636833046471,
58
+ "recall": 0.9935746412508032,
59
+ "f1-score": 0.9958141032521198,
60
+ "support": 4669.0
61
+ },
62
+ "utilities_services": {
63
+ "precision": 0.9935233160621761,
64
+ "recall": 0.9980481457384516,
65
+ "f1-score": 0.9957805907172996,
66
+ "support": 4611.0
67
+ },
68
+ "accuracy": 0.993292498813786,
69
+ "macro avg": {
70
+ "precision": 0.9932963028048476,
71
+ "recall": 0.9933058852525342,
72
+ "f1-score": 0.9932808795872896,
73
+ "support": 46366.0
74
+ },
75
+ "weighted avg": {
76
+ "precision": 0.9933072971261063,
77
+ "recall": 0.993292498813786,
78
+ "f1-score": 0.9932796167283928,
79
+ "support": 46366.0
80
+ }
81
+ },
82
+ "confusion_matrix": [
83
+ [
84
+ 4607,
85
+ 0,
86
+ 0,
87
+ 0,
88
+ 0,
89
+ 0,
90
+ 0,
91
+ 0,
92
+ 0,
93
+ 0
94
+ ],
95
+ [
96
+ 0,
97
+ 4630,
98
+ 0,
99
+ 0,
100
+ 0,
101
+ 0,
102
+ 0,
103
+ 0,
104
+ 0,
105
+ 0
106
+ ],
107
+ [
108
+ 0,
109
+ 0,
110
+ 4640,
111
+ 0,
112
+ 0,
113
+ 0,
114
+ 0,
115
+ 0,
116
+ 0,
117
+ 0
118
+ ],
119
+ [
120
+ 0,
121
+ 0,
122
+ 0,
123
+ 4732,
124
+ 0,
125
+ 0,
126
+ 0,
127
+ 15,
128
+ 0,
129
+ 0
130
+ ],
131
+ [
132
+ 0,
133
+ 0,
134
+ 0,
135
+ 0,
136
+ 4507,
137
+ 0,
138
+ 40,
139
+ 5,
140
+ 0,
141
+ 0
142
+ ],
143
+ [
144
+ 0,
145
+ 0,
146
+ 0,
147
+ 0,
148
+ 0,
149
+ 4580,
150
+ 0,
151
+ 36,
152
+ 0,
153
+ 0
154
+ ],
155
+ [
156
+ 0,
157
+ 0,
158
+ 0,
159
+ 0,
160
+ 15,
161
+ 0,
162
+ 4609,
163
+ 0,
164
+ 0,
165
+ 0
166
+ ],
167
+ [
168
+ 0,
169
+ 0,
170
+ 0,
171
+ 25,
172
+ 29,
173
+ 107,
174
+ 0,
175
+ 4509,
176
+ 0,
177
+ 0
178
+ ],
179
+ [
180
+ 0,
181
+ 0,
182
+ 0,
183
+ 0,
184
+ 0,
185
+ 0,
186
+ 0,
187
+ 0,
188
+ 4639,
189
+ 30
190
+ ],
191
+ [
192
+ 0,
193
+ 0,
194
+ 0,
195
+ 0,
196
+ 0,
197
+ 0,
198
+ 0,
199
+ 0,
200
+ 9,
201
+ 4602
202
+ ]
203
+ ],
204
+ "samples": 46366,
205
+ "confidence_threshold": 0.7,
206
+ "confidence_coverage": 0.9803735495837467,
207
+ "high_confidence_accuracy": 0.9994060190073918
208
+ },
209
+ "Canada": {
210
+ "model_name": "tfidf-validation-Canada",
211
+ "accuracy": 0.9933253208567627,
212
+ "macro_f1": 0.9933012012948575,
213
+ "weighted_f1": 0.9933166698331827,
214
+ "report": {
215
+ "charity_donations": {
216
+ "precision": 1.0,
217
+ "recall": 1.0,
218
+ "f1-score": 1.0,
219
+ "support": 4516.0
220
+ },
221
+ "entertainment_recreation": {
222
+ "precision": 1.0,
223
+ "recall": 1.0,
224
+ "f1-score": 1.0,
225
+ "support": 4848.0
226
+ },
227
+ "financial_services": {
228
+ "precision": 1.0,
229
+ "recall": 1.0,
230
+ "f1-score": 1.0,
231
+ "support": 4596.0
232
+ },
233
+ "food_dining": {
234
+ "precision": 0.9943538268506901,
235
+ "recall": 0.9949780288763339,
236
+ "f1-score": 0.9946658299341073,
237
+ "support": 4779.0
238
+ },
239
+ "government_legal": {
240
+ "precision": 0.9920566766852726,
241
+ "recall": 0.989083904109589,
242
+ "f1-score": 0.9905680600214363,
243
+ "support": 4672.0
244
+ },
245
+ "healthcare_medical": {
246
+ "precision": 0.9805279099956729,
247
+ "recall": 0.987794245858762,
248
+ "f1-score": 0.9841476655808903,
249
+ "support": 4588.0
250
+ },
251
+ "income": {
252
+ "precision": 0.9915346212285652,
253
+ "recall": 0.9982517482517482,
254
+ "f1-score": 0.9948818468909942,
255
+ "support": 4576.0
256
+ },
257
+ "shopping_retail": {
258
+ "precision": 0.9800173761946134,
259
+ "recall": 0.9686560755689136,
260
+ "f1-score": 0.9743036061325847,
261
+ "support": 4658.0
262
+ },
263
+ "transportation": {
264
+ "precision": 0.9984988205018228,
265
+ "recall": 0.9959358288770054,
266
+ "f1-score": 0.997215677875348,
267
+ "support": 4675.0
268
+ },
269
+ "utilities_services": {
270
+ "precision": 0.9959557258407833,
271
+ "recall": 0.9985061886470337,
272
+ "f1-score": 0.9972293265132139,
273
+ "support": 4686.0
274
+ },
275
+ "accuracy": 0.9933253208567627,
276
+ "macro avg": {
277
+ "precision": 0.993294495729742,
278
+ "recall": 0.9933206020189385,
279
+ "f1-score": 0.9933012012948575,
280
+ "support": 46594.0
281
+ },
282
+ "weighted avg": {
283
+ "precision": 0.993320634958268,
284
+ "recall": 0.9933253208567627,
285
+ "f1-score": 0.9933166698331827,
286
+ "support": 46594.0
287
+ }
288
+ },
289
+ "confusion_matrix": [
290
+ [
291
+ 4516,
292
+ 0,
293
+ 0,
294
+ 0,
295
+ 0,
296
+ 0,
297
+ 0,
298
+ 0,
299
+ 0,
300
+ 0
301
+ ],
302
+ [
303
+ 0,
304
+ 4848,
305
+ 0,
306
+ 0,
307
+ 0,
308
+ 0,
309
+ 0,
310
+ 0,
311
+ 0,
312
+ 0
313
+ ],
314
+ [
315
+ 0,
316
+ 0,
317
+ 4596,
318
+ 0,
319
+ 0,
320
+ 0,
321
+ 0,
322
+ 0,
323
+ 0,
324
+ 0
325
+ ],
326
+ [
327
+ 0,
328
+ 0,
329
+ 0,
330
+ 4755,
331
+ 0,
332
+ 0,
333
+ 0,
334
+ 24,
335
+ 0,
336
+ 0
337
+ ],
338
+ [
339
+ 0,
340
+ 0,
341
+ 0,
342
+ 0,
343
+ 4621,
344
+ 0,
345
+ 39,
346
+ 12,
347
+ 0,
348
+ 0
349
+ ],
350
+ [
351
+ 0,
352
+ 0,
353
+ 0,
354
+ 0,
355
+ 0,
356
+ 4532,
357
+ 0,
358
+ 56,
359
+ 0,
360
+ 0
361
+ ],
362
+ [
363
+ 0,
364
+ 0,
365
+ 0,
366
+ 0,
367
+ 8,
368
+ 0,
369
+ 4568,
370
+ 0,
371
+ 0,
372
+ 0
373
+ ],
374
+ [
375
+ 0,
376
+ 0,
377
+ 0,
378
+ 27,
379
+ 29,
380
+ 90,
381
+ 0,
382
+ 4512,
383
+ 0,
384
+ 0
385
+ ],
386
+ [
387
+ 0,
388
+ 0,
389
+ 0,
390
+ 0,
391
+ 0,
392
+ 0,
393
+ 0,
394
+ 0,
395
+ 4656,
396
+ 19
397
+ ],
398
+ [
399
+ 0,
400
+ 0,
401
+ 0,
402
+ 0,
403
+ 0,
404
+ 0,
405
+ 0,
406
+ 0,
407
+ 7,
408
+ 4679
409
+ ]
410
+ ],
411
+ "samples": 46594,
412
+ "confidence_threshold": 0.7,
413
+ "confidence_coverage": 0.9810705241018157,
414
+ "high_confidence_accuracy": 0.9995187259362969
415
+ },
416
+ "India": {
417
+ "model_name": "tfidf-validation-India",
418
+ "accuracy": 0.9886777043581557,
419
+ "macro_f1": 0.9895949199561311,
420
+ "weighted_f1": 0.988686656492644,
421
+ "report": {
422
+ "charity_donations": {
423
+ "precision": 1.0,
424
+ "recall": 1.0,
425
+ "f1-score": 1.0,
426
+ "support": 3841.0
427
+ },
428
+ "entertainment_recreation": {
429
+ "precision": 0.9985352584222641,
430
+ "recall": 0.9892205638474295,
431
+ "f1-score": 0.9938560866395918,
432
+ "support": 4824.0
433
+ },
434
+ "financial_services": {
435
+ "precision": 0.9703667481662591,
436
+ "recall": 0.9996977329974811,
437
+ "f1-score": 0.9848138957816377,
438
+ "support": 9925.0
439
+ },
440
+ "food_dining": {
441
+ "precision": 0.9961025641025641,
442
+ "recall": 0.9853896103896104,
443
+ "f1-score": 0.9907171274099765,
444
+ "support": 4928.0
445
+ },
446
+ "government_legal": {
447
+ "precision": 0.987936008392342,
448
+ "recall": 0.987936008392342,
449
+ "f1-score": 0.987936008392342,
450
+ "support": 3813.0
451
+ },
452
+ "healthcare_medical": {
453
+ "precision": 0.9786070834323746,
454
+ "recall": 0.9746685606060606,
455
+ "f1-score": 0.9766338512631954,
456
+ "support": 4224.0
457
+ },
458
+ "income": {
459
+ "precision": 0.9901700318809776,
460
+ "recall": 0.998392713635146,
461
+ "f1-score": 0.9942643724156329,
462
+ "support": 3733.0
463
+ },
464
+ "shopping_retail": {
465
+ "precision": 0.9919514884233738,
466
+ "recall": 0.9787859007832899,
467
+ "f1-score": 0.985324717993648,
468
+ "support": 9192.0
469
+ },
470
+ "transportation": {
471
+ "precision": 0.9979732468585326,
472
+ "recall": 0.9853912347408446,
473
+ "f1-score": 0.9916423320914308,
474
+ "support": 4997.0
475
+ },
476
+ "utilities_services": {
477
+ "precision": 0.9942765567765568,
478
+ "recall": 0.9872698340531939,
479
+ "f1-score": 0.9907608075738565,
480
+ "support": 4399.0
481
+ },
482
+ "accuracy": 0.9886777043581557,
483
+ "macro avg": {
484
+ "precision": 0.9905918986455244,
485
+ "recall": 0.9886752159445399,
486
+ "f1-score": 0.9895949199561311,
487
+ "support": 53876.0
488
+ },
489
+ "weighted avg": {
490
+ "precision": 0.9888126720043511,
491
+ "recall": 0.9886777043581557,
492
+ "f1-score": 0.988686656492644,
493
+ "support": 53876.0
494
+ }
495
+ },
496
+ "confusion_matrix": [
497
+ [
498
+ 3841,
499
+ 0,
500
+ 0,
501
+ 0,
502
+ 0,
503
+ 0,
504
+ 0,
505
+ 0,
506
+ 0,
507
+ 0
508
+ ],
509
+ [
510
+ 0,
511
+ 4772,
512
+ 48,
513
+ 0,
514
+ 1,
515
+ 1,
516
+ 0,
517
+ 1,
518
+ 1,
519
+ 0
520
+ ],
521
+ [
522
+ 0,
523
+ 1,
524
+ 9922,
525
+ 0,
526
+ 0,
527
+ 0,
528
+ 1,
529
+ 1,
530
+ 0,
531
+ 0
532
+ ],
533
+ [
534
+ 0,
535
+ 2,
536
+ 57,
537
+ 4856,
538
+ 0,
539
+ 0,
540
+ 0,
541
+ 13,
542
+ 0,
543
+ 0
544
+ ],
545
+ [
546
+ 0,
547
+ 1,
548
+ 0,
549
+ 0,
550
+ 3767,
551
+ 0,
552
+ 36,
553
+ 9,
554
+ 0,
555
+ 0
556
+ ],
557
+ [
558
+ 0,
559
+ 0,
560
+ 58,
561
+ 0,
562
+ 0,
563
+ 4117,
564
+ 0,
565
+ 49,
566
+ 0,
567
+ 0
568
+ ],
569
+ [
570
+ 0,
571
+ 0,
572
+ 0,
573
+ 0,
574
+ 6,
575
+ 0,
576
+ 3727,
577
+ 0,
578
+ 0,
579
+ 0
580
+ ],
581
+ [
582
+ 0,
583
+ 0,
584
+ 49,
585
+ 18,
586
+ 38,
587
+ 89,
588
+ 0,
589
+ 8997,
590
+ 1,
591
+ 0
592
+ ],
593
+ [
594
+ 0,
595
+ 2,
596
+ 45,
597
+ 1,
598
+ 0,
599
+ 0,
600
+ 0,
601
+ 0,
602
+ 4924,
603
+ 25
604
+ ],
605
+ [
606
+ 0,
607
+ 1,
608
+ 46,
609
+ 0,
610
+ 1,
611
+ 0,
612
+ 0,
613
+ 0,
614
+ 8,
615
+ 4343
616
+ ]
617
+ ],
618
+ "samples": 53876,
619
+ "confidence_threshold": 0.7,
620
+ "confidence_coverage": 0.9736803029178113,
621
+ "high_confidence_accuracy": 0.9995234282664227
622
+ },
623
+ "UK": {
624
+ "model_name": "tfidf-validation-UK",
625
+ "accuracy": 0.9936802734249048,
626
+ "macro_f1": 0.9936114329589104,
627
+ "weighted_f1": 0.9936673238616113,
628
+ "report": {
629
+ "charity_donations": {
630
+ "precision": 1.0,
631
+ "recall": 1.0,
632
+ "f1-score": 1.0,
633
+ "support": 4708.0
634
+ },
635
+ "entertainment_recreation": {
636
+ "precision": 1.0,
637
+ "recall": 1.0,
638
+ "f1-score": 1.0,
639
+ "support": 4706.0
640
+ },
641
+ "financial_services": {
642
+ "precision": 1.0,
643
+ "recall": 1.0,
644
+ "f1-score": 1.0,
645
+ "support": 4752.0
646
+ },
647
+ "food_dining": {
648
+ "precision": 0.99721269296741,
649
+ "recall": 0.998068669527897,
650
+ "f1-score": 0.9976404976404977,
651
+ "support": 4660.0
652
+ },
653
+ "government_legal": {
654
+ "precision": 0.9875909285408644,
655
+ "recall": 0.9922613929492692,
656
+ "f1-score": 0.9899206519408107,
657
+ "support": 4652.0
658
+ },
659
+ "healthcare_medical": {
660
+ "precision": 0.9787690328114947,
661
+ "recall": 0.9906663772520078,
662
+ "f1-score": 0.9846817691477886,
663
+ "support": 4607.0
664
+ },
665
+ "income": {
666
+ "precision": 0.9933862433862434,
667
+ "recall": 0.99734395750332,
668
+ "f1-score": 0.9953611663353215,
669
+ "support": 4518.0
670
+ },
671
+ "shopping_retail": {
672
+ "precision": 0.9870939029817535,
673
+ "recall": 0.9656073138876796,
674
+ "f1-score": 0.9762323943661971,
675
+ "support": 4594.0
676
+ },
677
+ "transportation": {
678
+ "precision": 0.9980574142024606,
679
+ "recall": 0.9941947968178886,
680
+ "f1-score": 0.996122361051271,
681
+ "support": 4651.0
682
+ },
683
+ "utilities_services": {
684
+ "precision": 0.994244297591132,
685
+ "recall": 0.9980740423710678,
686
+ "f1-score": 0.9961554891072192,
687
+ "support": 4673.0
688
+ },
689
+ "accuracy": 0.9936802734249048,
690
+ "macro avg": {
691
+ "precision": 0.9936354512481358,
692
+ "recall": 0.9936216550309129,
693
+ "f1-score": 0.9936114329589104,
694
+ "support": 46521.0
695
+ },
696
+ "weighted avg": {
697
+ "precision": 0.993688230196664,
698
+ "recall": 0.9936802734249048,
699
+ "f1-score": 0.9936673238616113,
700
+ "support": 46521.0
701
+ }
702
+ },
703
+ "confusion_matrix": [
704
+ [
705
+ 4708,
706
+ 0,
707
+ 0,
708
+ 0,
709
+ 0,
710
+ 0,
711
+ 0,
712
+ 0,
713
+ 0,
714
+ 0
715
+ ],
716
+ [
717
+ 0,
718
+ 4706,
719
+ 0,
720
+ 0,
721
+ 0,
722
+ 0,
723
+ 0,
724
+ 0,
725
+ 0,
726
+ 0
727
+ ],
728
+ [
729
+ 0,
730
+ 0,
731
+ 4752,
732
+ 0,
733
+ 0,
734
+ 0,
735
+ 0,
736
+ 0,
737
+ 0,
738
+ 0
739
+ ],
740
+ [
741
+ 0,
742
+ 0,
743
+ 0,
744
+ 4651,
745
+ 0,
746
+ 0,
747
+ 0,
748
+ 9,
749
+ 0,
750
+ 0
751
+ ],
752
+ [
753
+ 0,
754
+ 0,
755
+ 0,
756
+ 0,
757
+ 4616,
758
+ 0,
759
+ 30,
760
+ 6,
761
+ 0,
762
+ 0
763
+ ],
764
+ [
765
+ 0,
766
+ 0,
767
+ 0,
768
+ 0,
769
+ 0,
770
+ 4564,
771
+ 0,
772
+ 43,
773
+ 0,
774
+ 0
775
+ ],
776
+ [
777
+ 0,
778
+ 0,
779
+ 0,
780
+ 0,
781
+ 12,
782
+ 0,
783
+ 4506,
784
+ 0,
785
+ 0,
786
+ 0
787
+ ],
788
+ [
789
+ 0,
790
+ 0,
791
+ 0,
792
+ 13,
793
+ 46,
794
+ 99,
795
+ 0,
796
+ 4436,
797
+ 0,
798
+ 0
799
+ ],
800
+ [
801
+ 0,
802
+ 0,
803
+ 0,
804
+ 0,
805
+ 0,
806
+ 0,
807
+ 0,
808
+ 0,
809
+ 4624,
810
+ 27
811
+ ],
812
+ [
813
+ 0,
814
+ 0,
815
+ 0,
816
+ 0,
817
+ 0,
818
+ 0,
819
+ 0,
820
+ 0,
821
+ 9,
822
+ 4664
823
+ ]
824
+ ],
825
+ "samples": 46521,
826
+ "confidence_threshold": 0.7,
827
+ "confidence_coverage": 0.9809548376002235,
828
+ "high_confidence_accuracy": 0.9996932179248383
829
+ },
830
+ "USA": {
831
+ "model_name": "tfidf-validation-USA",
832
+ "accuracy": 0.9930501597168264,
833
+ "macro_f1": 0.9930662776516987,
834
+ "weighted_f1": 0.9930358443177167,
835
+ "report": {
836
+ "charity_donations": {
837
+ "precision": 1.0,
838
+ "recall": 1.0,
839
+ "f1-score": 1.0,
840
+ "support": 4488.0
841
+ },
842
+ "entertainment_recreation": {
843
+ "precision": 1.0,
844
+ "recall": 1.0,
845
+ "f1-score": 1.0,
846
+ "support": 4730.0
847
+ },
848
+ "financial_services": {
849
+ "precision": 1.0,
850
+ "recall": 1.0,
851
+ "f1-score": 1.0,
852
+ "support": 4626.0
853
+ },
854
+ "food_dining": {
855
+ "precision": 0.9957850368809273,
856
+ "recall": 0.9943181818181818,
857
+ "f1-score": 0.9950510687585553,
858
+ "support": 4752.0
859
+ },
860
+ "government_legal": {
861
+ "precision": 0.9896551724137931,
862
+ "recall": 0.9924357034795764,
863
+ "f1-score": 0.9910434876443294,
864
+ "support": 4627.0
865
+ },
866
+ "healthcare_medical": {
867
+ "precision": 0.9759313520301381,
868
+ "recall": 0.9902314716500319,
869
+ "f1-score": 0.9830294086644883,
870
+ "support": 4709.0
871
+ },
872
+ "income": {
873
+ "precision": 0.9928170594837261,
874
+ "recall": 0.9979693140794224,
875
+ "f1-score": 0.9953865196354226,
876
+ "support": 4432.0
877
+ },
878
+ "shopping_retail": {
879
+ "precision": 0.9830694810905892,
880
+ "recall": 0.9623331898407232,
881
+ "f1-score": 0.9725908201000653,
882
+ "support": 4646.0
883
+ },
884
+ "transportation": {
885
+ "precision": 0.9974699557242251,
886
+ "recall": 0.9962097283638661,
887
+ "f1-score": 0.9968394437420987,
888
+ "support": 4749.0
889
+ },
890
+ "utilities_services": {
891
+ "precision": 0.9960690107010264,
892
+ "recall": 0.9973759020336759,
893
+ "f1-score": 0.996722027972028,
894
+ "support": 4573.0
895
+ },
896
+ "accuracy": 0.9930501597168264,
897
+ "macro avg": {
898
+ "precision": 0.9930797068324425,
899
+ "recall": 0.9930873491265478,
900
+ "f1-score": 0.9930662776516987,
901
+ "support": 46332.0
902
+ },
903
+ "weighted avg": {
904
+ "precision": 0.9930562060169127,
905
+ "recall": 0.9930501597168264,
906
+ "f1-score": 0.9930358443177167,
907
+ "support": 46332.0
908
+ }
909
+ },
910
+ "confusion_matrix": [
911
+ [
912
+ 4488,
913
+ 0,
914
+ 0,
915
+ 0,
916
+ 0,
917
+ 0,
918
+ 0,
919
+ 0,
920
+ 0,
921
+ 0
922
+ ],
923
+ [
924
+ 0,
925
+ 4730,
926
+ 0,
927
+ 0,
928
+ 0,
929
+ 0,
930
+ 0,
931
+ 0,
932
+ 0,
933
+ 0
934
+ ],
935
+ [
936
+ 0,
937
+ 0,
938
+ 4626,
939
+ 0,
940
+ 0,
941
+ 0,
942
+ 0,
943
+ 0,
944
+ 0,
945
+ 0
946
+ ],
947
+ [
948
+ 0,
949
+ 0,
950
+ 0,
951
+ 4725,
952
+ 0,
953
+ 0,
954
+ 0,
955
+ 27,
956
+ 0,
957
+ 0
958
+ ],
959
+ [
960
+ 0,
961
+ 0,
962
+ 0,
963
+ 0,
964
+ 4592,
965
+ 0,
966
+ 31,
967
+ 4,
968
+ 0,
969
+ 0
970
+ ],
971
+ [
972
+ 0,
973
+ 0,
974
+ 0,
975
+ 0,
976
+ 0,
977
+ 4663,
978
+ 0,
979
+ 46,
980
+ 0,
981
+ 0
982
+ ],
983
+ [
984
+ 0,
985
+ 0,
986
+ 0,
987
+ 0,
988
+ 9,
989
+ 0,
990
+ 4423,
991
+ 0,
992
+ 0,
993
+ 0
994
+ ],
995
+ [
996
+ 0,
997
+ 0,
998
+ 0,
999
+ 20,
1000
+ 39,
1001
+ 115,
1002
+ 0,
1003
+ 4471,
1004
+ 1,
1005
+ 0
1006
+ ],
1007
+ [
1008
+ 0,
1009
+ 0,
1010
+ 0,
1011
+ 0,
1012
+ 0,
1013
+ 0,
1014
+ 0,
1015
+ 0,
1016
+ 4731,
1017
+ 18
1018
+ ],
1019
+ [
1020
+ 0,
1021
+ 0,
1022
+ 0,
1023
+ 0,
1024
+ 0,
1025
+ 0,
1026
+ 1,
1027
+ 0,
1028
+ 11,
1029
+ 4561
1030
+ ]
1031
+ ],
1032
+ "samples": 46332,
1033
+ "confidence_threshold": 0.7,
1034
+ "confidence_coverage": 0.9798843132176466,
1035
+ "high_confidence_accuracy": 0.999625550660793
1036
+ }
1037
+ }
reports/training_sampling.json ADDED
@@ -0,0 +1,43 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "full_train_rows": 1119554,
3
+ "baseline_rows": 1119554,
4
+ "transformer_rows": 1000000,
5
+ "baseline_country_counts": {
6
+ "India": 251754,
7
+ "Canada": 217494,
8
+ "Australia": 217170,
9
+ "UK": 216735,
10
+ "USA": 216401
11
+ },
12
+ "transformer_country_counts": {
13
+ "India": 232158,
14
+ "Canada": 192284,
15
+ "Australia": 192072,
16
+ "UK": 191777,
17
+ "USA": 191709
18
+ },
19
+ "baseline_class_counts": {
20
+ "financial_services": 133254,
21
+ "shopping_retail": 128988,
22
+ "food_dining": 111144,
23
+ "transportation": 111123,
24
+ "entertainment_recreation": 110799,
25
+ "utilities_services": 107050,
26
+ "healthcare_medical": 106117,
27
+ "government_legal": 104572,
28
+ "charity_donations": 104103,
29
+ "income": 102404
30
+ },
31
+ "transformer_class_counts": {
32
+ "financial_services": 117051,
33
+ "shopping_retail": 114479,
34
+ "food_dining": 99417,
35
+ "transportation": 99361,
36
+ "entertainment_recreation": 99241,
37
+ "utilities_services": 96078,
38
+ "healthcare_medical": 95281,
39
+ "government_legal": 93524,
40
+ "charity_donations": 93313,
41
+ "income": 92255
42
+ }
43
+ }