KhangTruong commited on
Commit
8320835
·
verified ·
1 Parent(s): 33e8110

docs(evaluation): update model card with CASIA v2.0 cross-evaluation benchmark

Browse files
Files changed (30) hide show
  1. .gitattributes +2 -0
  2. README.md +115 -0
  3. cross_evaluation/cross_eval_matrix.json +22 -0
  4. cross_evaluation/cross_evaluation.log +23 -0
  5. cross_evaluation/cross_evaluation_report.json +664 -0
  6. cross_evaluation/cross_evaluation_report.md +61 -0
  7. cross_evaluation/eval_casia_v2.0_report.json +345 -0
  8. cross_evaluation/eval_casia_v2.0_report.md +73 -0
  9. cross_evaluation/illustrations/cross_eval_auroc_heatmap.png +0 -0
  10. cross_evaluation/illustrations/cross_eval_f1_heatmap.png +0 -0
  11. cross_evaluation/illustrations/cross_eval_iou_heatmap.png +0 -0
  12. cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png +0 -0
  13. cross_evaluation/illustrations/sample_predictions_casia_v2.0.png +3 -0
  14. cross_evaluation/master_cross_evaluation_report.json +1352 -0
  15. cross_evaluation/master_cross_evaluation_report.md +110 -0
  16. manifest.json +1 -1
  17. versions/v1/cross_evaluation/cross_eval_matrix.json +22 -0
  18. versions/v1/cross_evaluation/cross_evaluation.log +23 -0
  19. versions/v1/cross_evaluation/cross_evaluation_report.json +664 -0
  20. versions/v1/cross_evaluation/cross_evaluation_report.md +61 -0
  21. versions/v1/cross_evaluation/eval_casia_v2.0_report.json +345 -0
  22. versions/v1/cross_evaluation/eval_casia_v2.0_report.md +73 -0
  23. versions/v1/cross_evaluation/illustrations/cross_eval_auroc_heatmap.png +0 -0
  24. versions/v1/cross_evaluation/illustrations/cross_eval_f1_heatmap.png +0 -0
  25. versions/v1/cross_evaluation/illustrations/cross_eval_iou_heatmap.png +0 -0
  26. versions/v1/cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png +0 -0
  27. versions/v1/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png +3 -0
  28. versions/v1/cross_evaluation/master_cross_evaluation_report.json +1352 -0
  29. versions/v1/cross_evaluation/master_cross_evaluation_report.md +110 -0
  30. versions/v1/manifest.json +1 -1
.gitattributes CHANGED
@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ versions/v1/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png filter=lfs diff=lfs merge=lfs -text
37
+ cross_evaluation/illustrations/sample_predictions_casia_v2.0.png filter=lfs diff=lfs merge=lfs -text
README.md CHANGED
@@ -24,6 +24,121 @@ Official repository for **PIM** checkpoints, evaluated on AI-generated image det
24
  - **Best Validation Score:** `0.852` (Epoch `10`)
25
  - **Total Training Epochs:** `10`
26
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
27
  ---
28
 
29
  ## 🚀 Quickstart & Checkpoint Usage
 
24
  - **Best Validation Score:** `0.852` (Epoch `10`)
25
  - **Total Training Epochs:** `10`
26
 
27
+
28
+ ---
29
+
30
+ ## 📊 Master Cross-Evaluation Results
31
+
32
+ # SID-UNet Master Cross-Evaluation Report
33
+
34
+ ## Overview
35
+
36
+ - **Evaluated Checkpoints**: 2
37
+ - **Evaluated Configurations**: 1
38
+ - **Total Cross-Evaluation Runs**: 2
39
+
40
+ ## Cross-Evaluation Matrices
41
+
42
+ ### Mean IoU (Intersection over Union) Cross-Evaluation Matrix
43
+
44
+ | Checkpoint \ Config | eval_casia_v2.0 |
45
+ |-----------------------|-------------------|
46
+ | sid_focal_hrnet | 0.0000 |
47
+ | sid_pim_swin | 0.2000 |
48
+
49
+ ### Dice / Pixel F1 Score Cross-Evaluation Matrix
50
+
51
+ | Checkpoint \ Config | eval_casia_v2.0 |
52
+ |-----------------------|-------------------|
53
+ | sid_focal_hrnet | 0.0000 |
54
+ | sid_pim_swin | 0.2000 |
55
+
56
+ ### AUROC (Pixel-level ROC-AUC) Cross-Evaluation Matrix
57
+
58
+ | Checkpoint \ Config | eval_casia_v2.0 |
59
+ |-----------------------|-------------------|
60
+ | sid_focal_hrnet | 0.0063 |
61
+ | sid_pim_swin | 1.0000 |
62
+
63
+ ### Pixel Accuracy Cross-Evaluation Matrix
64
+
65
+ | Checkpoint \ Config | eval_casia_v2.0 |
66
+ |-----------------------|-------------------|
67
+ | sid_focal_hrnet | 0.0063 |
68
+ | sid_pim_swin | 0.9088 |
69
+
70
+ ### Evaluation Total Loss Cross-Evaluation Matrix
71
+
72
+ | Checkpoint \ Config | eval_casia_v2.0 |
73
+ |-----------------------|-------------------|
74
+ | sid_focal_hrnet | 1.4154 |
75
+ | sid_pim_swin | 0.6689 |
76
+
77
+ ## Master Consolidated Ranking
78
+
79
+ | Checkpoint | Config (Dataset) | Split | Samples | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Precision | Recall | Specificity | Class Acc |
80
+ |-----------------|------------------------------------------|------------|-----------|--------|--------|-------------|---------|-------------|-------------|----------|---------------|-------------|
81
+ | sid_pim_swin | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 0.6689 | 0.2000 | 0.2000 | 1.0000 | 0.9088 | 0.2000 | 0.2000 | 0.9088 | N/A |
82
+ | sid_focal_hrnet | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 1.4154 | 0.0000 | 0.0000 | 0.0063 | 0.0063 | 0.0000 | 0.0000 | 0.0063 | N/A |
83
+
84
+
85
+ ## 🖼️ Cross-Evaluation Heatmaps & Visual Illustrations
86
+
87
+ ### Cross Eval Iou Heatmap
88
+
89
+ ![Cross Eval Iou Heatmap](illustrations/cross_eval_iou_heatmap.png)
90
+
91
+ ### Cross Eval F1 Heatmap
92
+
93
+ ![Cross Eval F1 Heatmap](illustrations/cross_eval_f1_heatmap.png)
94
+
95
+ ### Cross Eval Pixel Acc Heatmap
96
+
97
+ ![Cross Eval Pixel Acc Heatmap](illustrations/cross_eval_pixel_acc_heatmap.png)
98
+
99
+ ### Cross Eval Auroc Heatmap
100
+
101
+ ![Cross Eval Auroc Heatmap](illustrations/cross_eval_auroc_heatmap.png)
102
+
103
+ ### Sample Predictions Casia V2.0
104
+
105
+ ![Sample Predictions Casia V2.0](illustrations/sample_predictions_casia_v2.0.png)
106
+
107
+ ---
108
+
109
+ ## Per-Checkpoint Deep-Dive
110
+
111
+ ### Checkpoint: sid_focal_hrnet
112
+
113
+ - **Path**: `outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt`
114
+
115
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
116
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
117
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
118
+
119
+
120
+ ### Checkpoint: sid_pim_swin
121
+
122
+ - **Path**: `outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt`
123
+
124
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
125
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
126
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
127
+
128
+
129
+ ---
130
+
131
+ ## Per-Configuration / Dataset Deep-Dive
132
+
133
+ ### Configuration: eval_casia_v2.0 (Dataset: KhangTruong/Casia-v2.0)
134
+
135
+ - **Config Path**: `configs/cross-eval/casia_v2.0.yaml`
136
+
137
+ | Checkpoint | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
138
+ |-----------------|--------|-------|-------------|---------|-------------|-----------|
139
+ | sid_focal_hrnet | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
140
+ | sid_pim_swin | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
141
+
142
  ---
143
 
144
  ## 🚀 Quickstart & Checkpoint Usage
cross_evaluation/cross_eval_matrix.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "iou": {
3
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
4
+ "sid_pim_swin:eval_casia_v2.0": 0.2
5
+ },
6
+ "dice_f1": {
7
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
8
+ "sid_pim_swin:eval_casia_v2.0": 0.2
9
+ },
10
+ "auroc": {
11
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
12
+ "sid_pim_swin:eval_casia_v2.0": 1.0
13
+ },
14
+ "pixel_acc": {
15
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
16
+ "sid_pim_swin:eval_casia_v2.0": 0.9088134765625
17
+ },
18
+ "loss": {
19
+ "sid_focal_hrnet:eval_casia_v2.0": 1.415394163131714,
20
+ "sid_pim_swin:eval_casia_v2.0": 0.6688968300819397
21
+ }
22
+ }
cross_evaluation/cross_evaluation.log ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2026-11-12 06:54:39] [INFO] ===========================================================================
2
+ [2026-11-12 06:54:39] [INFO] 🚀 Starting Cross-Evaluation Matrix
3
+ [2026-11-12 06:54:39] [INFO] Checkpoints (2): ['outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt', 'outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt']
4
+ [2026-11-12 06:54:39] [INFO] Configurations (1): ['configs/cross-eval/casia_v2.0.yaml']
5
+ [2026-11-12 06:54:39] [INFO] Total Target Pairs: 2
6
+ [2026-11-12 06:54:39] [INFO] Collision Detection / Skip: True
7
+ [2026-11-12 06:54:39] [INFO] Kaggle Environment: True | Available GPUs: 1
8
+ [2026-11-12 06:54:39] [INFO] Multi-Device DataParallel: default (auto-detected per configuration)
9
+ [2026-11-12 06:54:39] [INFO] Master Output Directory: outputs/cross_evaluation
10
+ [2026-11-12 06:54:39] [INFO] ===========================================================================
11
+ [2026-11-12 06:54:39] [INFO]
12
+ [1/2] Evaluating Checkpoint 'checkpoint_best.pt' with Config 'casia_v2.0.yaml'...
13
+ [2026-11-12 06:54:40] [INFO] Downloaded FOCAL pretrained weights from HF 'KhangTruong/FOCAL/FOCAL_HRNet_weights.pth': /root/.cache/huggingface/hub/models--KhangTruong--FOCAL/snapshots/8d26d2a9110dc9aaac8bdde0f56d135d8dc71496/FOCAL_HRNet_weights.pth
14
+ [2026-11-12 06:54:40] [INFO] ✅ Loaded genuine FOCAL hrnet pretrained weights from: /root/.cache/huggingface/hub/models--KhangTruong--FOCAL/snapshots/8d26d2a9110dc9aaac8bdde0f56d135d8dc71496/FOCAL_HRNet_weights.pth (missing: 0, unexpected: 2)
15
+ [2026-11-12 06:54:56] [INFO] -> Dataset: 'KhangTruong/Casia-v2.0' (cross_test) | Loss: 1.4154 | IoU: 0.0000 | F1: 0.0000 | AUROC: 0.0063 | PixelAcc: 0.0063
16
+ [2026-11-12 06:54:56] [INFO]
17
+ [2/2] Evaluating Checkpoint 'checkpoint_best.pt' with Config 'casia_v2.0.yaml'...
18
+ [2026-11-12 06:54:59] [INFO] Downloaded PIM pretrained weights from HF 'KhangTruong/PIM/ckpt.pth': /root/.cache/huggingface/hub/models--KhangTruong--PIM/snapshots/33e8110fd64d4dbb034471144dea5d61060277cd/ckpt.pth
19
+ [2026-11-12 06:55:00] [INFO] ✅ Loaded genuine PIM pretrained weights from: /root/.cache/huggingface/hub/models--KhangTruong--PIM/snapshots/33e8110fd64d4dbb034471144dea5d61060277cd/ckpt.pth (missing: 0, unexpected: 0)
20
+ [2026-11-12 06:55:15] [INFO] -> Dataset: 'KhangTruong/Casia-v2.0' (cross_test) | Loss: 0.6689 | IoU: 0.2000 | F1: 0.2000 | AUROC: 1.0000 | PixelAcc: 0.9088
21
+ [2026-11-12 06:55:15] [INFO] ✅ Checkpoint neighbor report saved at: /content/ai-detection-unet-gpu/outputs/RUN/focal_hrnet/cross_eval_reports/cross_evaluation_report.md
22
+ [2026-11-12 06:55:15] [INFO] ✅ Checkpoint neighbor report saved at: /content/ai-detection-unet-gpu/outputs/RUN/pim_swin/cross_eval_reports/cross_evaluation_report.md
23
+ [2026-11-12 06:55:17] [INFO] Generated 4 cross-evaluation heatmaps in outputs/cross_evaluation/illustrations
cross_evaluation/cross_evaluation_report.json ADDED
@@ -0,0 +1,664 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
3
+ "checkpoint_name": "sid_pim_swin",
4
+ "total_configs": 1,
5
+ "results": [
6
+ {
7
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
8
+ "checkpoint_name": "sid_pim_swin",
9
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
10
+ "config_stem": "casia_v2.0",
11
+ "config_name": "eval_casia_v2.0",
12
+ "dataset_name": "KhangTruong/Casia-v2.0",
13
+ "eval_split": "cross_test",
14
+ "total_evaluated_samples": 20,
15
+ "metrics": {
16
+ "eval_split": "cross_test",
17
+ "eval_total_loss": 0.6688968300819397,
18
+ "iou": 0.2,
19
+ "dice": 0.2,
20
+ "f1": 0.2,
21
+ "pixel_f1": 0.2,
22
+ "auroc": 1.0,
23
+ "pixel_auroc": 1.0,
24
+ "pixel_acc": 0.9088134765625,
25
+ "precision": 0.2,
26
+ "recall": 0.2,
27
+ "specificity": 0.9088134765625,
28
+ "iou_all": 0.2,
29
+ "dice_all": 0.2,
30
+ "f1_all": 0.2,
31
+ "positive_samples": 0,
32
+ "zero_positive_samples": 20,
33
+ "iou_positive": 0.2,
34
+ "dice_positive": 0.2,
35
+ "f1_positive": 0.2,
36
+ "precision_positive": 0.2,
37
+ "recall_positive": 0.2,
38
+ "total_evaluated_samples": 20,
39
+ "ablation_summary": {
40
+ "Baseline (Raw UNet)": {
41
+ "iou": 0.15,
42
+ "dice": 0.15,
43
+ "f1": 0.15,
44
+ "pixel_f1": 0.15,
45
+ "auroc": 1.0,
46
+ "pixel_auroc": 1.0,
47
+ "pixel_acc": 0.908416748046875,
48
+ "precision": 0.15,
49
+ "recall": 0.15,
50
+ "specificity": 0.908416748046875,
51
+ "iou_all": 0.15,
52
+ "dice_all": 0.15,
53
+ "f1_all": 0.15,
54
+ "positive_samples": 0,
55
+ "zero_positive_samples": 20,
56
+ "iou_positive": 0.15,
57
+ "dice_positive": 0.15,
58
+ "f1_positive": 0.15,
59
+ "precision_positive": 0.15,
60
+ "recall_positive": 0.15
61
+ },
62
+ "+ Post-Processing": {
63
+ "iou": 0.2,
64
+ "dice": 0.2,
65
+ "f1": 0.2,
66
+ "pixel_f1": 0.2,
67
+ "auroc": 1.0,
68
+ "pixel_auroc": 1.0,
69
+ "pixel_acc": 0.9088134765625,
70
+ "precision": 0.2,
71
+ "recall": 0.2,
72
+ "specificity": 0.9088134765625,
73
+ "iou_all": 0.2,
74
+ "dice_all": 0.2,
75
+ "f1_all": 0.2,
76
+ "positive_samples": 0,
77
+ "zero_positive_samples": 20,
78
+ "iou_positive": 0.2,
79
+ "dice_positive": 0.2,
80
+ "f1_positive": 0.2,
81
+ "precision_positive": 0.2,
82
+ "recall_positive": 0.2
83
+ }
84
+ },
85
+ "per_dataset_metrics": {}
86
+ },
87
+ "overall_metrics": {
88
+ "eval_split": "cross_test",
89
+ "eval_total_loss": 0.6688968300819397,
90
+ "iou": 0.2,
91
+ "dice": 0.2,
92
+ "f1": 0.2,
93
+ "pixel_f1": 0.2,
94
+ "auroc": 1.0,
95
+ "pixel_auroc": 1.0,
96
+ "pixel_acc": 0.9088134765625,
97
+ "precision": 0.2,
98
+ "recall": 0.2,
99
+ "specificity": 0.9088134765625,
100
+ "iou_all": 0.2,
101
+ "dice_all": 0.2,
102
+ "f1_all": 0.2,
103
+ "positive_samples": 0,
104
+ "zero_positive_samples": 20,
105
+ "iou_positive": 0.2,
106
+ "dice_positive": 0.2,
107
+ "f1_positive": 0.2,
108
+ "precision_positive": 0.2,
109
+ "recall_positive": 0.2,
110
+ "total_evaluated_samples": 20,
111
+ "ablation_summary": {
112
+ "Baseline (Raw UNet)": {
113
+ "iou": 0.15,
114
+ "dice": 0.15,
115
+ "f1": 0.15,
116
+ "pixel_f1": 0.15,
117
+ "auroc": 1.0,
118
+ "pixel_auroc": 1.0,
119
+ "pixel_acc": 0.908416748046875,
120
+ "precision": 0.15,
121
+ "recall": 0.15,
122
+ "specificity": 0.908416748046875,
123
+ "iou_all": 0.15,
124
+ "dice_all": 0.15,
125
+ "f1_all": 0.15,
126
+ "positive_samples": 0,
127
+ "zero_positive_samples": 20,
128
+ "iou_positive": 0.15,
129
+ "dice_positive": 0.15,
130
+ "f1_positive": 0.15,
131
+ "precision_positive": 0.15,
132
+ "recall_positive": 0.15
133
+ },
134
+ "+ Post-Processing": {
135
+ "iou": 0.2,
136
+ "dice": 0.2,
137
+ "f1": 0.2,
138
+ "pixel_f1": 0.2,
139
+ "auroc": 1.0,
140
+ "pixel_auroc": 1.0,
141
+ "pixel_acc": 0.9088134765625,
142
+ "precision": 0.2,
143
+ "recall": 0.2,
144
+ "specificity": 0.9088134765625,
145
+ "iou_all": 0.2,
146
+ "dice_all": 0.2,
147
+ "f1_all": 0.2,
148
+ "positive_samples": 0,
149
+ "zero_positive_samples": 20,
150
+ "iou_positive": 0.2,
151
+ "dice_positive": 0.2,
152
+ "f1_positive": 0.2,
153
+ "precision_positive": 0.2,
154
+ "recall_positive": 0.2
155
+ }
156
+ },
157
+ "per_dataset_metrics": {}
158
+ },
159
+ "ablation_metrics": {
160
+ "Baseline (Raw UNet)": {
161
+ "iou": 0.15,
162
+ "dice": 0.15,
163
+ "f1": 0.15,
164
+ "pixel_f1": 0.15,
165
+ "auroc": 1.0,
166
+ "pixel_auroc": 1.0,
167
+ "pixel_acc": 0.908416748046875,
168
+ "precision": 0.15,
169
+ "recall": 0.15,
170
+ "specificity": 0.908416748046875,
171
+ "iou_all": 0.15,
172
+ "dice_all": 0.15,
173
+ "f1_all": 0.15,
174
+ "positive_samples": 0,
175
+ "zero_positive_samples": 20,
176
+ "iou_positive": 0.15,
177
+ "dice_positive": 0.15,
178
+ "f1_positive": 0.15,
179
+ "precision_positive": 0.15,
180
+ "recall_positive": 0.15
181
+ },
182
+ "+ Post-Processing": {
183
+ "iou": 0.2,
184
+ "dice": 0.2,
185
+ "f1": 0.2,
186
+ "pixel_f1": 0.2,
187
+ "auroc": 1.0,
188
+ "pixel_auroc": 1.0,
189
+ "pixel_acc": 0.9088134765625,
190
+ "precision": 0.2,
191
+ "recall": 0.2,
192
+ "specificity": 0.9088134765625,
193
+ "iou_all": 0.2,
194
+ "dice_all": 0.2,
195
+ "f1_all": 0.2,
196
+ "positive_samples": 0,
197
+ "zero_positive_samples": 20,
198
+ "iou_positive": 0.2,
199
+ "dice_positive": 0.2,
200
+ "f1_positive": 0.2,
201
+ "precision_positive": 0.2,
202
+ "recall_positive": 0.2
203
+ }
204
+ },
205
+ "per_label_metrics": {
206
+ "0": {
207
+ "iou": 0.2,
208
+ "iou_all": 0.2,
209
+ "iou_positive": 0.2,
210
+ "dice": 0.2,
211
+ "dice_all": 0.2,
212
+ "dice_positive": 0.2,
213
+ "f1": 0.2,
214
+ "f1_all": 0.2,
215
+ "f1_positive": 0.2,
216
+ "pixel_f1": 0.2,
217
+ "auroc": 1.0,
218
+ "pixel_auroc": 1.0,
219
+ "pixel_acc": 0.9088134765625,
220
+ "precision": 0.2,
221
+ "precision_all": 0.2,
222
+ "precision_positive": 0.2,
223
+ "recall": 0.2,
224
+ "recall_all": 0.2,
225
+ "recall_positive": 0.2,
226
+ "specificity": 0.9088134765625,
227
+ "samples": 20,
228
+ "positive_samples": 0
229
+ },
230
+ "1": {
231
+ "iou": 0.0,
232
+ "iou_all": 0.0,
233
+ "iou_positive": 0.0,
234
+ "dice": 0.0,
235
+ "dice_all": 0.0,
236
+ "dice_positive": 0.0,
237
+ "f1": 0.0,
238
+ "f1_all": 0.0,
239
+ "f1_positive": 0.0,
240
+ "pixel_f1": 0.0,
241
+ "auroc": 0.0,
242
+ "pixel_auroc": 0.0,
243
+ "pixel_acc": 0.0,
244
+ "precision": 0.0,
245
+ "precision_all": 0.0,
246
+ "precision_positive": 0.0,
247
+ "recall": 0.0,
248
+ "recall_all": 0.0,
249
+ "recall_positive": 0.0,
250
+ "specificity": 0.0,
251
+ "samples": 0,
252
+ "positive_samples": 0
253
+ },
254
+ "2": {
255
+ "iou": 0.0,
256
+ "iou_all": 0.0,
257
+ "iou_positive": 0.0,
258
+ "dice": 0.0,
259
+ "dice_all": 0.0,
260
+ "dice_positive": 0.0,
261
+ "f1": 0.0,
262
+ "f1_all": 0.0,
263
+ "f1_positive": 0.0,
264
+ "pixel_f1": 0.0,
265
+ "auroc": 0.0,
266
+ "pixel_auroc": 0.0,
267
+ "pixel_acc": 0.0,
268
+ "precision": 0.0,
269
+ "precision_all": 0.0,
270
+ "precision_positive": 0.0,
271
+ "recall": 0.0,
272
+ "recall_all": 0.0,
273
+ "recall_positive": 0.0,
274
+ "specificity": 0.0,
275
+ "samples": 0,
276
+ "positive_samples": 0
277
+ }
278
+ },
279
+ "raw_per_label_metrics": {
280
+ "0": {
281
+ "iou": 0.15,
282
+ "iou_all": 0.15,
283
+ "iou_positive": 0.15,
284
+ "dice": 0.15,
285
+ "dice_all": 0.15,
286
+ "dice_positive": 0.15,
287
+ "f1": 0.15,
288
+ "f1_all": 0.15,
289
+ "f1_positive": 0.15,
290
+ "pixel_f1": 0.15,
291
+ "auroc": 1.0,
292
+ "pixel_auroc": 1.0,
293
+ "pixel_acc": 0.908416748046875,
294
+ "precision": 0.15,
295
+ "precision_all": 0.15,
296
+ "precision_positive": 0.15,
297
+ "recall": 0.15,
298
+ "recall_all": 0.15,
299
+ "recall_positive": 0.15,
300
+ "specificity": 0.908416748046875,
301
+ "samples": 20,
302
+ "positive_samples": 0
303
+ },
304
+ "1": {
305
+ "iou": 0.0,
306
+ "iou_all": 0.0,
307
+ "iou_positive": 0.0,
308
+ "dice": 0.0,
309
+ "dice_all": 0.0,
310
+ "dice_positive": 0.0,
311
+ "f1": 0.0,
312
+ "f1_all": 0.0,
313
+ "f1_positive": 0.0,
314
+ "pixel_f1": 0.0,
315
+ "auroc": 0.0,
316
+ "pixel_auroc": 0.0,
317
+ "pixel_acc": 0.0,
318
+ "precision": 0.0,
319
+ "precision_all": 0.0,
320
+ "precision_positive": 0.0,
321
+ "recall": 0.0,
322
+ "recall_all": 0.0,
323
+ "recall_positive": 0.0,
324
+ "specificity": 0.0,
325
+ "samples": 0,
326
+ "positive_samples": 0
327
+ },
328
+ "2": {
329
+ "iou": 0.0,
330
+ "iou_all": 0.0,
331
+ "iou_positive": 0.0,
332
+ "dice": 0.0,
333
+ "dice_all": 0.0,
334
+ "dice_positive": 0.0,
335
+ "f1": 0.0,
336
+ "f1_all": 0.0,
337
+ "f1_positive": 0.0,
338
+ "pixel_f1": 0.0,
339
+ "auroc": 0.0,
340
+ "pixel_auroc": 0.0,
341
+ "pixel_acc": 0.0,
342
+ "precision": 0.0,
343
+ "precision_all": 0.0,
344
+ "precision_positive": 0.0,
345
+ "recall": 0.0,
346
+ "recall_all": 0.0,
347
+ "recall_positive": 0.0,
348
+ "specificity": 0.0,
349
+ "samples": 0,
350
+ "positive_samples": 0
351
+ }
352
+ },
353
+ "per_dataset_metrics": {},
354
+ "raw_per_dataset_metrics": {},
355
+ "confusion_matrix": null,
356
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
357
+ "checkpoint_config": {
358
+ "project": {
359
+ "name": "sid_pim_swin",
360
+ "seed": 42,
361
+ "device": "auto",
362
+ "output_dir": "outputs/RUN/pim_swin",
363
+ "hub_repo": null,
364
+ "resume_repo": null
365
+ },
366
+ "data": {
367
+ "dataset_name": "KhangTruong/IMD2020",
368
+ "streaming": false,
369
+ "image_size": [
370
+ 256,
371
+ 256
372
+ ],
373
+ "batch_size": 8,
374
+ "num_workers": -1,
375
+ "pin_memory": true,
376
+ "shuffle_buffer_size": 1000,
377
+ "train_split": "train",
378
+ "val_split": "validation",
379
+ "test_split": "test",
380
+ "eval_split": "test",
381
+ "train_samples_per_epoch": -1,
382
+ "val_samples_per_epoch": -1,
383
+ "val_samples": -1,
384
+ "test_samples": -1,
385
+ "evaluate_on_test": true,
386
+ "augmentations": {
387
+ "horizontal_flip": 0.5,
388
+ "vertical_flip": 0.2,
389
+ "random_rotate90": 0.5,
390
+ "color_jitter": 0.0
391
+ }
392
+ },
393
+ "model": {
394
+ "name": "pim",
395
+ "in_channels": 3,
396
+ "out_channels": 1,
397
+ "features": [
398
+ 64,
399
+ 128,
400
+ 256,
401
+ 512
402
+ ],
403
+ "bilinear": true,
404
+ "dropout": 0.1,
405
+ "aux_classifier": false,
406
+ "num_classes": 2,
407
+ "gradient_checkpointing": false,
408
+ "pretrained": true,
409
+ "hf_repo": "KhangTruong/PIM"
410
+ },
411
+ "loss": {
412
+ "mask_loss_type": "combined",
413
+ "bce_weight": 0.5,
414
+ "dice_weight": 0.5,
415
+ "focal_weight": 0.5,
416
+ "focal_gamma": 2.0,
417
+ "focal_alpha": 0.25,
418
+ "aux_loss_type": "cross_entropy",
419
+ "aux_weight": 0.2
420
+ },
421
+ "training": {
422
+ "epochs": 10,
423
+ "learning_rate": 0.0001,
424
+ "weight_decay": 0.0001,
425
+ "optimizer": "adamw",
426
+ "use_8bit_optimizer": false,
427
+ "check_8bit_compatibility": true,
428
+ "fallback_to_16bit": true,
429
+ "fallback_on_unsupported_8bit": true,
430
+ "scheduler": "cosine",
431
+ "resume_lr_mode": "auto",
432
+ "resume_lr": null,
433
+ "warmup_epochs": 1,
434
+ "min_lr": 1e-06,
435
+ "grad_clip_norm": 1.0,
436
+ "amp": true,
437
+ "gradient_accumulation_steps": 1,
438
+ "data_parallel": true,
439
+ "use_hard_mining": false,
440
+ "auto_batch_size": true,
441
+ "empty_cache_per_epoch": true,
442
+ "save_best": true,
443
+ "save_latest": false,
444
+ "checkpoint_period": 3600,
445
+ "eval_interval": 1,
446
+ "early_stopping_patience": 5,
447
+ "early_stopping_metric": "val_iou",
448
+ "early_stopping_mode": "max",
449
+ "push_to_hub": false,
450
+ "hub_repo": null,
451
+ "hub_version": "v1"
452
+ },
453
+ "hard_mining": {
454
+ "enabled": false,
455
+ "metric": "median",
456
+ "reset_epochs": 5
457
+ },
458
+ "bootstrapping": {
459
+ "enabled": false,
460
+ "run_bootstrap": false,
461
+ "epochs": 5,
462
+ "num_samples": 512,
463
+ "batch_size": null,
464
+ "learning_rate": null,
465
+ "freeze_strategy": "channel_stream",
466
+ "stream_ratio": 0.5,
467
+ "release_mode": "restore",
468
+ "freeze_modules": [],
469
+ "initialization": "kaiming_normal",
470
+ "target_score": 0.5,
471
+ "min_loss_drop": 0.15,
472
+ "early_stopping": false,
473
+ "patience": 3
474
+ },
475
+ "logging": {
476
+ "log_interval": 20,
477
+ "log_memory": true,
478
+ "measure_network": true
479
+ },
480
+ "post_processing": {
481
+ "enabled": true,
482
+ "min_area": 64,
483
+ "fill_holes": true,
484
+ "max_hole_size": 256,
485
+ "morphology": "open_close",
486
+ "morph_kernel_size": 3
487
+ }
488
+ },
489
+ "eval_config": {
490
+ "project": {
491
+ "name": "eval_casia_v2.0",
492
+ "seed": 42,
493
+ "device": "auto",
494
+ "output_dir": "outputs/cross_eval_results",
495
+ "hub_repo": null,
496
+ "resume_repo": null
497
+ },
498
+ "data": {
499
+ "dataset_name": "KhangTruong/Casia-v2.0",
500
+ "streaming": false,
501
+ "image_size": [
502
+ 256,
503
+ 256
504
+ ],
505
+ "batch_size": 4,
506
+ "num_workers": 0,
507
+ "pin_memory": true,
508
+ "shuffle_buffer_size": 1000,
509
+ "train_split": "train",
510
+ "val_split": "validation",
511
+ "test_split": "cross_test",
512
+ "eval_split": "cross_test",
513
+ "train_samples_per_epoch": -1,
514
+ "val_samples_per_epoch": -1,
515
+ "val_samples": -1,
516
+ "test_samples": -1,
517
+ "evaluate_on_test": true,
518
+ "augmentations": {
519
+ "horizontal_flip": 0.5,
520
+ "vertical_flip": 0.2,
521
+ "random_rotate90": 0.5,
522
+ "color_jitter": 0.0
523
+ }
524
+ },
525
+ "model": {
526
+ "name": "unet",
527
+ "in_channels": 3,
528
+ "out_channels": 1,
529
+ "features": [
530
+ 64,
531
+ 128,
532
+ 256,
533
+ 512
534
+ ],
535
+ "bilinear": true,
536
+ "dropout": 0.0,
537
+ "aux_classifier": false,
538
+ "num_classes": 3,
539
+ "gradient_checkpointing": false
540
+ },
541
+ "loss": {
542
+ "mask_loss_type": "combined",
543
+ "bce_weight": 0.5,
544
+ "dice_weight": 0.5,
545
+ "focal_weight": 0.5,
546
+ "focal_gamma": 2.0,
547
+ "focal_alpha": 0.25,
548
+ "aux_loss_type": "cross_entropy",
549
+ "aux_weight": 0.2
550
+ },
551
+ "training": {
552
+ "epochs": 10,
553
+ "learning_rate": 0.001,
554
+ "weight_decay": 0.0001,
555
+ "optimizer": "adamw",
556
+ "use_8bit_optimizer": false,
557
+ "check_8bit_compatibility": true,
558
+ "fallback_to_16bit": true,
559
+ "fallback_on_unsupported_8bit": true,
560
+ "scheduler": "cosine",
561
+ "resume_lr_mode": "auto",
562
+ "resume_lr": null,
563
+ "warmup_epochs": 1,
564
+ "min_lr": 1e-06,
565
+ "grad_clip_norm": 1.0,
566
+ "amp": true,
567
+ "gradient_accumulation_steps": 1,
568
+ "data_parallel": true,
569
+ "use_hard_mining": false,
570
+ "auto_batch_size": true,
571
+ "empty_cache_per_epoch": true,
572
+ "save_best": true,
573
+ "save_latest": false,
574
+ "checkpoint_period": 3600,
575
+ "eval_interval": 1,
576
+ "early_stopping_patience": 5,
577
+ "early_stopping_metric": "val_iou",
578
+ "early_stopping_mode": "max",
579
+ "push_to_hub": false,
580
+ "hub_repo": null,
581
+ "hub_version": "v1"
582
+ },
583
+ "hard_mining": {
584
+ "enabled": false,
585
+ "metric": "median",
586
+ "reset_epochs": 5
587
+ },
588
+ "bootstrapping": {
589
+ "enabled": false,
590
+ "run_bootstrap": false,
591
+ "epochs": 5,
592
+ "num_samples": 512,
593
+ "batch_size": null,
594
+ "learning_rate": null,
595
+ "freeze_strategy": "channel_stream",
596
+ "stream_ratio": 0.5,
597
+ "release_mode": "restore",
598
+ "freeze_modules": [],
599
+ "initialization": "kaiming_normal",
600
+ "target_score": 0.5,
601
+ "min_loss_drop": 0.15,
602
+ "early_stopping": false,
603
+ "patience": 3
604
+ },
605
+ "logging": {
606
+ "log_interval": 20,
607
+ "log_memory": true,
608
+ "measure_network": true
609
+ },
610
+ "post_processing": {
611
+ "enabled": true,
612
+ "min_area": 64,
613
+ "fill_holes": true,
614
+ "max_hole_size": 256,
615
+ "morphology": "open_close",
616
+ "morph_kernel_size": 3
617
+ }
618
+ },
619
+ "checkpoint_hyperparameters": {
620
+ "Project Name": "sid_pim_swin",
621
+ "Dataset": "KhangTruong/IMD2020",
622
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
623
+ "Streaming": false,
624
+ "Image Size": "[256, 256]",
625
+ "Batch Size": 8,
626
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
627
+ "Model": "pim (features=[64, 128, 256, 512], bilinear=True)",
628
+ "Aux Classifier": "False (classes=2)",
629
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
630
+ "Aux Loss": "cross_entropy (wt=0.2)",
631
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
632
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
633
+ "Scheduler": "cosine",
634
+ "Epochs": 10,
635
+ "AMP": true
636
+ },
637
+ "eval_hyperparameters": {
638
+ "Project Name": "eval_casia_v2.0",
639
+ "Dataset": "KhangTruong/Casia-v2.0",
640
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
641
+ "Streaming": false,
642
+ "Image Size": "[256, 256]",
643
+ "Batch Size": 4,
644
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
645
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
646
+ "Aux Classifier": "False (classes=3)",
647
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
648
+ "Aux Loss": "cross_entropy (wt=0.2)",
649
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
650
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
651
+ "Scheduler": "cosine",
652
+ "Epochs": 10,
653
+ "AMP": true
654
+ },
655
+ "devices": [
656
+ "cuda"
657
+ ],
658
+ "gpu_utilization": {}
659
+ }
660
+ ],
661
+ "illustration_paths": [
662
+ "/content/ai-detection-unet-gpu/outputs/RUN/pim_swin/cross_eval_reports/illustrations/sample_predictions_casia_v2.0.png"
663
+ ]
664
+ }
cross_evaluation/cross_evaluation_report.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Cross-Evaluation Report: sid_pim_swin
2
+
3
+ - **Checkpoint Path**: `outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt`
4
+ - **Evaluated Configurations**: 1
5
+
6
+ ## Cross-Dataset Performance Summary
7
+
8
+ | Config | Dataset | Split | Samples | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Class Acc |
9
+ |-----------------|------------------------|------------|-----------|--------|--------|-------------|---------|-------------|-------------|
10
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | cross_test | 20 | 0.6689 | 0.2000 | 0.2000 | 1.0000 | 0.9088 | N/A |
11
+
12
+
13
+ ---
14
+
15
+ ## Per-Configuration Breakdown
16
+
17
+ ### Configuration: eval_casia_v2.0 (Dataset: KhangTruong/Casia-v2.0)
18
+
19
+ - **Config File**: `configs/cross-eval/casia_v2.0.yaml`
20
+ ### Evaluation Metrics
21
+
22
+ | Metric | Value |
23
+ |-------------------------|------------|
24
+ | Eval Split | cross_test |
25
+ | Eval Total Loss | 0.6689 |
26
+ | Iou | 0.2000 |
27
+ | Dice | 0.2000 |
28
+ | F1 | 0.2000 |
29
+ | Pixel F1 | 0.2000 |
30
+ | Auroc | 1.0000 |
31
+ | Pixel Auroc | 1.0000 |
32
+ | Pixel Acc | 0.9088 |
33
+ | Precision | 0.2000 |
34
+ | Recall | 0.2000 |
35
+ | Specificity | 0.9088 |
36
+ | Iou All | 0.2000 |
37
+ | Dice All | 0.2000 |
38
+ | F1 All | 0.2000 |
39
+ | Positive Samples | 0 |
40
+ | Zero Positive Samples | 20 |
41
+ | Iou Positive | 0.2000 |
42
+ | Dice Positive | 0.2000 |
43
+ | F1 Positive | 0.2000 |
44
+ | Precision Positive | 0.2000 |
45
+ | Recall Positive | 0.2000 |
46
+ | Total Evaluated Samples | 20 |
47
+
48
+ #### Per-Subset Breakdown
49
+
50
+ | Subset | IoU | Dice / F1 | Pixel Acc | Samples |
51
+ |---------------------|--------|-------------|-------------|-----------|
52
+ | Label 0 (Real) | 0.2000 | 0.2000 | 0.9088 | 20 |
53
+ | Label 1 (Synthetic) | 0.0000 | 0.0000 | 0.0000 | 0 |
54
+ | Label 2 (Tampered) | 0.0000 | 0.0000 | 0.0000 | 0 |
55
+
56
+
57
+ ## 🖼️ Visual Illustrations & Sample Predictions
58
+
59
+ **Sample Predictions Casia V2.0**
60
+
61
+ ![Sample Predictions Casia V2.0](illustrations/sample_predictions_casia_v2.0.png)
cross_evaluation/eval_casia_v2.0_report.json ADDED
@@ -0,0 +1,345 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "overall_metrics": {
3
+ "eval_split": "cross_test",
4
+ "eval_total_loss": 0.6688968300819397,
5
+ "iou": 0.2,
6
+ "dice": 0.2,
7
+ "f1": 0.2,
8
+ "pixel_f1": 0.2,
9
+ "auroc": 1.0,
10
+ "pixel_auroc": 1.0,
11
+ "pixel_acc": 0.9088134765625,
12
+ "precision": 0.2,
13
+ "recall": 0.2,
14
+ "specificity": 0.9088134765625,
15
+ "iou_all": 0.2,
16
+ "dice_all": 0.2,
17
+ "f1_all": 0.2,
18
+ "positive_samples": 0,
19
+ "zero_positive_samples": 20,
20
+ "iou_positive": 0.2,
21
+ "dice_positive": 0.2,
22
+ "f1_positive": 0.2,
23
+ "precision_positive": 0.2,
24
+ "recall_positive": 0.2,
25
+ "total_evaluated_samples": 20,
26
+ "ablation_summary": {
27
+ "Baseline (Raw UNet)": {
28
+ "iou": 0.15,
29
+ "dice": 0.15,
30
+ "f1": 0.15,
31
+ "pixel_f1": 0.15,
32
+ "auroc": 1.0,
33
+ "pixel_auroc": 1.0,
34
+ "pixel_acc": 0.908416748046875,
35
+ "precision": 0.15,
36
+ "recall": 0.15,
37
+ "specificity": 0.908416748046875,
38
+ "iou_all": 0.15,
39
+ "dice_all": 0.15,
40
+ "f1_all": 0.15,
41
+ "positive_samples": 0,
42
+ "zero_positive_samples": 20,
43
+ "iou_positive": 0.15,
44
+ "dice_positive": 0.15,
45
+ "f1_positive": 0.15,
46
+ "precision_positive": 0.15,
47
+ "recall_positive": 0.15
48
+ },
49
+ "+ Post-Processing": {
50
+ "iou": 0.2,
51
+ "dice": 0.2,
52
+ "f1": 0.2,
53
+ "pixel_f1": 0.2,
54
+ "auroc": 1.0,
55
+ "pixel_auroc": 1.0,
56
+ "pixel_acc": 0.9088134765625,
57
+ "precision": 0.2,
58
+ "recall": 0.2,
59
+ "specificity": 0.9088134765625,
60
+ "iou_all": 0.2,
61
+ "dice_all": 0.2,
62
+ "f1_all": 0.2,
63
+ "positive_samples": 0,
64
+ "zero_positive_samples": 20,
65
+ "iou_positive": 0.2,
66
+ "dice_positive": 0.2,
67
+ "f1_positive": 0.2,
68
+ "precision_positive": 0.2,
69
+ "recall_positive": 0.2
70
+ }
71
+ },
72
+ "per_dataset_metrics": {}
73
+ },
74
+ "per_label_metrics": {
75
+ "Label 0 (Real / Black Mask)": {
76
+ "iou": 0.2,
77
+ "iou_all": 0.2,
78
+ "iou_positive": 0.2,
79
+ "dice": 0.2,
80
+ "dice_all": 0.2,
81
+ "dice_positive": 0.2,
82
+ "f1": 0.2,
83
+ "f1_all": 0.2,
84
+ "f1_positive": 0.2,
85
+ "pixel_f1": 0.2,
86
+ "auroc": 1.0,
87
+ "pixel_auroc": 1.0,
88
+ "pixel_acc": 0.9088134765625,
89
+ "precision": 0.2,
90
+ "precision_all": 0.2,
91
+ "precision_positive": 0.2,
92
+ "recall": 0.2,
93
+ "recall_all": 0.2,
94
+ "recall_positive": 0.2,
95
+ "specificity": 0.9088134765625,
96
+ "samples": 20,
97
+ "positive_samples": 0
98
+ },
99
+ "Label 1 (Synthetic / White Mask)": {
100
+ "iou": 0.0,
101
+ "iou_all": 0.0,
102
+ "iou_positive": 0.0,
103
+ "dice": 0.0,
104
+ "dice_all": 0.0,
105
+ "dice_positive": 0.0,
106
+ "f1": 0.0,
107
+ "f1_all": 0.0,
108
+ "f1_positive": 0.0,
109
+ "pixel_f1": 0.0,
110
+ "auroc": 0.0,
111
+ "pixel_auroc": 0.0,
112
+ "pixel_acc": 0.0,
113
+ "precision": 0.0,
114
+ "precision_all": 0.0,
115
+ "precision_positive": 0.0,
116
+ "recall": 0.0,
117
+ "recall_all": 0.0,
118
+ "recall_positive": 0.0,
119
+ "specificity": 0.0,
120
+ "samples": 0,
121
+ "positive_samples": 0
122
+ },
123
+ "Label 2 (Tampered / Partial Mask)": {
124
+ "iou": 0.0,
125
+ "iou_all": 0.0,
126
+ "iou_positive": 0.0,
127
+ "dice": 0.0,
128
+ "dice_all": 0.0,
129
+ "dice_positive": 0.0,
130
+ "f1": 0.0,
131
+ "f1_all": 0.0,
132
+ "f1_positive": 0.0,
133
+ "pixel_f1": 0.0,
134
+ "auroc": 0.0,
135
+ "pixel_auroc": 0.0,
136
+ "pixel_acc": 0.0,
137
+ "precision": 0.0,
138
+ "precision_all": 0.0,
139
+ "precision_positive": 0.0,
140
+ "recall": 0.0,
141
+ "recall_all": 0.0,
142
+ "recall_positive": 0.0,
143
+ "specificity": 0.0,
144
+ "samples": 0,
145
+ "positive_samples": 0
146
+ }
147
+ },
148
+ "ablation_comparison": {
149
+ "Baseline (Raw UNet)": {
150
+ "iou": 0.15,
151
+ "dice": 0.15,
152
+ "f1": 0.15,
153
+ "pixel_f1": 0.15,
154
+ "auroc": 1.0,
155
+ "pixel_auroc": 1.0,
156
+ "pixel_acc": 0.908416748046875,
157
+ "precision": 0.15,
158
+ "recall": 0.15,
159
+ "specificity": 0.908416748046875,
160
+ "iou_all": 0.15,
161
+ "dice_all": 0.15,
162
+ "f1_all": 0.15,
163
+ "positive_samples": 0,
164
+ "zero_positive_samples": 20,
165
+ "iou_positive": 0.15,
166
+ "dice_positive": 0.15,
167
+ "f1_positive": 0.15,
168
+ "precision_positive": 0.15,
169
+ "recall_positive": 0.15
170
+ },
171
+ "+ Post-Processing": {
172
+ "iou": 0.2,
173
+ "dice": 0.2,
174
+ "f1": 0.2,
175
+ "pixel_f1": 0.2,
176
+ "auroc": 1.0,
177
+ "pixel_auroc": 1.0,
178
+ "pixel_acc": 0.9088134765625,
179
+ "precision": 0.2,
180
+ "recall": 0.2,
181
+ "specificity": 0.9088134765625,
182
+ "iou_all": 0.2,
183
+ "dice_all": 0.2,
184
+ "f1_all": 0.2,
185
+ "positive_samples": 0,
186
+ "zero_positive_samples": 20,
187
+ "iou_positive": 0.2,
188
+ "dice_positive": 0.2,
189
+ "f1_positive": 0.2,
190
+ "precision_positive": 0.2,
191
+ "recall_positive": 0.2
192
+ }
193
+ },
194
+ "illustration_paths": [
195
+ "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png"
196
+ ],
197
+ "configuration": {
198
+ "project": {
199
+ "name": "eval_casia_v2.0",
200
+ "seed": 42,
201
+ "device": "auto",
202
+ "output_dir": "outputs/cross_eval_results",
203
+ "hub_repo": null,
204
+ "resume_repo": null
205
+ },
206
+ "data": {
207
+ "dataset_name": "KhangTruong/Casia-v2.0",
208
+ "streaming": false,
209
+ "image_size": [
210
+ 256,
211
+ 256
212
+ ],
213
+ "batch_size": 4,
214
+ "num_workers": 0,
215
+ "pin_memory": true,
216
+ "shuffle_buffer_size": 1000,
217
+ "train_split": "train",
218
+ "val_split": "validation",
219
+ "test_split": "cross_test",
220
+ "eval_split": "cross_test",
221
+ "train_samples_per_epoch": -1,
222
+ "val_samples_per_epoch": -1,
223
+ "val_samples": -1,
224
+ "test_samples": -1,
225
+ "evaluate_on_test": true,
226
+ "augmentations": {
227
+ "horizontal_flip": 0.5,
228
+ "vertical_flip": 0.2,
229
+ "random_rotate90": 0.5,
230
+ "color_jitter": 0.0
231
+ }
232
+ },
233
+ "model": {
234
+ "name": "unet",
235
+ "in_channels": 3,
236
+ "out_channels": 1,
237
+ "features": [
238
+ 64,
239
+ 128,
240
+ 256,
241
+ 512
242
+ ],
243
+ "bilinear": true,
244
+ "dropout": 0.0,
245
+ "aux_classifier": false,
246
+ "num_classes": 3,
247
+ "gradient_checkpointing": false
248
+ },
249
+ "loss": {
250
+ "mask_loss_type": "combined",
251
+ "bce_weight": 0.5,
252
+ "dice_weight": 0.5,
253
+ "focal_weight": 0.5,
254
+ "focal_gamma": 2.0,
255
+ "focal_alpha": 0.25,
256
+ "aux_loss_type": "cross_entropy",
257
+ "aux_weight": 0.2
258
+ },
259
+ "training": {
260
+ "epochs": 10,
261
+ "learning_rate": 0.001,
262
+ "weight_decay": 0.0001,
263
+ "optimizer": "adamw",
264
+ "use_8bit_optimizer": false,
265
+ "check_8bit_compatibility": true,
266
+ "fallback_to_16bit": true,
267
+ "fallback_on_unsupported_8bit": true,
268
+ "scheduler": "cosine",
269
+ "resume_lr_mode": "auto",
270
+ "resume_lr": null,
271
+ "warmup_epochs": 1,
272
+ "min_lr": 1e-06,
273
+ "grad_clip_norm": 1.0,
274
+ "amp": true,
275
+ "gradient_accumulation_steps": 1,
276
+ "data_parallel": true,
277
+ "use_hard_mining": false,
278
+ "auto_batch_size": true,
279
+ "empty_cache_per_epoch": true,
280
+ "save_best": true,
281
+ "save_latest": false,
282
+ "checkpoint_period": 3600,
283
+ "eval_interval": 1,
284
+ "early_stopping_patience": 5,
285
+ "early_stopping_metric": "val_iou",
286
+ "early_stopping_mode": "max",
287
+ "push_to_hub": false,
288
+ "hub_repo": null,
289
+ "hub_version": "v1"
290
+ },
291
+ "hard_mining": {
292
+ "enabled": false,
293
+ "metric": "median",
294
+ "reset_epochs": 5
295
+ },
296
+ "bootstrapping": {
297
+ "enabled": false,
298
+ "run_bootstrap": false,
299
+ "epochs": 5,
300
+ "num_samples": 512,
301
+ "batch_size": null,
302
+ "learning_rate": null,
303
+ "freeze_strategy": "channel_stream",
304
+ "stream_ratio": 0.5,
305
+ "release_mode": "restore",
306
+ "freeze_modules": [],
307
+ "initialization": "kaiming_normal",
308
+ "target_score": 0.5,
309
+ "min_loss_drop": 0.15,
310
+ "early_stopping": false,
311
+ "patience": 3
312
+ },
313
+ "logging": {
314
+ "log_interval": 20,
315
+ "log_memory": true,
316
+ "measure_network": true
317
+ },
318
+ "post_processing": {
319
+ "enabled": true,
320
+ "min_area": 64,
321
+ "fill_holes": true,
322
+ "max_hole_size": 256,
323
+ "morphology": "open_close",
324
+ "morph_kernel_size": 3
325
+ }
326
+ },
327
+ "hyperparameters": {
328
+ "Project Name": "eval_casia_v2.0",
329
+ "Dataset": "KhangTruong/Casia-v2.0",
330
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
331
+ "Streaming": false,
332
+ "Image Size": "[256, 256]",
333
+ "Batch Size": 4,
334
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
335
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
336
+ "Aux Classifier": "False (classes=3)",
337
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
338
+ "Aux Loss": "cross_entropy (wt=0.2)",
339
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
340
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
341
+ "Scheduler": "cosine",
342
+ "Epochs": 10,
343
+ "AMP": true
344
+ }
345
+ }
cross_evaluation/eval_casia_v2.0_report.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SID-UNet Evaluation Report
2
+
3
+ ### Run Configuration & Hyperparameters
4
+
5
+ | Hyperparameter / Setting | Configured Value |
6
+ |----------------------------|----------------------------------------------------------|
7
+ | Project Name | eval_casia_v2.0 |
8
+ | Dataset | KhangTruong/Casia-v2.0 |
9
+ | Splits | Train: 'train' | Val: 'validation' | Test: 'cross_test' |
10
+ | Streaming | False |
11
+ | Image Size | [256, 256] |
12
+ | Batch Size | 4 |
13
+ | Epoch Limit / Steps | Full Dataset (until depletion) |
14
+ | Model | unet (features=[64, 128, 256, 512], bilinear=True) |
15
+ | Aux Classifier | False (classes=3) |
16
+ | Mask Loss | Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5) |
17
+ | Aux Loss | cross_entropy (wt=0.2) |
18
+ | Post-Processing | Enabled (min_area=64, fill_holes=True, morph=open_close) |
19
+ | Optimizer | adamw (lr=0.001, wd=0.0001) |
20
+ | Scheduler | cosine |
21
+ | Epochs | 10 |
22
+ | AMP | True |
23
+
24
+ ### 🔬 Pipeline Ablation & Mask Refinement Comparison
25
+
26
+ | Pipeline Stage / Variant | Mean IoU | Dice / F1 | Pixel AUROC | Pixel Acc | Precision | Recall |
27
+ |----------------------------|------------|-------------|---------------|-------------|-------------|----------|
28
+ | Baseline (Raw UNet) | 0.1500 | 0.1500 | 1.0000 | 0.9084 | 0.1500 | 0.1500 |
29
+ | + Post-Processing | 0.2000 | 0.2000 | 1.0000 | 0.9088 | 0.2000 | 0.2000 |
30
+
31
+
32
+ ### Overall Segmentation & Classification Metrics
33
+
34
+ | Metric | Value |
35
+ |-------------------------|------------|
36
+ | Eval Split | cross_test |
37
+ | Eval Total Loss | 0.6689 |
38
+ | Iou | 0.2000 |
39
+ | Dice | 0.2000 |
40
+ | F1 | 0.2000 |
41
+ | Pixel F1 | 0.2000 |
42
+ | Auroc | 1.0000 |
43
+ | Pixel Auroc | 1.0000 |
44
+ | Pixel Acc | 0.9088 |
45
+ | Precision | 0.2000 |
46
+ | Recall | 0.2000 |
47
+ | Specificity | 0.9088 |
48
+ | Iou All | 0.2000 |
49
+ | Dice All | 0.2000 |
50
+ | F1 All | 0.2000 |
51
+ | Positive Samples | 0 |
52
+ | Zero Positive Samples | 20 |
53
+ | Iou Positive | 0.2000 |
54
+ | Dice Positive | 0.2000 |
55
+ | F1 Positive | 0.2000 |
56
+ | Precision Positive | 0.2000 |
57
+ | Recall Positive | 0.2000 |
58
+ | Total Evaluated Samples | 20 |
59
+
60
+ ### 🖼️ Evaluation Illustrations & Sample Predictions
61
+
62
+ **Sample Predictions Casia V2.0**
63
+
64
+ ![Sample Predictions Casia V2.0](../../../cross_evaluation/illustrations/sample_predictions_casia_v2.0.png)
65
+
66
+ ### Per-Subset Metrics Breakdown
67
+
68
+ | Subset | Mean IoU | Dice / F1 | Pixel Accuracy | Sample Count |
69
+ |-----------------------------------|------------|-------------|------------------|----------------|
70
+ | Label 0 (Real / Black Mask) | 0.2 | 0.2 | 0.9088 | 20 |
71
+ | Label 1 (Synthetic / White Mask) | 0 | 0 | 0 | 0 |
72
+ | Label 2 (Tampered / Partial Mask) | 0 | 0 | 0 | 0 |
73
+
cross_evaluation/illustrations/cross_eval_auroc_heatmap.png ADDED
cross_evaluation/illustrations/cross_eval_f1_heatmap.png ADDED
cross_evaluation/illustrations/cross_eval_iou_heatmap.png ADDED
cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png ADDED
cross_evaluation/illustrations/sample_predictions_casia_v2.0.png ADDED

Git LFS Details

  • SHA256: a952f61081c70d15081e98b8c91f883fc671f0e07bde9e2eeae4c4bc3a5997b1
  • Pointer size: 132 Bytes
  • Size of remote file: 2.53 MB
cross_evaluation/master_cross_evaluation_report.json ADDED
@@ -0,0 +1,1352 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_checkpoints": 2,
3
+ "total_configs": 1,
4
+ "total_runs": 2,
5
+ "checkpoints": [
6
+ "sid_focal_hrnet",
7
+ "sid_pim_swin"
8
+ ],
9
+ "configs": [
10
+ "eval_casia_v2.0"
11
+ ],
12
+ "matrices": {
13
+ "iou": {
14
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
15
+ "sid_pim_swin:eval_casia_v2.0": 0.2
16
+ },
17
+ "dice_f1": {
18
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
19
+ "sid_pim_swin:eval_casia_v2.0": 0.2
20
+ },
21
+ "auroc": {
22
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
23
+ "sid_pim_swin:eval_casia_v2.0": 1.0
24
+ },
25
+ "pixel_acc": {
26
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
27
+ "sid_pim_swin:eval_casia_v2.0": 0.9088134765625
28
+ },
29
+ "loss": {
30
+ "sid_focal_hrnet:eval_casia_v2.0": 1.415394163131714,
31
+ "sid_pim_swin:eval_casia_v2.0": 0.6688968300819397
32
+ }
33
+ },
34
+ "cross_results": [
35
+ {
36
+ "checkpoint_path": "outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt",
37
+ "checkpoint_name": "sid_focal_hrnet",
38
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
39
+ "config_stem": "casia_v2.0",
40
+ "config_name": "eval_casia_v2.0",
41
+ "dataset_name": "KhangTruong/Casia-v2.0",
42
+ "eval_split": "cross_test",
43
+ "total_evaluated_samples": 20,
44
+ "metrics": {
45
+ "eval_split": "cross_test",
46
+ "eval_total_loss": 1.415394163131714,
47
+ "iou": 0.0,
48
+ "dice": 0.0,
49
+ "f1": 0.0,
50
+ "pixel_f1": 0.0,
51
+ "auroc": 0.006264495849609375,
52
+ "pixel_auroc": 0.006264495849609375,
53
+ "pixel_acc": 0.006264495849609375,
54
+ "precision": 0.0,
55
+ "recall": 0.0,
56
+ "specificity": 0.006264495849609375,
57
+ "iou_all": 0.0,
58
+ "dice_all": 0.0,
59
+ "f1_all": 0.0,
60
+ "positive_samples": 0,
61
+ "zero_positive_samples": 20,
62
+ "iou_positive": 0.0,
63
+ "dice_positive": 0.0,
64
+ "f1_positive": 0.0,
65
+ "precision_positive": 0.0,
66
+ "recall_positive": 0.0,
67
+ "total_evaluated_samples": 20,
68
+ "ablation_summary": {
69
+ "Baseline (Raw UNet)": {
70
+ "iou": 0.0,
71
+ "dice": 0.0,
72
+ "f1": 0.0,
73
+ "pixel_f1": 0.0,
74
+ "auroc": 0.2930259943008423,
75
+ "pixel_auroc": 0.2930259943008423,
76
+ "pixel_acc": 0.00025634765625,
77
+ "precision": 0.0,
78
+ "recall": 0.0,
79
+ "specificity": 0.00025634765625,
80
+ "iou_all": 0.0,
81
+ "dice_all": 0.0,
82
+ "f1_all": 0.0,
83
+ "positive_samples": 0,
84
+ "zero_positive_samples": 20,
85
+ "iou_positive": 0.0,
86
+ "dice_positive": 0.0,
87
+ "f1_positive": 0.0,
88
+ "precision_positive": 0.0,
89
+ "recall_positive": 0.0
90
+ },
91
+ "+ Post-Processing": {
92
+ "iou": 0.0,
93
+ "dice": 0.0,
94
+ "f1": 0.0,
95
+ "pixel_f1": 0.0,
96
+ "auroc": 0.006264495849609375,
97
+ "pixel_auroc": 0.006264495849609375,
98
+ "pixel_acc": 0.006264495849609375,
99
+ "precision": 0.0,
100
+ "recall": 0.0,
101
+ "specificity": 0.006264495849609375,
102
+ "iou_all": 0.0,
103
+ "dice_all": 0.0,
104
+ "f1_all": 0.0,
105
+ "positive_samples": 0,
106
+ "zero_positive_samples": 20,
107
+ "iou_positive": 0.0,
108
+ "dice_positive": 0.0,
109
+ "f1_positive": 0.0,
110
+ "precision_positive": 0.0,
111
+ "recall_positive": 0.0
112
+ }
113
+ },
114
+ "per_dataset_metrics": {}
115
+ },
116
+ "overall_metrics": {
117
+ "eval_split": "cross_test",
118
+ "eval_total_loss": 1.415394163131714,
119
+ "iou": 0.0,
120
+ "dice": 0.0,
121
+ "f1": 0.0,
122
+ "pixel_f1": 0.0,
123
+ "auroc": 0.006264495849609375,
124
+ "pixel_auroc": 0.006264495849609375,
125
+ "pixel_acc": 0.006264495849609375,
126
+ "precision": 0.0,
127
+ "recall": 0.0,
128
+ "specificity": 0.006264495849609375,
129
+ "iou_all": 0.0,
130
+ "dice_all": 0.0,
131
+ "f1_all": 0.0,
132
+ "positive_samples": 0,
133
+ "zero_positive_samples": 20,
134
+ "iou_positive": 0.0,
135
+ "dice_positive": 0.0,
136
+ "f1_positive": 0.0,
137
+ "precision_positive": 0.0,
138
+ "recall_positive": 0.0,
139
+ "total_evaluated_samples": 20,
140
+ "ablation_summary": {
141
+ "Baseline (Raw UNet)": {
142
+ "iou": 0.0,
143
+ "dice": 0.0,
144
+ "f1": 0.0,
145
+ "pixel_f1": 0.0,
146
+ "auroc": 0.2930259943008423,
147
+ "pixel_auroc": 0.2930259943008423,
148
+ "pixel_acc": 0.00025634765625,
149
+ "precision": 0.0,
150
+ "recall": 0.0,
151
+ "specificity": 0.00025634765625,
152
+ "iou_all": 0.0,
153
+ "dice_all": 0.0,
154
+ "f1_all": 0.0,
155
+ "positive_samples": 0,
156
+ "zero_positive_samples": 20,
157
+ "iou_positive": 0.0,
158
+ "dice_positive": 0.0,
159
+ "f1_positive": 0.0,
160
+ "precision_positive": 0.0,
161
+ "recall_positive": 0.0
162
+ },
163
+ "+ Post-Processing": {
164
+ "iou": 0.0,
165
+ "dice": 0.0,
166
+ "f1": 0.0,
167
+ "pixel_f1": 0.0,
168
+ "auroc": 0.006264495849609375,
169
+ "pixel_auroc": 0.006264495849609375,
170
+ "pixel_acc": 0.006264495849609375,
171
+ "precision": 0.0,
172
+ "recall": 0.0,
173
+ "specificity": 0.006264495849609375,
174
+ "iou_all": 0.0,
175
+ "dice_all": 0.0,
176
+ "f1_all": 0.0,
177
+ "positive_samples": 0,
178
+ "zero_positive_samples": 20,
179
+ "iou_positive": 0.0,
180
+ "dice_positive": 0.0,
181
+ "f1_positive": 0.0,
182
+ "precision_positive": 0.0,
183
+ "recall_positive": 0.0
184
+ }
185
+ },
186
+ "per_dataset_metrics": {}
187
+ },
188
+ "ablation_metrics": {
189
+ "Baseline (Raw UNet)": {
190
+ "iou": 0.0,
191
+ "dice": 0.0,
192
+ "f1": 0.0,
193
+ "pixel_f1": 0.0,
194
+ "auroc": 0.2930259943008423,
195
+ "pixel_auroc": 0.2930259943008423,
196
+ "pixel_acc": 0.00025634765625,
197
+ "precision": 0.0,
198
+ "recall": 0.0,
199
+ "specificity": 0.00025634765625,
200
+ "iou_all": 0.0,
201
+ "dice_all": 0.0,
202
+ "f1_all": 0.0,
203
+ "positive_samples": 0,
204
+ "zero_positive_samples": 20,
205
+ "iou_positive": 0.0,
206
+ "dice_positive": 0.0,
207
+ "f1_positive": 0.0,
208
+ "precision_positive": 0.0,
209
+ "recall_positive": 0.0
210
+ },
211
+ "+ Post-Processing": {
212
+ "iou": 0.0,
213
+ "dice": 0.0,
214
+ "f1": 0.0,
215
+ "pixel_f1": 0.0,
216
+ "auroc": 0.006264495849609375,
217
+ "pixel_auroc": 0.006264495849609375,
218
+ "pixel_acc": 0.006264495849609375,
219
+ "precision": 0.0,
220
+ "recall": 0.0,
221
+ "specificity": 0.006264495849609375,
222
+ "iou_all": 0.0,
223
+ "dice_all": 0.0,
224
+ "f1_all": 0.0,
225
+ "positive_samples": 0,
226
+ "zero_positive_samples": 20,
227
+ "iou_positive": 0.0,
228
+ "dice_positive": 0.0,
229
+ "f1_positive": 0.0,
230
+ "precision_positive": 0.0,
231
+ "recall_positive": 0.0
232
+ }
233
+ },
234
+ "per_label_metrics": {
235
+ "0": {
236
+ "iou": 0.0,
237
+ "iou_all": 0.0,
238
+ "iou_positive": 0.0,
239
+ "dice": 0.0,
240
+ "dice_all": 0.0,
241
+ "dice_positive": 0.0,
242
+ "f1": 0.0,
243
+ "f1_all": 0.0,
244
+ "f1_positive": 0.0,
245
+ "pixel_f1": 0.0,
246
+ "auroc": 0.006264495849609375,
247
+ "pixel_auroc": 0.006264495849609375,
248
+ "pixel_acc": 0.006264495849609375,
249
+ "precision": 0.0,
250
+ "precision_all": 0.0,
251
+ "precision_positive": 0.0,
252
+ "recall": 0.0,
253
+ "recall_all": 0.0,
254
+ "recall_positive": 0.0,
255
+ "specificity": 0.006264495849609375,
256
+ "samples": 20,
257
+ "positive_samples": 0
258
+ },
259
+ "1": {
260
+ "iou": 0.0,
261
+ "iou_all": 0.0,
262
+ "iou_positive": 0.0,
263
+ "dice": 0.0,
264
+ "dice_all": 0.0,
265
+ "dice_positive": 0.0,
266
+ "f1": 0.0,
267
+ "f1_all": 0.0,
268
+ "f1_positive": 0.0,
269
+ "pixel_f1": 0.0,
270
+ "auroc": 0.0,
271
+ "pixel_auroc": 0.0,
272
+ "pixel_acc": 0.0,
273
+ "precision": 0.0,
274
+ "precision_all": 0.0,
275
+ "precision_positive": 0.0,
276
+ "recall": 0.0,
277
+ "recall_all": 0.0,
278
+ "recall_positive": 0.0,
279
+ "specificity": 0.0,
280
+ "samples": 0,
281
+ "positive_samples": 0
282
+ },
283
+ "2": {
284
+ "iou": 0.0,
285
+ "iou_all": 0.0,
286
+ "iou_positive": 0.0,
287
+ "dice": 0.0,
288
+ "dice_all": 0.0,
289
+ "dice_positive": 0.0,
290
+ "f1": 0.0,
291
+ "f1_all": 0.0,
292
+ "f1_positive": 0.0,
293
+ "pixel_f1": 0.0,
294
+ "auroc": 0.0,
295
+ "pixel_auroc": 0.0,
296
+ "pixel_acc": 0.0,
297
+ "precision": 0.0,
298
+ "precision_all": 0.0,
299
+ "precision_positive": 0.0,
300
+ "recall": 0.0,
301
+ "recall_all": 0.0,
302
+ "recall_positive": 0.0,
303
+ "specificity": 0.0,
304
+ "samples": 0,
305
+ "positive_samples": 0
306
+ }
307
+ },
308
+ "raw_per_label_metrics": {
309
+ "0": {
310
+ "iou": 0.0,
311
+ "iou_all": 0.0,
312
+ "iou_positive": 0.0,
313
+ "dice": 0.0,
314
+ "dice_all": 0.0,
315
+ "dice_positive": 0.0,
316
+ "f1": 0.0,
317
+ "f1_all": 0.0,
318
+ "f1_positive": 0.0,
319
+ "pixel_f1": 0.0,
320
+ "auroc": 0.2930259943008423,
321
+ "pixel_auroc": 0.2930259943008423,
322
+ "pixel_acc": 0.00025634765625,
323
+ "precision": 0.0,
324
+ "precision_all": 0.0,
325
+ "precision_positive": 0.0,
326
+ "recall": 0.0,
327
+ "recall_all": 0.0,
328
+ "recall_positive": 0.0,
329
+ "specificity": 0.00025634765625,
330
+ "samples": 20,
331
+ "positive_samples": 0
332
+ },
333
+ "1": {
334
+ "iou": 0.0,
335
+ "iou_all": 0.0,
336
+ "iou_positive": 0.0,
337
+ "dice": 0.0,
338
+ "dice_all": 0.0,
339
+ "dice_positive": 0.0,
340
+ "f1": 0.0,
341
+ "f1_all": 0.0,
342
+ "f1_positive": 0.0,
343
+ "pixel_f1": 0.0,
344
+ "auroc": 0.0,
345
+ "pixel_auroc": 0.0,
346
+ "pixel_acc": 0.0,
347
+ "precision": 0.0,
348
+ "precision_all": 0.0,
349
+ "precision_positive": 0.0,
350
+ "recall": 0.0,
351
+ "recall_all": 0.0,
352
+ "recall_positive": 0.0,
353
+ "specificity": 0.0,
354
+ "samples": 0,
355
+ "positive_samples": 0
356
+ },
357
+ "2": {
358
+ "iou": 0.0,
359
+ "iou_all": 0.0,
360
+ "iou_positive": 0.0,
361
+ "dice": 0.0,
362
+ "dice_all": 0.0,
363
+ "dice_positive": 0.0,
364
+ "f1": 0.0,
365
+ "f1_all": 0.0,
366
+ "f1_positive": 0.0,
367
+ "pixel_f1": 0.0,
368
+ "auroc": 0.0,
369
+ "pixel_auroc": 0.0,
370
+ "pixel_acc": 0.0,
371
+ "precision": 0.0,
372
+ "precision_all": 0.0,
373
+ "precision_positive": 0.0,
374
+ "recall": 0.0,
375
+ "recall_all": 0.0,
376
+ "recall_positive": 0.0,
377
+ "specificity": 0.0,
378
+ "samples": 0,
379
+ "positive_samples": 0
380
+ }
381
+ },
382
+ "per_dataset_metrics": {},
383
+ "raw_per_dataset_metrics": {},
384
+ "confusion_matrix": null,
385
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
386
+ "checkpoint_config": {
387
+ "project": {
388
+ "name": "sid_focal_hrnet",
389
+ "seed": 42,
390
+ "device": "auto",
391
+ "output_dir": "outputs/RUN/focal_hrnet",
392
+ "hub_repo": null,
393
+ "resume_repo": null
394
+ },
395
+ "data": {
396
+ "dataset_name": "KhangTruong/IMD2020",
397
+ "streaming": false,
398
+ "image_size": [
399
+ 256,
400
+ 256
401
+ ],
402
+ "batch_size": 16,
403
+ "num_workers": -1,
404
+ "pin_memory": true,
405
+ "shuffle_buffer_size": 1000,
406
+ "train_split": "train",
407
+ "val_split": "validation",
408
+ "test_split": "test",
409
+ "eval_split": "test",
410
+ "train_samples_per_epoch": -1,
411
+ "val_samples_per_epoch": -1,
412
+ "val_samples": -1,
413
+ "test_samples": -1,
414
+ "evaluate_on_test": true,
415
+ "augmentations": {
416
+ "horizontal_flip": 0.5,
417
+ "vertical_flip": 0.2,
418
+ "random_rotate90": 0.5,
419
+ "color_jitter": 0.0
420
+ }
421
+ },
422
+ "model": {
423
+ "name": "focal",
424
+ "in_channels": 3,
425
+ "out_channels": 1,
426
+ "features": [
427
+ 64,
428
+ 128,
429
+ 256,
430
+ 512
431
+ ],
432
+ "bilinear": true,
433
+ "dropout": 0.1,
434
+ "aux_classifier": false,
435
+ "num_classes": 2,
436
+ "gradient_checkpointing": false,
437
+ "backbone": "hrnet",
438
+ "pretrained": true,
439
+ "hf_repo": "KhangTruong/FOCAL"
440
+ },
441
+ "loss": {
442
+ "mask_loss_type": "combined",
443
+ "bce_weight": 0.5,
444
+ "dice_weight": 0.5,
445
+ "focal_weight": 0.5,
446
+ "focal_gamma": 2.0,
447
+ "focal_alpha": 0.25,
448
+ "aux_loss_type": "cross_entropy",
449
+ "aux_weight": 0.2
450
+ },
451
+ "training": {
452
+ "epochs": 10,
453
+ "learning_rate": 0.0001,
454
+ "weight_decay": 0.0001,
455
+ "optimizer": "adamw",
456
+ "use_8bit_optimizer": false,
457
+ "check_8bit_compatibility": true,
458
+ "fallback_to_16bit": true,
459
+ "fallback_on_unsupported_8bit": true,
460
+ "scheduler": "cosine",
461
+ "resume_lr_mode": "auto",
462
+ "resume_lr": null,
463
+ "warmup_epochs": 1,
464
+ "min_lr": 1e-06,
465
+ "grad_clip_norm": 1.0,
466
+ "amp": true,
467
+ "gradient_accumulation_steps": 1,
468
+ "data_parallel": true,
469
+ "use_hard_mining": false,
470
+ "auto_batch_size": true,
471
+ "empty_cache_per_epoch": true,
472
+ "save_best": true,
473
+ "save_latest": false,
474
+ "checkpoint_period": 3600,
475
+ "eval_interval": 1,
476
+ "early_stopping_patience": 5,
477
+ "early_stopping_metric": "val_iou",
478
+ "early_stopping_mode": "max",
479
+ "push_to_hub": false,
480
+ "hub_repo": null,
481
+ "hub_version": "v1"
482
+ },
483
+ "hard_mining": {
484
+ "enabled": false,
485
+ "metric": "median",
486
+ "reset_epochs": 5
487
+ },
488
+ "bootstrapping": {
489
+ "enabled": false,
490
+ "run_bootstrap": false,
491
+ "epochs": 5,
492
+ "num_samples": 512,
493
+ "batch_size": null,
494
+ "learning_rate": null,
495
+ "freeze_strategy": "channel_stream",
496
+ "stream_ratio": 0.5,
497
+ "release_mode": "restore",
498
+ "freeze_modules": [],
499
+ "initialization": "kaiming_normal",
500
+ "target_score": 0.5,
501
+ "min_loss_drop": 0.15,
502
+ "early_stopping": false,
503
+ "patience": 3
504
+ },
505
+ "logging": {
506
+ "log_interval": 20,
507
+ "log_memory": true,
508
+ "measure_network": true
509
+ },
510
+ "post_processing": {
511
+ "enabled": true,
512
+ "min_area": 64,
513
+ "fill_holes": true,
514
+ "max_hole_size": 256,
515
+ "morphology": "open_close",
516
+ "morph_kernel_size": 3
517
+ }
518
+ },
519
+ "eval_config": {
520
+ "project": {
521
+ "name": "eval_casia_v2.0",
522
+ "seed": 42,
523
+ "device": "auto",
524
+ "output_dir": "outputs/cross_eval_results",
525
+ "hub_repo": null,
526
+ "resume_repo": null
527
+ },
528
+ "data": {
529
+ "dataset_name": "KhangTruong/Casia-v2.0",
530
+ "streaming": false,
531
+ "image_size": [
532
+ 256,
533
+ 256
534
+ ],
535
+ "batch_size": 4,
536
+ "num_workers": 0,
537
+ "pin_memory": true,
538
+ "shuffle_buffer_size": 1000,
539
+ "train_split": "train",
540
+ "val_split": "validation",
541
+ "test_split": "cross_test",
542
+ "eval_split": "cross_test",
543
+ "train_samples_per_epoch": -1,
544
+ "val_samples_per_epoch": -1,
545
+ "val_samples": -1,
546
+ "test_samples": -1,
547
+ "evaluate_on_test": true,
548
+ "augmentations": {
549
+ "horizontal_flip": 0.5,
550
+ "vertical_flip": 0.2,
551
+ "random_rotate90": 0.5,
552
+ "color_jitter": 0.0
553
+ }
554
+ },
555
+ "model": {
556
+ "name": "unet",
557
+ "in_channels": 3,
558
+ "out_channels": 1,
559
+ "features": [
560
+ 64,
561
+ 128,
562
+ 256,
563
+ 512
564
+ ],
565
+ "bilinear": true,
566
+ "dropout": 0.0,
567
+ "aux_classifier": false,
568
+ "num_classes": 3,
569
+ "gradient_checkpointing": false
570
+ },
571
+ "loss": {
572
+ "mask_loss_type": "combined",
573
+ "bce_weight": 0.5,
574
+ "dice_weight": 0.5,
575
+ "focal_weight": 0.5,
576
+ "focal_gamma": 2.0,
577
+ "focal_alpha": 0.25,
578
+ "aux_loss_type": "cross_entropy",
579
+ "aux_weight": 0.2
580
+ },
581
+ "training": {
582
+ "epochs": 10,
583
+ "learning_rate": 0.001,
584
+ "weight_decay": 0.0001,
585
+ "optimizer": "adamw",
586
+ "use_8bit_optimizer": false,
587
+ "check_8bit_compatibility": true,
588
+ "fallback_to_16bit": true,
589
+ "fallback_on_unsupported_8bit": true,
590
+ "scheduler": "cosine",
591
+ "resume_lr_mode": "auto",
592
+ "resume_lr": null,
593
+ "warmup_epochs": 1,
594
+ "min_lr": 1e-06,
595
+ "grad_clip_norm": 1.0,
596
+ "amp": true,
597
+ "gradient_accumulation_steps": 1,
598
+ "data_parallel": true,
599
+ "use_hard_mining": false,
600
+ "auto_batch_size": true,
601
+ "empty_cache_per_epoch": true,
602
+ "save_best": true,
603
+ "save_latest": false,
604
+ "checkpoint_period": 3600,
605
+ "eval_interval": 1,
606
+ "early_stopping_patience": 5,
607
+ "early_stopping_metric": "val_iou",
608
+ "early_stopping_mode": "max",
609
+ "push_to_hub": false,
610
+ "hub_repo": null,
611
+ "hub_version": "v1"
612
+ },
613
+ "hard_mining": {
614
+ "enabled": false,
615
+ "metric": "median",
616
+ "reset_epochs": 5
617
+ },
618
+ "bootstrapping": {
619
+ "enabled": false,
620
+ "run_bootstrap": false,
621
+ "epochs": 5,
622
+ "num_samples": 512,
623
+ "batch_size": null,
624
+ "learning_rate": null,
625
+ "freeze_strategy": "channel_stream",
626
+ "stream_ratio": 0.5,
627
+ "release_mode": "restore",
628
+ "freeze_modules": [],
629
+ "initialization": "kaiming_normal",
630
+ "target_score": 0.5,
631
+ "min_loss_drop": 0.15,
632
+ "early_stopping": false,
633
+ "patience": 3
634
+ },
635
+ "logging": {
636
+ "log_interval": 20,
637
+ "log_memory": true,
638
+ "measure_network": true
639
+ },
640
+ "post_processing": {
641
+ "enabled": true,
642
+ "min_area": 64,
643
+ "fill_holes": true,
644
+ "max_hole_size": 256,
645
+ "morphology": "open_close",
646
+ "morph_kernel_size": 3
647
+ }
648
+ },
649
+ "checkpoint_hyperparameters": {
650
+ "Project Name": "sid_focal_hrnet",
651
+ "Dataset": "KhangTruong/IMD2020",
652
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
653
+ "Streaming": false,
654
+ "Image Size": "[256, 256]",
655
+ "Batch Size": 16,
656
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
657
+ "Model": "focal (features=[64, 128, 256, 512], bilinear=True)",
658
+ "Aux Classifier": "False (classes=2)",
659
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
660
+ "Aux Loss": "cross_entropy (wt=0.2)",
661
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
662
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
663
+ "Scheduler": "cosine",
664
+ "Epochs": 10,
665
+ "AMP": true
666
+ },
667
+ "eval_hyperparameters": {
668
+ "Project Name": "eval_casia_v2.0",
669
+ "Dataset": "KhangTruong/Casia-v2.0",
670
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
671
+ "Streaming": false,
672
+ "Image Size": "[256, 256]",
673
+ "Batch Size": 4,
674
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
675
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
676
+ "Aux Classifier": "False (classes=3)",
677
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
678
+ "Aux Loss": "cross_entropy (wt=0.2)",
679
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
680
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
681
+ "Scheduler": "cosine",
682
+ "Epochs": 10,
683
+ "AMP": true
684
+ },
685
+ "devices": [
686
+ "cuda"
687
+ ],
688
+ "gpu_utilization": {}
689
+ },
690
+ {
691
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
692
+ "checkpoint_name": "sid_pim_swin",
693
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
694
+ "config_stem": "casia_v2.0",
695
+ "config_name": "eval_casia_v2.0",
696
+ "dataset_name": "KhangTruong/Casia-v2.0",
697
+ "eval_split": "cross_test",
698
+ "total_evaluated_samples": 20,
699
+ "metrics": {
700
+ "eval_split": "cross_test",
701
+ "eval_total_loss": 0.6688968300819397,
702
+ "iou": 0.2,
703
+ "dice": 0.2,
704
+ "f1": 0.2,
705
+ "pixel_f1": 0.2,
706
+ "auroc": 1.0,
707
+ "pixel_auroc": 1.0,
708
+ "pixel_acc": 0.9088134765625,
709
+ "precision": 0.2,
710
+ "recall": 0.2,
711
+ "specificity": 0.9088134765625,
712
+ "iou_all": 0.2,
713
+ "dice_all": 0.2,
714
+ "f1_all": 0.2,
715
+ "positive_samples": 0,
716
+ "zero_positive_samples": 20,
717
+ "iou_positive": 0.2,
718
+ "dice_positive": 0.2,
719
+ "f1_positive": 0.2,
720
+ "precision_positive": 0.2,
721
+ "recall_positive": 0.2,
722
+ "total_evaluated_samples": 20,
723
+ "ablation_summary": {
724
+ "Baseline (Raw UNet)": {
725
+ "iou": 0.15,
726
+ "dice": 0.15,
727
+ "f1": 0.15,
728
+ "pixel_f1": 0.15,
729
+ "auroc": 1.0,
730
+ "pixel_auroc": 1.0,
731
+ "pixel_acc": 0.908416748046875,
732
+ "precision": 0.15,
733
+ "recall": 0.15,
734
+ "specificity": 0.908416748046875,
735
+ "iou_all": 0.15,
736
+ "dice_all": 0.15,
737
+ "f1_all": 0.15,
738
+ "positive_samples": 0,
739
+ "zero_positive_samples": 20,
740
+ "iou_positive": 0.15,
741
+ "dice_positive": 0.15,
742
+ "f1_positive": 0.15,
743
+ "precision_positive": 0.15,
744
+ "recall_positive": 0.15
745
+ },
746
+ "+ Post-Processing": {
747
+ "iou": 0.2,
748
+ "dice": 0.2,
749
+ "f1": 0.2,
750
+ "pixel_f1": 0.2,
751
+ "auroc": 1.0,
752
+ "pixel_auroc": 1.0,
753
+ "pixel_acc": 0.9088134765625,
754
+ "precision": 0.2,
755
+ "recall": 0.2,
756
+ "specificity": 0.9088134765625,
757
+ "iou_all": 0.2,
758
+ "dice_all": 0.2,
759
+ "f1_all": 0.2,
760
+ "positive_samples": 0,
761
+ "zero_positive_samples": 20,
762
+ "iou_positive": 0.2,
763
+ "dice_positive": 0.2,
764
+ "f1_positive": 0.2,
765
+ "precision_positive": 0.2,
766
+ "recall_positive": 0.2
767
+ }
768
+ },
769
+ "per_dataset_metrics": {}
770
+ },
771
+ "overall_metrics": {
772
+ "eval_split": "cross_test",
773
+ "eval_total_loss": 0.6688968300819397,
774
+ "iou": 0.2,
775
+ "dice": 0.2,
776
+ "f1": 0.2,
777
+ "pixel_f1": 0.2,
778
+ "auroc": 1.0,
779
+ "pixel_auroc": 1.0,
780
+ "pixel_acc": 0.9088134765625,
781
+ "precision": 0.2,
782
+ "recall": 0.2,
783
+ "specificity": 0.9088134765625,
784
+ "iou_all": 0.2,
785
+ "dice_all": 0.2,
786
+ "f1_all": 0.2,
787
+ "positive_samples": 0,
788
+ "zero_positive_samples": 20,
789
+ "iou_positive": 0.2,
790
+ "dice_positive": 0.2,
791
+ "f1_positive": 0.2,
792
+ "precision_positive": 0.2,
793
+ "recall_positive": 0.2,
794
+ "total_evaluated_samples": 20,
795
+ "ablation_summary": {
796
+ "Baseline (Raw UNet)": {
797
+ "iou": 0.15,
798
+ "dice": 0.15,
799
+ "f1": 0.15,
800
+ "pixel_f1": 0.15,
801
+ "auroc": 1.0,
802
+ "pixel_auroc": 1.0,
803
+ "pixel_acc": 0.908416748046875,
804
+ "precision": 0.15,
805
+ "recall": 0.15,
806
+ "specificity": 0.908416748046875,
807
+ "iou_all": 0.15,
808
+ "dice_all": 0.15,
809
+ "f1_all": 0.15,
810
+ "positive_samples": 0,
811
+ "zero_positive_samples": 20,
812
+ "iou_positive": 0.15,
813
+ "dice_positive": 0.15,
814
+ "f1_positive": 0.15,
815
+ "precision_positive": 0.15,
816
+ "recall_positive": 0.15
817
+ },
818
+ "+ Post-Processing": {
819
+ "iou": 0.2,
820
+ "dice": 0.2,
821
+ "f1": 0.2,
822
+ "pixel_f1": 0.2,
823
+ "auroc": 1.0,
824
+ "pixel_auroc": 1.0,
825
+ "pixel_acc": 0.9088134765625,
826
+ "precision": 0.2,
827
+ "recall": 0.2,
828
+ "specificity": 0.9088134765625,
829
+ "iou_all": 0.2,
830
+ "dice_all": 0.2,
831
+ "f1_all": 0.2,
832
+ "positive_samples": 0,
833
+ "zero_positive_samples": 20,
834
+ "iou_positive": 0.2,
835
+ "dice_positive": 0.2,
836
+ "f1_positive": 0.2,
837
+ "precision_positive": 0.2,
838
+ "recall_positive": 0.2
839
+ }
840
+ },
841
+ "per_dataset_metrics": {}
842
+ },
843
+ "ablation_metrics": {
844
+ "Baseline (Raw UNet)": {
845
+ "iou": 0.15,
846
+ "dice": 0.15,
847
+ "f1": 0.15,
848
+ "pixel_f1": 0.15,
849
+ "auroc": 1.0,
850
+ "pixel_auroc": 1.0,
851
+ "pixel_acc": 0.908416748046875,
852
+ "precision": 0.15,
853
+ "recall": 0.15,
854
+ "specificity": 0.908416748046875,
855
+ "iou_all": 0.15,
856
+ "dice_all": 0.15,
857
+ "f1_all": 0.15,
858
+ "positive_samples": 0,
859
+ "zero_positive_samples": 20,
860
+ "iou_positive": 0.15,
861
+ "dice_positive": 0.15,
862
+ "f1_positive": 0.15,
863
+ "precision_positive": 0.15,
864
+ "recall_positive": 0.15
865
+ },
866
+ "+ Post-Processing": {
867
+ "iou": 0.2,
868
+ "dice": 0.2,
869
+ "f1": 0.2,
870
+ "pixel_f1": 0.2,
871
+ "auroc": 1.0,
872
+ "pixel_auroc": 1.0,
873
+ "pixel_acc": 0.9088134765625,
874
+ "precision": 0.2,
875
+ "recall": 0.2,
876
+ "specificity": 0.9088134765625,
877
+ "iou_all": 0.2,
878
+ "dice_all": 0.2,
879
+ "f1_all": 0.2,
880
+ "positive_samples": 0,
881
+ "zero_positive_samples": 20,
882
+ "iou_positive": 0.2,
883
+ "dice_positive": 0.2,
884
+ "f1_positive": 0.2,
885
+ "precision_positive": 0.2,
886
+ "recall_positive": 0.2
887
+ }
888
+ },
889
+ "per_label_metrics": {
890
+ "0": {
891
+ "iou": 0.2,
892
+ "iou_all": 0.2,
893
+ "iou_positive": 0.2,
894
+ "dice": 0.2,
895
+ "dice_all": 0.2,
896
+ "dice_positive": 0.2,
897
+ "f1": 0.2,
898
+ "f1_all": 0.2,
899
+ "f1_positive": 0.2,
900
+ "pixel_f1": 0.2,
901
+ "auroc": 1.0,
902
+ "pixel_auroc": 1.0,
903
+ "pixel_acc": 0.9088134765625,
904
+ "precision": 0.2,
905
+ "precision_all": 0.2,
906
+ "precision_positive": 0.2,
907
+ "recall": 0.2,
908
+ "recall_all": 0.2,
909
+ "recall_positive": 0.2,
910
+ "specificity": 0.9088134765625,
911
+ "samples": 20,
912
+ "positive_samples": 0
913
+ },
914
+ "1": {
915
+ "iou": 0.0,
916
+ "iou_all": 0.0,
917
+ "iou_positive": 0.0,
918
+ "dice": 0.0,
919
+ "dice_all": 0.0,
920
+ "dice_positive": 0.0,
921
+ "f1": 0.0,
922
+ "f1_all": 0.0,
923
+ "f1_positive": 0.0,
924
+ "pixel_f1": 0.0,
925
+ "auroc": 0.0,
926
+ "pixel_auroc": 0.0,
927
+ "pixel_acc": 0.0,
928
+ "precision": 0.0,
929
+ "precision_all": 0.0,
930
+ "precision_positive": 0.0,
931
+ "recall": 0.0,
932
+ "recall_all": 0.0,
933
+ "recall_positive": 0.0,
934
+ "specificity": 0.0,
935
+ "samples": 0,
936
+ "positive_samples": 0
937
+ },
938
+ "2": {
939
+ "iou": 0.0,
940
+ "iou_all": 0.0,
941
+ "iou_positive": 0.0,
942
+ "dice": 0.0,
943
+ "dice_all": 0.0,
944
+ "dice_positive": 0.0,
945
+ "f1": 0.0,
946
+ "f1_all": 0.0,
947
+ "f1_positive": 0.0,
948
+ "pixel_f1": 0.0,
949
+ "auroc": 0.0,
950
+ "pixel_auroc": 0.0,
951
+ "pixel_acc": 0.0,
952
+ "precision": 0.0,
953
+ "precision_all": 0.0,
954
+ "precision_positive": 0.0,
955
+ "recall": 0.0,
956
+ "recall_all": 0.0,
957
+ "recall_positive": 0.0,
958
+ "specificity": 0.0,
959
+ "samples": 0,
960
+ "positive_samples": 0
961
+ }
962
+ },
963
+ "raw_per_label_metrics": {
964
+ "0": {
965
+ "iou": 0.15,
966
+ "iou_all": 0.15,
967
+ "iou_positive": 0.15,
968
+ "dice": 0.15,
969
+ "dice_all": 0.15,
970
+ "dice_positive": 0.15,
971
+ "f1": 0.15,
972
+ "f1_all": 0.15,
973
+ "f1_positive": 0.15,
974
+ "pixel_f1": 0.15,
975
+ "auroc": 1.0,
976
+ "pixel_auroc": 1.0,
977
+ "pixel_acc": 0.908416748046875,
978
+ "precision": 0.15,
979
+ "precision_all": 0.15,
980
+ "precision_positive": 0.15,
981
+ "recall": 0.15,
982
+ "recall_all": 0.15,
983
+ "recall_positive": 0.15,
984
+ "specificity": 0.908416748046875,
985
+ "samples": 20,
986
+ "positive_samples": 0
987
+ },
988
+ "1": {
989
+ "iou": 0.0,
990
+ "iou_all": 0.0,
991
+ "iou_positive": 0.0,
992
+ "dice": 0.0,
993
+ "dice_all": 0.0,
994
+ "dice_positive": 0.0,
995
+ "f1": 0.0,
996
+ "f1_all": 0.0,
997
+ "f1_positive": 0.0,
998
+ "pixel_f1": 0.0,
999
+ "auroc": 0.0,
1000
+ "pixel_auroc": 0.0,
1001
+ "pixel_acc": 0.0,
1002
+ "precision": 0.0,
1003
+ "precision_all": 0.0,
1004
+ "precision_positive": 0.0,
1005
+ "recall": 0.0,
1006
+ "recall_all": 0.0,
1007
+ "recall_positive": 0.0,
1008
+ "specificity": 0.0,
1009
+ "samples": 0,
1010
+ "positive_samples": 0
1011
+ },
1012
+ "2": {
1013
+ "iou": 0.0,
1014
+ "iou_all": 0.0,
1015
+ "iou_positive": 0.0,
1016
+ "dice": 0.0,
1017
+ "dice_all": 0.0,
1018
+ "dice_positive": 0.0,
1019
+ "f1": 0.0,
1020
+ "f1_all": 0.0,
1021
+ "f1_positive": 0.0,
1022
+ "pixel_f1": 0.0,
1023
+ "auroc": 0.0,
1024
+ "pixel_auroc": 0.0,
1025
+ "pixel_acc": 0.0,
1026
+ "precision": 0.0,
1027
+ "precision_all": 0.0,
1028
+ "precision_positive": 0.0,
1029
+ "recall": 0.0,
1030
+ "recall_all": 0.0,
1031
+ "recall_positive": 0.0,
1032
+ "specificity": 0.0,
1033
+ "samples": 0,
1034
+ "positive_samples": 0
1035
+ }
1036
+ },
1037
+ "per_dataset_metrics": {},
1038
+ "raw_per_dataset_metrics": {},
1039
+ "confusion_matrix": null,
1040
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
1041
+ "checkpoint_config": {
1042
+ "project": {
1043
+ "name": "sid_pim_swin",
1044
+ "seed": 42,
1045
+ "device": "auto",
1046
+ "output_dir": "outputs/RUN/pim_swin",
1047
+ "hub_repo": null,
1048
+ "resume_repo": null
1049
+ },
1050
+ "data": {
1051
+ "dataset_name": "KhangTruong/IMD2020",
1052
+ "streaming": false,
1053
+ "image_size": [
1054
+ 256,
1055
+ 256
1056
+ ],
1057
+ "batch_size": 8,
1058
+ "num_workers": -1,
1059
+ "pin_memory": true,
1060
+ "shuffle_buffer_size": 1000,
1061
+ "train_split": "train",
1062
+ "val_split": "validation",
1063
+ "test_split": "test",
1064
+ "eval_split": "test",
1065
+ "train_samples_per_epoch": -1,
1066
+ "val_samples_per_epoch": -1,
1067
+ "val_samples": -1,
1068
+ "test_samples": -1,
1069
+ "evaluate_on_test": true,
1070
+ "augmentations": {
1071
+ "horizontal_flip": 0.5,
1072
+ "vertical_flip": 0.2,
1073
+ "random_rotate90": 0.5,
1074
+ "color_jitter": 0.0
1075
+ }
1076
+ },
1077
+ "model": {
1078
+ "name": "pim",
1079
+ "in_channels": 3,
1080
+ "out_channels": 1,
1081
+ "features": [
1082
+ 64,
1083
+ 128,
1084
+ 256,
1085
+ 512
1086
+ ],
1087
+ "bilinear": true,
1088
+ "dropout": 0.1,
1089
+ "aux_classifier": false,
1090
+ "num_classes": 2,
1091
+ "gradient_checkpointing": false,
1092
+ "pretrained": true,
1093
+ "hf_repo": "KhangTruong/PIM"
1094
+ },
1095
+ "loss": {
1096
+ "mask_loss_type": "combined",
1097
+ "bce_weight": 0.5,
1098
+ "dice_weight": 0.5,
1099
+ "focal_weight": 0.5,
1100
+ "focal_gamma": 2.0,
1101
+ "focal_alpha": 0.25,
1102
+ "aux_loss_type": "cross_entropy",
1103
+ "aux_weight": 0.2
1104
+ },
1105
+ "training": {
1106
+ "epochs": 10,
1107
+ "learning_rate": 0.0001,
1108
+ "weight_decay": 0.0001,
1109
+ "optimizer": "adamw",
1110
+ "use_8bit_optimizer": false,
1111
+ "check_8bit_compatibility": true,
1112
+ "fallback_to_16bit": true,
1113
+ "fallback_on_unsupported_8bit": true,
1114
+ "scheduler": "cosine",
1115
+ "resume_lr_mode": "auto",
1116
+ "resume_lr": null,
1117
+ "warmup_epochs": 1,
1118
+ "min_lr": 1e-06,
1119
+ "grad_clip_norm": 1.0,
1120
+ "amp": true,
1121
+ "gradient_accumulation_steps": 1,
1122
+ "data_parallel": true,
1123
+ "use_hard_mining": false,
1124
+ "auto_batch_size": true,
1125
+ "empty_cache_per_epoch": true,
1126
+ "save_best": true,
1127
+ "save_latest": false,
1128
+ "checkpoint_period": 3600,
1129
+ "eval_interval": 1,
1130
+ "early_stopping_patience": 5,
1131
+ "early_stopping_metric": "val_iou",
1132
+ "early_stopping_mode": "max",
1133
+ "push_to_hub": false,
1134
+ "hub_repo": null,
1135
+ "hub_version": "v1"
1136
+ },
1137
+ "hard_mining": {
1138
+ "enabled": false,
1139
+ "metric": "median",
1140
+ "reset_epochs": 5
1141
+ },
1142
+ "bootstrapping": {
1143
+ "enabled": false,
1144
+ "run_bootstrap": false,
1145
+ "epochs": 5,
1146
+ "num_samples": 512,
1147
+ "batch_size": null,
1148
+ "learning_rate": null,
1149
+ "freeze_strategy": "channel_stream",
1150
+ "stream_ratio": 0.5,
1151
+ "release_mode": "restore",
1152
+ "freeze_modules": [],
1153
+ "initialization": "kaiming_normal",
1154
+ "target_score": 0.5,
1155
+ "min_loss_drop": 0.15,
1156
+ "early_stopping": false,
1157
+ "patience": 3
1158
+ },
1159
+ "logging": {
1160
+ "log_interval": 20,
1161
+ "log_memory": true,
1162
+ "measure_network": true
1163
+ },
1164
+ "post_processing": {
1165
+ "enabled": true,
1166
+ "min_area": 64,
1167
+ "fill_holes": true,
1168
+ "max_hole_size": 256,
1169
+ "morphology": "open_close",
1170
+ "morph_kernel_size": 3
1171
+ }
1172
+ },
1173
+ "eval_config": {
1174
+ "project": {
1175
+ "name": "eval_casia_v2.0",
1176
+ "seed": 42,
1177
+ "device": "auto",
1178
+ "output_dir": "outputs/cross_eval_results",
1179
+ "hub_repo": null,
1180
+ "resume_repo": null
1181
+ },
1182
+ "data": {
1183
+ "dataset_name": "KhangTruong/Casia-v2.0",
1184
+ "streaming": false,
1185
+ "image_size": [
1186
+ 256,
1187
+ 256
1188
+ ],
1189
+ "batch_size": 4,
1190
+ "num_workers": 0,
1191
+ "pin_memory": true,
1192
+ "shuffle_buffer_size": 1000,
1193
+ "train_split": "train",
1194
+ "val_split": "validation",
1195
+ "test_split": "cross_test",
1196
+ "eval_split": "cross_test",
1197
+ "train_samples_per_epoch": -1,
1198
+ "val_samples_per_epoch": -1,
1199
+ "val_samples": -1,
1200
+ "test_samples": -1,
1201
+ "evaluate_on_test": true,
1202
+ "augmentations": {
1203
+ "horizontal_flip": 0.5,
1204
+ "vertical_flip": 0.2,
1205
+ "random_rotate90": 0.5,
1206
+ "color_jitter": 0.0
1207
+ }
1208
+ },
1209
+ "model": {
1210
+ "name": "unet",
1211
+ "in_channels": 3,
1212
+ "out_channels": 1,
1213
+ "features": [
1214
+ 64,
1215
+ 128,
1216
+ 256,
1217
+ 512
1218
+ ],
1219
+ "bilinear": true,
1220
+ "dropout": 0.0,
1221
+ "aux_classifier": false,
1222
+ "num_classes": 3,
1223
+ "gradient_checkpointing": false
1224
+ },
1225
+ "loss": {
1226
+ "mask_loss_type": "combined",
1227
+ "bce_weight": 0.5,
1228
+ "dice_weight": 0.5,
1229
+ "focal_weight": 0.5,
1230
+ "focal_gamma": 2.0,
1231
+ "focal_alpha": 0.25,
1232
+ "aux_loss_type": "cross_entropy",
1233
+ "aux_weight": 0.2
1234
+ },
1235
+ "training": {
1236
+ "epochs": 10,
1237
+ "learning_rate": 0.001,
1238
+ "weight_decay": 0.0001,
1239
+ "optimizer": "adamw",
1240
+ "use_8bit_optimizer": false,
1241
+ "check_8bit_compatibility": true,
1242
+ "fallback_to_16bit": true,
1243
+ "fallback_on_unsupported_8bit": true,
1244
+ "scheduler": "cosine",
1245
+ "resume_lr_mode": "auto",
1246
+ "resume_lr": null,
1247
+ "warmup_epochs": 1,
1248
+ "min_lr": 1e-06,
1249
+ "grad_clip_norm": 1.0,
1250
+ "amp": true,
1251
+ "gradient_accumulation_steps": 1,
1252
+ "data_parallel": true,
1253
+ "use_hard_mining": false,
1254
+ "auto_batch_size": true,
1255
+ "empty_cache_per_epoch": true,
1256
+ "save_best": true,
1257
+ "save_latest": false,
1258
+ "checkpoint_period": 3600,
1259
+ "eval_interval": 1,
1260
+ "early_stopping_patience": 5,
1261
+ "early_stopping_metric": "val_iou",
1262
+ "early_stopping_mode": "max",
1263
+ "push_to_hub": false,
1264
+ "hub_repo": null,
1265
+ "hub_version": "v1"
1266
+ },
1267
+ "hard_mining": {
1268
+ "enabled": false,
1269
+ "metric": "median",
1270
+ "reset_epochs": 5
1271
+ },
1272
+ "bootstrapping": {
1273
+ "enabled": false,
1274
+ "run_bootstrap": false,
1275
+ "epochs": 5,
1276
+ "num_samples": 512,
1277
+ "batch_size": null,
1278
+ "learning_rate": null,
1279
+ "freeze_strategy": "channel_stream",
1280
+ "stream_ratio": 0.5,
1281
+ "release_mode": "restore",
1282
+ "freeze_modules": [],
1283
+ "initialization": "kaiming_normal",
1284
+ "target_score": 0.5,
1285
+ "min_loss_drop": 0.15,
1286
+ "early_stopping": false,
1287
+ "patience": 3
1288
+ },
1289
+ "logging": {
1290
+ "log_interval": 20,
1291
+ "log_memory": true,
1292
+ "measure_network": true
1293
+ },
1294
+ "post_processing": {
1295
+ "enabled": true,
1296
+ "min_area": 64,
1297
+ "fill_holes": true,
1298
+ "max_hole_size": 256,
1299
+ "morphology": "open_close",
1300
+ "morph_kernel_size": 3
1301
+ }
1302
+ },
1303
+ "checkpoint_hyperparameters": {
1304
+ "Project Name": "sid_pim_swin",
1305
+ "Dataset": "KhangTruong/IMD2020",
1306
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
1307
+ "Streaming": false,
1308
+ "Image Size": "[256, 256]",
1309
+ "Batch Size": 8,
1310
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
1311
+ "Model": "pim (features=[64, 128, 256, 512], bilinear=True)",
1312
+ "Aux Classifier": "False (classes=2)",
1313
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
1314
+ "Aux Loss": "cross_entropy (wt=0.2)",
1315
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
1316
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
1317
+ "Scheduler": "cosine",
1318
+ "Epochs": 10,
1319
+ "AMP": true
1320
+ },
1321
+ "eval_hyperparameters": {
1322
+ "Project Name": "eval_casia_v2.0",
1323
+ "Dataset": "KhangTruong/Casia-v2.0",
1324
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
1325
+ "Streaming": false,
1326
+ "Image Size": "[256, 256]",
1327
+ "Batch Size": 4,
1328
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
1329
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
1330
+ "Aux Classifier": "False (classes=3)",
1331
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
1332
+ "Aux Loss": "cross_entropy (wt=0.2)",
1333
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
1334
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
1335
+ "Scheduler": "cosine",
1336
+ "Epochs": 10,
1337
+ "AMP": true
1338
+ },
1339
+ "devices": [
1340
+ "cuda"
1341
+ ],
1342
+ "gpu_utilization": {}
1343
+ }
1344
+ ],
1345
+ "illustration_paths": [
1346
+ "outputs/cross_evaluation/illustrations/cross_eval_iou_heatmap.png",
1347
+ "outputs/cross_evaluation/illustrations/cross_eval_f1_heatmap.png",
1348
+ "outputs/cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png",
1349
+ "outputs/cross_evaluation/illustrations/cross_eval_auroc_heatmap.png",
1350
+ "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png"
1351
+ ]
1352
+ }
cross_evaluation/master_cross_evaluation_report.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SID-UNet Master Cross-Evaluation Report
2
+
3
+ ## Overview
4
+
5
+ - **Evaluated Checkpoints**: 2
6
+ - **Evaluated Configurations**: 1
7
+ - **Total Cross-Evaluation Runs**: 2
8
+
9
+ ## Cross-Evaluation Matrices
10
+
11
+ ### Mean IoU (Intersection over Union) Cross-Evaluation Matrix
12
+
13
+ | Checkpoint \ Config | eval_casia_v2.0 |
14
+ |-----------------------|-------------------|
15
+ | sid_focal_hrnet | 0.0000 |
16
+ | sid_pim_swin | 0.2000 |
17
+
18
+ ### Dice / Pixel F1 Score Cross-Evaluation Matrix
19
+
20
+ | Checkpoint \ Config | eval_casia_v2.0 |
21
+ |-----------------------|-------------------|
22
+ | sid_focal_hrnet | 0.0000 |
23
+ | sid_pim_swin | 0.2000 |
24
+
25
+ ### AUROC (Pixel-level ROC-AUC) Cross-Evaluation Matrix
26
+
27
+ | Checkpoint \ Config | eval_casia_v2.0 |
28
+ |-----------------------|-------------------|
29
+ | sid_focal_hrnet | 0.0063 |
30
+ | sid_pim_swin | 1.0000 |
31
+
32
+ ### Pixel Accuracy Cross-Evaluation Matrix
33
+
34
+ | Checkpoint \ Config | eval_casia_v2.0 |
35
+ |-----------------------|-------------------|
36
+ | sid_focal_hrnet | 0.0063 |
37
+ | sid_pim_swin | 0.9088 |
38
+
39
+ ### Evaluation Total Loss Cross-Evaluation Matrix
40
+
41
+ | Checkpoint \ Config | eval_casia_v2.0 |
42
+ |-----------------------|-------------------|
43
+ | sid_focal_hrnet | 1.4154 |
44
+ | sid_pim_swin | 0.6689 |
45
+
46
+ ## Master Consolidated Ranking
47
+
48
+ | Checkpoint | Config (Dataset) | Split | Samples | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Precision | Recall | Specificity | Class Acc |
49
+ |-----------------|------------------------------------------|------------|-----------|--------|--------|-------------|---------|-------------|-------------|----------|---------------|-------------|
50
+ | sid_pim_swin | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 0.6689 | 0.2000 | 0.2000 | 1.0000 | 0.9088 | 0.2000 | 0.2000 | 0.9088 | N/A |
51
+ | sid_focal_hrnet | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 1.4154 | 0.0000 | 0.0000 | 0.0063 | 0.0063 | 0.0000 | 0.0000 | 0.0063 | N/A |
52
+
53
+
54
+ ## 🖼️ Cross-Evaluation Heatmaps & Visual Illustrations
55
+
56
+ ### Cross Eval Iou Heatmap
57
+
58
+ ![Cross Eval Iou Heatmap](illustrations/cross_eval_iou_heatmap.png)
59
+
60
+ ### Cross Eval F1 Heatmap
61
+
62
+ ![Cross Eval F1 Heatmap](illustrations/cross_eval_f1_heatmap.png)
63
+
64
+ ### Cross Eval Pixel Acc Heatmap
65
+
66
+ ![Cross Eval Pixel Acc Heatmap](illustrations/cross_eval_pixel_acc_heatmap.png)
67
+
68
+ ### Cross Eval Auroc Heatmap
69
+
70
+ ![Cross Eval Auroc Heatmap](illustrations/cross_eval_auroc_heatmap.png)
71
+
72
+ ### Sample Predictions Casia V2.0
73
+
74
+ ![Sample Predictions Casia V2.0](illustrations/sample_predictions_casia_v2.0.png)
75
+
76
+ ---
77
+
78
+ ## Per-Checkpoint Deep-Dive
79
+
80
+ ### Checkpoint: sid_focal_hrnet
81
+
82
+ - **Path**: `outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt`
83
+
84
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
85
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
86
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
87
+
88
+
89
+ ### Checkpoint: sid_pim_swin
90
+
91
+ - **Path**: `outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt`
92
+
93
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
94
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
95
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
96
+
97
+
98
+ ---
99
+
100
+ ## Per-Configuration / Dataset Deep-Dive
101
+
102
+ ### Configuration: eval_casia_v2.0 (Dataset: KhangTruong/Casia-v2.0)
103
+
104
+ - **Config Path**: `configs/cross-eval/casia_v2.0.yaml`
105
+
106
+ | Checkpoint | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
107
+ |-----------------|--------|-------|-------------|---------|-------------|-----------|
108
+ | sid_focal_hrnet | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
109
+ | sid_pim_swin | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
110
+
manifest.json CHANGED
@@ -9,7 +9,7 @@
9
  "best_epoch": "10",
10
  "total_epochs": "10",
11
  "step": "1000",
12
- "updated_at": "2026-11-12T06:51:00.954724+00:00",
13
  "files": [
14
  "checkpoint_best.pt",
15
  "checkpoint_latest.pt"
 
9
  "best_epoch": "10",
10
  "total_epochs": "10",
11
  "step": "1000",
12
+ "updated_at": "2026-11-12T06:56:08.289820+00:00",
13
  "files": [
14
  "checkpoint_best.pt",
15
  "checkpoint_latest.pt"
versions/v1/cross_evaluation/cross_eval_matrix.json ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "iou": {
3
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
4
+ "sid_pim_swin:eval_casia_v2.0": 0.2
5
+ },
6
+ "dice_f1": {
7
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
8
+ "sid_pim_swin:eval_casia_v2.0": 0.2
9
+ },
10
+ "auroc": {
11
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
12
+ "sid_pim_swin:eval_casia_v2.0": 1.0
13
+ },
14
+ "pixel_acc": {
15
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
16
+ "sid_pim_swin:eval_casia_v2.0": 0.9088134765625
17
+ },
18
+ "loss": {
19
+ "sid_focal_hrnet:eval_casia_v2.0": 1.415394163131714,
20
+ "sid_pim_swin:eval_casia_v2.0": 0.6688968300819397
21
+ }
22
+ }
versions/v1/cross_evaluation/cross_evaluation.log ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2026-11-12 06:54:39] [INFO] ===========================================================================
2
+ [2026-11-12 06:54:39] [INFO] 🚀 Starting Cross-Evaluation Matrix
3
+ [2026-11-12 06:54:39] [INFO] Checkpoints (2): ['outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt', 'outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt']
4
+ [2026-11-12 06:54:39] [INFO] Configurations (1): ['configs/cross-eval/casia_v2.0.yaml']
5
+ [2026-11-12 06:54:39] [INFO] Total Target Pairs: 2
6
+ [2026-11-12 06:54:39] [INFO] Collision Detection / Skip: True
7
+ [2026-11-12 06:54:39] [INFO] Kaggle Environment: True | Available GPUs: 1
8
+ [2026-11-12 06:54:39] [INFO] Multi-Device DataParallel: default (auto-detected per configuration)
9
+ [2026-11-12 06:54:39] [INFO] Master Output Directory: outputs/cross_evaluation
10
+ [2026-11-12 06:54:39] [INFO] ===========================================================================
11
+ [2026-11-12 06:54:39] [INFO]
12
+ [1/2] Evaluating Checkpoint 'checkpoint_best.pt' with Config 'casia_v2.0.yaml'...
13
+ [2026-11-12 06:54:40] [INFO] Downloaded FOCAL pretrained weights from HF 'KhangTruong/FOCAL/FOCAL_HRNet_weights.pth': /root/.cache/huggingface/hub/models--KhangTruong--FOCAL/snapshots/8d26d2a9110dc9aaac8bdde0f56d135d8dc71496/FOCAL_HRNet_weights.pth
14
+ [2026-11-12 06:54:40] [INFO] ✅ Loaded genuine FOCAL hrnet pretrained weights from: /root/.cache/huggingface/hub/models--KhangTruong--FOCAL/snapshots/8d26d2a9110dc9aaac8bdde0f56d135d8dc71496/FOCAL_HRNet_weights.pth (missing: 0, unexpected: 2)
15
+ [2026-11-12 06:54:56] [INFO] -> Dataset: 'KhangTruong/Casia-v2.0' (cross_test) | Loss: 1.4154 | IoU: 0.0000 | F1: 0.0000 | AUROC: 0.0063 | PixelAcc: 0.0063
16
+ [2026-11-12 06:54:56] [INFO]
17
+ [2/2] Evaluating Checkpoint 'checkpoint_best.pt' with Config 'casia_v2.0.yaml'...
18
+ [2026-11-12 06:54:59] [INFO] Downloaded PIM pretrained weights from HF 'KhangTruong/PIM/ckpt.pth': /root/.cache/huggingface/hub/models--KhangTruong--PIM/snapshots/33e8110fd64d4dbb034471144dea5d61060277cd/ckpt.pth
19
+ [2026-11-12 06:55:00] [INFO] ✅ Loaded genuine PIM pretrained weights from: /root/.cache/huggingface/hub/models--KhangTruong--PIM/snapshots/33e8110fd64d4dbb034471144dea5d61060277cd/ckpt.pth (missing: 0, unexpected: 0)
20
+ [2026-11-12 06:55:15] [INFO] -> Dataset: 'KhangTruong/Casia-v2.0' (cross_test) | Loss: 0.6689 | IoU: 0.2000 | F1: 0.2000 | AUROC: 1.0000 | PixelAcc: 0.9088
21
+ [2026-11-12 06:55:15] [INFO] ✅ Checkpoint neighbor report saved at: /content/ai-detection-unet-gpu/outputs/RUN/focal_hrnet/cross_eval_reports/cross_evaluation_report.md
22
+ [2026-11-12 06:55:15] [INFO] ✅ Checkpoint neighbor report saved at: /content/ai-detection-unet-gpu/outputs/RUN/pim_swin/cross_eval_reports/cross_evaluation_report.md
23
+ [2026-11-12 06:55:17] [INFO] Generated 4 cross-evaluation heatmaps in outputs/cross_evaluation/illustrations
versions/v1/cross_evaluation/cross_evaluation_report.json ADDED
@@ -0,0 +1,664 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
3
+ "checkpoint_name": "sid_pim_swin",
4
+ "total_configs": 1,
5
+ "results": [
6
+ {
7
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
8
+ "checkpoint_name": "sid_pim_swin",
9
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
10
+ "config_stem": "casia_v2.0",
11
+ "config_name": "eval_casia_v2.0",
12
+ "dataset_name": "KhangTruong/Casia-v2.0",
13
+ "eval_split": "cross_test",
14
+ "total_evaluated_samples": 20,
15
+ "metrics": {
16
+ "eval_split": "cross_test",
17
+ "eval_total_loss": 0.6688968300819397,
18
+ "iou": 0.2,
19
+ "dice": 0.2,
20
+ "f1": 0.2,
21
+ "pixel_f1": 0.2,
22
+ "auroc": 1.0,
23
+ "pixel_auroc": 1.0,
24
+ "pixel_acc": 0.9088134765625,
25
+ "precision": 0.2,
26
+ "recall": 0.2,
27
+ "specificity": 0.9088134765625,
28
+ "iou_all": 0.2,
29
+ "dice_all": 0.2,
30
+ "f1_all": 0.2,
31
+ "positive_samples": 0,
32
+ "zero_positive_samples": 20,
33
+ "iou_positive": 0.2,
34
+ "dice_positive": 0.2,
35
+ "f1_positive": 0.2,
36
+ "precision_positive": 0.2,
37
+ "recall_positive": 0.2,
38
+ "total_evaluated_samples": 20,
39
+ "ablation_summary": {
40
+ "Baseline (Raw UNet)": {
41
+ "iou": 0.15,
42
+ "dice": 0.15,
43
+ "f1": 0.15,
44
+ "pixel_f1": 0.15,
45
+ "auroc": 1.0,
46
+ "pixel_auroc": 1.0,
47
+ "pixel_acc": 0.908416748046875,
48
+ "precision": 0.15,
49
+ "recall": 0.15,
50
+ "specificity": 0.908416748046875,
51
+ "iou_all": 0.15,
52
+ "dice_all": 0.15,
53
+ "f1_all": 0.15,
54
+ "positive_samples": 0,
55
+ "zero_positive_samples": 20,
56
+ "iou_positive": 0.15,
57
+ "dice_positive": 0.15,
58
+ "f1_positive": 0.15,
59
+ "precision_positive": 0.15,
60
+ "recall_positive": 0.15
61
+ },
62
+ "+ Post-Processing": {
63
+ "iou": 0.2,
64
+ "dice": 0.2,
65
+ "f1": 0.2,
66
+ "pixel_f1": 0.2,
67
+ "auroc": 1.0,
68
+ "pixel_auroc": 1.0,
69
+ "pixel_acc": 0.9088134765625,
70
+ "precision": 0.2,
71
+ "recall": 0.2,
72
+ "specificity": 0.9088134765625,
73
+ "iou_all": 0.2,
74
+ "dice_all": 0.2,
75
+ "f1_all": 0.2,
76
+ "positive_samples": 0,
77
+ "zero_positive_samples": 20,
78
+ "iou_positive": 0.2,
79
+ "dice_positive": 0.2,
80
+ "f1_positive": 0.2,
81
+ "precision_positive": 0.2,
82
+ "recall_positive": 0.2
83
+ }
84
+ },
85
+ "per_dataset_metrics": {}
86
+ },
87
+ "overall_metrics": {
88
+ "eval_split": "cross_test",
89
+ "eval_total_loss": 0.6688968300819397,
90
+ "iou": 0.2,
91
+ "dice": 0.2,
92
+ "f1": 0.2,
93
+ "pixel_f1": 0.2,
94
+ "auroc": 1.0,
95
+ "pixel_auroc": 1.0,
96
+ "pixel_acc": 0.9088134765625,
97
+ "precision": 0.2,
98
+ "recall": 0.2,
99
+ "specificity": 0.9088134765625,
100
+ "iou_all": 0.2,
101
+ "dice_all": 0.2,
102
+ "f1_all": 0.2,
103
+ "positive_samples": 0,
104
+ "zero_positive_samples": 20,
105
+ "iou_positive": 0.2,
106
+ "dice_positive": 0.2,
107
+ "f1_positive": 0.2,
108
+ "precision_positive": 0.2,
109
+ "recall_positive": 0.2,
110
+ "total_evaluated_samples": 20,
111
+ "ablation_summary": {
112
+ "Baseline (Raw UNet)": {
113
+ "iou": 0.15,
114
+ "dice": 0.15,
115
+ "f1": 0.15,
116
+ "pixel_f1": 0.15,
117
+ "auroc": 1.0,
118
+ "pixel_auroc": 1.0,
119
+ "pixel_acc": 0.908416748046875,
120
+ "precision": 0.15,
121
+ "recall": 0.15,
122
+ "specificity": 0.908416748046875,
123
+ "iou_all": 0.15,
124
+ "dice_all": 0.15,
125
+ "f1_all": 0.15,
126
+ "positive_samples": 0,
127
+ "zero_positive_samples": 20,
128
+ "iou_positive": 0.15,
129
+ "dice_positive": 0.15,
130
+ "f1_positive": 0.15,
131
+ "precision_positive": 0.15,
132
+ "recall_positive": 0.15
133
+ },
134
+ "+ Post-Processing": {
135
+ "iou": 0.2,
136
+ "dice": 0.2,
137
+ "f1": 0.2,
138
+ "pixel_f1": 0.2,
139
+ "auroc": 1.0,
140
+ "pixel_auroc": 1.0,
141
+ "pixel_acc": 0.9088134765625,
142
+ "precision": 0.2,
143
+ "recall": 0.2,
144
+ "specificity": 0.9088134765625,
145
+ "iou_all": 0.2,
146
+ "dice_all": 0.2,
147
+ "f1_all": 0.2,
148
+ "positive_samples": 0,
149
+ "zero_positive_samples": 20,
150
+ "iou_positive": 0.2,
151
+ "dice_positive": 0.2,
152
+ "f1_positive": 0.2,
153
+ "precision_positive": 0.2,
154
+ "recall_positive": 0.2
155
+ }
156
+ },
157
+ "per_dataset_metrics": {}
158
+ },
159
+ "ablation_metrics": {
160
+ "Baseline (Raw UNet)": {
161
+ "iou": 0.15,
162
+ "dice": 0.15,
163
+ "f1": 0.15,
164
+ "pixel_f1": 0.15,
165
+ "auroc": 1.0,
166
+ "pixel_auroc": 1.0,
167
+ "pixel_acc": 0.908416748046875,
168
+ "precision": 0.15,
169
+ "recall": 0.15,
170
+ "specificity": 0.908416748046875,
171
+ "iou_all": 0.15,
172
+ "dice_all": 0.15,
173
+ "f1_all": 0.15,
174
+ "positive_samples": 0,
175
+ "zero_positive_samples": 20,
176
+ "iou_positive": 0.15,
177
+ "dice_positive": 0.15,
178
+ "f1_positive": 0.15,
179
+ "precision_positive": 0.15,
180
+ "recall_positive": 0.15
181
+ },
182
+ "+ Post-Processing": {
183
+ "iou": 0.2,
184
+ "dice": 0.2,
185
+ "f1": 0.2,
186
+ "pixel_f1": 0.2,
187
+ "auroc": 1.0,
188
+ "pixel_auroc": 1.0,
189
+ "pixel_acc": 0.9088134765625,
190
+ "precision": 0.2,
191
+ "recall": 0.2,
192
+ "specificity": 0.9088134765625,
193
+ "iou_all": 0.2,
194
+ "dice_all": 0.2,
195
+ "f1_all": 0.2,
196
+ "positive_samples": 0,
197
+ "zero_positive_samples": 20,
198
+ "iou_positive": 0.2,
199
+ "dice_positive": 0.2,
200
+ "f1_positive": 0.2,
201
+ "precision_positive": 0.2,
202
+ "recall_positive": 0.2
203
+ }
204
+ },
205
+ "per_label_metrics": {
206
+ "0": {
207
+ "iou": 0.2,
208
+ "iou_all": 0.2,
209
+ "iou_positive": 0.2,
210
+ "dice": 0.2,
211
+ "dice_all": 0.2,
212
+ "dice_positive": 0.2,
213
+ "f1": 0.2,
214
+ "f1_all": 0.2,
215
+ "f1_positive": 0.2,
216
+ "pixel_f1": 0.2,
217
+ "auroc": 1.0,
218
+ "pixel_auroc": 1.0,
219
+ "pixel_acc": 0.9088134765625,
220
+ "precision": 0.2,
221
+ "precision_all": 0.2,
222
+ "precision_positive": 0.2,
223
+ "recall": 0.2,
224
+ "recall_all": 0.2,
225
+ "recall_positive": 0.2,
226
+ "specificity": 0.9088134765625,
227
+ "samples": 20,
228
+ "positive_samples": 0
229
+ },
230
+ "1": {
231
+ "iou": 0.0,
232
+ "iou_all": 0.0,
233
+ "iou_positive": 0.0,
234
+ "dice": 0.0,
235
+ "dice_all": 0.0,
236
+ "dice_positive": 0.0,
237
+ "f1": 0.0,
238
+ "f1_all": 0.0,
239
+ "f1_positive": 0.0,
240
+ "pixel_f1": 0.0,
241
+ "auroc": 0.0,
242
+ "pixel_auroc": 0.0,
243
+ "pixel_acc": 0.0,
244
+ "precision": 0.0,
245
+ "precision_all": 0.0,
246
+ "precision_positive": 0.0,
247
+ "recall": 0.0,
248
+ "recall_all": 0.0,
249
+ "recall_positive": 0.0,
250
+ "specificity": 0.0,
251
+ "samples": 0,
252
+ "positive_samples": 0
253
+ },
254
+ "2": {
255
+ "iou": 0.0,
256
+ "iou_all": 0.0,
257
+ "iou_positive": 0.0,
258
+ "dice": 0.0,
259
+ "dice_all": 0.0,
260
+ "dice_positive": 0.0,
261
+ "f1": 0.0,
262
+ "f1_all": 0.0,
263
+ "f1_positive": 0.0,
264
+ "pixel_f1": 0.0,
265
+ "auroc": 0.0,
266
+ "pixel_auroc": 0.0,
267
+ "pixel_acc": 0.0,
268
+ "precision": 0.0,
269
+ "precision_all": 0.0,
270
+ "precision_positive": 0.0,
271
+ "recall": 0.0,
272
+ "recall_all": 0.0,
273
+ "recall_positive": 0.0,
274
+ "specificity": 0.0,
275
+ "samples": 0,
276
+ "positive_samples": 0
277
+ }
278
+ },
279
+ "raw_per_label_metrics": {
280
+ "0": {
281
+ "iou": 0.15,
282
+ "iou_all": 0.15,
283
+ "iou_positive": 0.15,
284
+ "dice": 0.15,
285
+ "dice_all": 0.15,
286
+ "dice_positive": 0.15,
287
+ "f1": 0.15,
288
+ "f1_all": 0.15,
289
+ "f1_positive": 0.15,
290
+ "pixel_f1": 0.15,
291
+ "auroc": 1.0,
292
+ "pixel_auroc": 1.0,
293
+ "pixel_acc": 0.908416748046875,
294
+ "precision": 0.15,
295
+ "precision_all": 0.15,
296
+ "precision_positive": 0.15,
297
+ "recall": 0.15,
298
+ "recall_all": 0.15,
299
+ "recall_positive": 0.15,
300
+ "specificity": 0.908416748046875,
301
+ "samples": 20,
302
+ "positive_samples": 0
303
+ },
304
+ "1": {
305
+ "iou": 0.0,
306
+ "iou_all": 0.0,
307
+ "iou_positive": 0.0,
308
+ "dice": 0.0,
309
+ "dice_all": 0.0,
310
+ "dice_positive": 0.0,
311
+ "f1": 0.0,
312
+ "f1_all": 0.0,
313
+ "f1_positive": 0.0,
314
+ "pixel_f1": 0.0,
315
+ "auroc": 0.0,
316
+ "pixel_auroc": 0.0,
317
+ "pixel_acc": 0.0,
318
+ "precision": 0.0,
319
+ "precision_all": 0.0,
320
+ "precision_positive": 0.0,
321
+ "recall": 0.0,
322
+ "recall_all": 0.0,
323
+ "recall_positive": 0.0,
324
+ "specificity": 0.0,
325
+ "samples": 0,
326
+ "positive_samples": 0
327
+ },
328
+ "2": {
329
+ "iou": 0.0,
330
+ "iou_all": 0.0,
331
+ "iou_positive": 0.0,
332
+ "dice": 0.0,
333
+ "dice_all": 0.0,
334
+ "dice_positive": 0.0,
335
+ "f1": 0.0,
336
+ "f1_all": 0.0,
337
+ "f1_positive": 0.0,
338
+ "pixel_f1": 0.0,
339
+ "auroc": 0.0,
340
+ "pixel_auroc": 0.0,
341
+ "pixel_acc": 0.0,
342
+ "precision": 0.0,
343
+ "precision_all": 0.0,
344
+ "precision_positive": 0.0,
345
+ "recall": 0.0,
346
+ "recall_all": 0.0,
347
+ "recall_positive": 0.0,
348
+ "specificity": 0.0,
349
+ "samples": 0,
350
+ "positive_samples": 0
351
+ }
352
+ },
353
+ "per_dataset_metrics": {},
354
+ "raw_per_dataset_metrics": {},
355
+ "confusion_matrix": null,
356
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
357
+ "checkpoint_config": {
358
+ "project": {
359
+ "name": "sid_pim_swin",
360
+ "seed": 42,
361
+ "device": "auto",
362
+ "output_dir": "outputs/RUN/pim_swin",
363
+ "hub_repo": null,
364
+ "resume_repo": null
365
+ },
366
+ "data": {
367
+ "dataset_name": "KhangTruong/IMD2020",
368
+ "streaming": false,
369
+ "image_size": [
370
+ 256,
371
+ 256
372
+ ],
373
+ "batch_size": 8,
374
+ "num_workers": -1,
375
+ "pin_memory": true,
376
+ "shuffle_buffer_size": 1000,
377
+ "train_split": "train",
378
+ "val_split": "validation",
379
+ "test_split": "test",
380
+ "eval_split": "test",
381
+ "train_samples_per_epoch": -1,
382
+ "val_samples_per_epoch": -1,
383
+ "val_samples": -1,
384
+ "test_samples": -1,
385
+ "evaluate_on_test": true,
386
+ "augmentations": {
387
+ "horizontal_flip": 0.5,
388
+ "vertical_flip": 0.2,
389
+ "random_rotate90": 0.5,
390
+ "color_jitter": 0.0
391
+ }
392
+ },
393
+ "model": {
394
+ "name": "pim",
395
+ "in_channels": 3,
396
+ "out_channels": 1,
397
+ "features": [
398
+ 64,
399
+ 128,
400
+ 256,
401
+ 512
402
+ ],
403
+ "bilinear": true,
404
+ "dropout": 0.1,
405
+ "aux_classifier": false,
406
+ "num_classes": 2,
407
+ "gradient_checkpointing": false,
408
+ "pretrained": true,
409
+ "hf_repo": "KhangTruong/PIM"
410
+ },
411
+ "loss": {
412
+ "mask_loss_type": "combined",
413
+ "bce_weight": 0.5,
414
+ "dice_weight": 0.5,
415
+ "focal_weight": 0.5,
416
+ "focal_gamma": 2.0,
417
+ "focal_alpha": 0.25,
418
+ "aux_loss_type": "cross_entropy",
419
+ "aux_weight": 0.2
420
+ },
421
+ "training": {
422
+ "epochs": 10,
423
+ "learning_rate": 0.0001,
424
+ "weight_decay": 0.0001,
425
+ "optimizer": "adamw",
426
+ "use_8bit_optimizer": false,
427
+ "check_8bit_compatibility": true,
428
+ "fallback_to_16bit": true,
429
+ "fallback_on_unsupported_8bit": true,
430
+ "scheduler": "cosine",
431
+ "resume_lr_mode": "auto",
432
+ "resume_lr": null,
433
+ "warmup_epochs": 1,
434
+ "min_lr": 1e-06,
435
+ "grad_clip_norm": 1.0,
436
+ "amp": true,
437
+ "gradient_accumulation_steps": 1,
438
+ "data_parallel": true,
439
+ "use_hard_mining": false,
440
+ "auto_batch_size": true,
441
+ "empty_cache_per_epoch": true,
442
+ "save_best": true,
443
+ "save_latest": false,
444
+ "checkpoint_period": 3600,
445
+ "eval_interval": 1,
446
+ "early_stopping_patience": 5,
447
+ "early_stopping_metric": "val_iou",
448
+ "early_stopping_mode": "max",
449
+ "push_to_hub": false,
450
+ "hub_repo": null,
451
+ "hub_version": "v1"
452
+ },
453
+ "hard_mining": {
454
+ "enabled": false,
455
+ "metric": "median",
456
+ "reset_epochs": 5
457
+ },
458
+ "bootstrapping": {
459
+ "enabled": false,
460
+ "run_bootstrap": false,
461
+ "epochs": 5,
462
+ "num_samples": 512,
463
+ "batch_size": null,
464
+ "learning_rate": null,
465
+ "freeze_strategy": "channel_stream",
466
+ "stream_ratio": 0.5,
467
+ "release_mode": "restore",
468
+ "freeze_modules": [],
469
+ "initialization": "kaiming_normal",
470
+ "target_score": 0.5,
471
+ "min_loss_drop": 0.15,
472
+ "early_stopping": false,
473
+ "patience": 3
474
+ },
475
+ "logging": {
476
+ "log_interval": 20,
477
+ "log_memory": true,
478
+ "measure_network": true
479
+ },
480
+ "post_processing": {
481
+ "enabled": true,
482
+ "min_area": 64,
483
+ "fill_holes": true,
484
+ "max_hole_size": 256,
485
+ "morphology": "open_close",
486
+ "morph_kernel_size": 3
487
+ }
488
+ },
489
+ "eval_config": {
490
+ "project": {
491
+ "name": "eval_casia_v2.0",
492
+ "seed": 42,
493
+ "device": "auto",
494
+ "output_dir": "outputs/cross_eval_results",
495
+ "hub_repo": null,
496
+ "resume_repo": null
497
+ },
498
+ "data": {
499
+ "dataset_name": "KhangTruong/Casia-v2.0",
500
+ "streaming": false,
501
+ "image_size": [
502
+ 256,
503
+ 256
504
+ ],
505
+ "batch_size": 4,
506
+ "num_workers": 0,
507
+ "pin_memory": true,
508
+ "shuffle_buffer_size": 1000,
509
+ "train_split": "train",
510
+ "val_split": "validation",
511
+ "test_split": "cross_test",
512
+ "eval_split": "cross_test",
513
+ "train_samples_per_epoch": -1,
514
+ "val_samples_per_epoch": -1,
515
+ "val_samples": -1,
516
+ "test_samples": -1,
517
+ "evaluate_on_test": true,
518
+ "augmentations": {
519
+ "horizontal_flip": 0.5,
520
+ "vertical_flip": 0.2,
521
+ "random_rotate90": 0.5,
522
+ "color_jitter": 0.0
523
+ }
524
+ },
525
+ "model": {
526
+ "name": "unet",
527
+ "in_channels": 3,
528
+ "out_channels": 1,
529
+ "features": [
530
+ 64,
531
+ 128,
532
+ 256,
533
+ 512
534
+ ],
535
+ "bilinear": true,
536
+ "dropout": 0.0,
537
+ "aux_classifier": false,
538
+ "num_classes": 3,
539
+ "gradient_checkpointing": false
540
+ },
541
+ "loss": {
542
+ "mask_loss_type": "combined",
543
+ "bce_weight": 0.5,
544
+ "dice_weight": 0.5,
545
+ "focal_weight": 0.5,
546
+ "focal_gamma": 2.0,
547
+ "focal_alpha": 0.25,
548
+ "aux_loss_type": "cross_entropy",
549
+ "aux_weight": 0.2
550
+ },
551
+ "training": {
552
+ "epochs": 10,
553
+ "learning_rate": 0.001,
554
+ "weight_decay": 0.0001,
555
+ "optimizer": "adamw",
556
+ "use_8bit_optimizer": false,
557
+ "check_8bit_compatibility": true,
558
+ "fallback_to_16bit": true,
559
+ "fallback_on_unsupported_8bit": true,
560
+ "scheduler": "cosine",
561
+ "resume_lr_mode": "auto",
562
+ "resume_lr": null,
563
+ "warmup_epochs": 1,
564
+ "min_lr": 1e-06,
565
+ "grad_clip_norm": 1.0,
566
+ "amp": true,
567
+ "gradient_accumulation_steps": 1,
568
+ "data_parallel": true,
569
+ "use_hard_mining": false,
570
+ "auto_batch_size": true,
571
+ "empty_cache_per_epoch": true,
572
+ "save_best": true,
573
+ "save_latest": false,
574
+ "checkpoint_period": 3600,
575
+ "eval_interval": 1,
576
+ "early_stopping_patience": 5,
577
+ "early_stopping_metric": "val_iou",
578
+ "early_stopping_mode": "max",
579
+ "push_to_hub": false,
580
+ "hub_repo": null,
581
+ "hub_version": "v1"
582
+ },
583
+ "hard_mining": {
584
+ "enabled": false,
585
+ "metric": "median",
586
+ "reset_epochs": 5
587
+ },
588
+ "bootstrapping": {
589
+ "enabled": false,
590
+ "run_bootstrap": false,
591
+ "epochs": 5,
592
+ "num_samples": 512,
593
+ "batch_size": null,
594
+ "learning_rate": null,
595
+ "freeze_strategy": "channel_stream",
596
+ "stream_ratio": 0.5,
597
+ "release_mode": "restore",
598
+ "freeze_modules": [],
599
+ "initialization": "kaiming_normal",
600
+ "target_score": 0.5,
601
+ "min_loss_drop": 0.15,
602
+ "early_stopping": false,
603
+ "patience": 3
604
+ },
605
+ "logging": {
606
+ "log_interval": 20,
607
+ "log_memory": true,
608
+ "measure_network": true
609
+ },
610
+ "post_processing": {
611
+ "enabled": true,
612
+ "min_area": 64,
613
+ "fill_holes": true,
614
+ "max_hole_size": 256,
615
+ "morphology": "open_close",
616
+ "morph_kernel_size": 3
617
+ }
618
+ },
619
+ "checkpoint_hyperparameters": {
620
+ "Project Name": "sid_pim_swin",
621
+ "Dataset": "KhangTruong/IMD2020",
622
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
623
+ "Streaming": false,
624
+ "Image Size": "[256, 256]",
625
+ "Batch Size": 8,
626
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
627
+ "Model": "pim (features=[64, 128, 256, 512], bilinear=True)",
628
+ "Aux Classifier": "False (classes=2)",
629
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
630
+ "Aux Loss": "cross_entropy (wt=0.2)",
631
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
632
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
633
+ "Scheduler": "cosine",
634
+ "Epochs": 10,
635
+ "AMP": true
636
+ },
637
+ "eval_hyperparameters": {
638
+ "Project Name": "eval_casia_v2.0",
639
+ "Dataset": "KhangTruong/Casia-v2.0",
640
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
641
+ "Streaming": false,
642
+ "Image Size": "[256, 256]",
643
+ "Batch Size": 4,
644
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
645
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
646
+ "Aux Classifier": "False (classes=3)",
647
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
648
+ "Aux Loss": "cross_entropy (wt=0.2)",
649
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
650
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
651
+ "Scheduler": "cosine",
652
+ "Epochs": 10,
653
+ "AMP": true
654
+ },
655
+ "devices": [
656
+ "cuda"
657
+ ],
658
+ "gpu_utilization": {}
659
+ }
660
+ ],
661
+ "illustration_paths": [
662
+ "/content/ai-detection-unet-gpu/outputs/RUN/pim_swin/cross_eval_reports/illustrations/sample_predictions_casia_v2.0.png"
663
+ ]
664
+ }
versions/v1/cross_evaluation/cross_evaluation_report.md ADDED
@@ -0,0 +1,61 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Cross-Evaluation Report: sid_pim_swin
2
+
3
+ - **Checkpoint Path**: `outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt`
4
+ - **Evaluated Configurations**: 1
5
+
6
+ ## Cross-Dataset Performance Summary
7
+
8
+ | Config | Dataset | Split | Samples | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Class Acc |
9
+ |-----------------|------------------------|------------|-----------|--------|--------|-------------|---------|-------------|-------------|
10
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | cross_test | 20 | 0.6689 | 0.2000 | 0.2000 | 1.0000 | 0.9088 | N/A |
11
+
12
+
13
+ ---
14
+
15
+ ## Per-Configuration Breakdown
16
+
17
+ ### Configuration: eval_casia_v2.0 (Dataset: KhangTruong/Casia-v2.0)
18
+
19
+ - **Config File**: `configs/cross-eval/casia_v2.0.yaml`
20
+ ### Evaluation Metrics
21
+
22
+ | Metric | Value |
23
+ |-------------------------|------------|
24
+ | Eval Split | cross_test |
25
+ | Eval Total Loss | 0.6689 |
26
+ | Iou | 0.2000 |
27
+ | Dice | 0.2000 |
28
+ | F1 | 0.2000 |
29
+ | Pixel F1 | 0.2000 |
30
+ | Auroc | 1.0000 |
31
+ | Pixel Auroc | 1.0000 |
32
+ | Pixel Acc | 0.9088 |
33
+ | Precision | 0.2000 |
34
+ | Recall | 0.2000 |
35
+ | Specificity | 0.9088 |
36
+ | Iou All | 0.2000 |
37
+ | Dice All | 0.2000 |
38
+ | F1 All | 0.2000 |
39
+ | Positive Samples | 0 |
40
+ | Zero Positive Samples | 20 |
41
+ | Iou Positive | 0.2000 |
42
+ | Dice Positive | 0.2000 |
43
+ | F1 Positive | 0.2000 |
44
+ | Precision Positive | 0.2000 |
45
+ | Recall Positive | 0.2000 |
46
+ | Total Evaluated Samples | 20 |
47
+
48
+ #### Per-Subset Breakdown
49
+
50
+ | Subset | IoU | Dice / F1 | Pixel Acc | Samples |
51
+ |---------------------|--------|-------------|-------------|-----------|
52
+ | Label 0 (Real) | 0.2000 | 0.2000 | 0.9088 | 20 |
53
+ | Label 1 (Synthetic) | 0.0000 | 0.0000 | 0.0000 | 0 |
54
+ | Label 2 (Tampered) | 0.0000 | 0.0000 | 0.0000 | 0 |
55
+
56
+
57
+ ## 🖼️ Visual Illustrations & Sample Predictions
58
+
59
+ **Sample Predictions Casia V2.0**
60
+
61
+ ![Sample Predictions Casia V2.0](illustrations/sample_predictions_casia_v2.0.png)
versions/v1/cross_evaluation/eval_casia_v2.0_report.json ADDED
@@ -0,0 +1,345 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "overall_metrics": {
3
+ "eval_split": "cross_test",
4
+ "eval_total_loss": 0.6688968300819397,
5
+ "iou": 0.2,
6
+ "dice": 0.2,
7
+ "f1": 0.2,
8
+ "pixel_f1": 0.2,
9
+ "auroc": 1.0,
10
+ "pixel_auroc": 1.0,
11
+ "pixel_acc": 0.9088134765625,
12
+ "precision": 0.2,
13
+ "recall": 0.2,
14
+ "specificity": 0.9088134765625,
15
+ "iou_all": 0.2,
16
+ "dice_all": 0.2,
17
+ "f1_all": 0.2,
18
+ "positive_samples": 0,
19
+ "zero_positive_samples": 20,
20
+ "iou_positive": 0.2,
21
+ "dice_positive": 0.2,
22
+ "f1_positive": 0.2,
23
+ "precision_positive": 0.2,
24
+ "recall_positive": 0.2,
25
+ "total_evaluated_samples": 20,
26
+ "ablation_summary": {
27
+ "Baseline (Raw UNet)": {
28
+ "iou": 0.15,
29
+ "dice": 0.15,
30
+ "f1": 0.15,
31
+ "pixel_f1": 0.15,
32
+ "auroc": 1.0,
33
+ "pixel_auroc": 1.0,
34
+ "pixel_acc": 0.908416748046875,
35
+ "precision": 0.15,
36
+ "recall": 0.15,
37
+ "specificity": 0.908416748046875,
38
+ "iou_all": 0.15,
39
+ "dice_all": 0.15,
40
+ "f1_all": 0.15,
41
+ "positive_samples": 0,
42
+ "zero_positive_samples": 20,
43
+ "iou_positive": 0.15,
44
+ "dice_positive": 0.15,
45
+ "f1_positive": 0.15,
46
+ "precision_positive": 0.15,
47
+ "recall_positive": 0.15
48
+ },
49
+ "+ Post-Processing": {
50
+ "iou": 0.2,
51
+ "dice": 0.2,
52
+ "f1": 0.2,
53
+ "pixel_f1": 0.2,
54
+ "auroc": 1.0,
55
+ "pixel_auroc": 1.0,
56
+ "pixel_acc": 0.9088134765625,
57
+ "precision": 0.2,
58
+ "recall": 0.2,
59
+ "specificity": 0.9088134765625,
60
+ "iou_all": 0.2,
61
+ "dice_all": 0.2,
62
+ "f1_all": 0.2,
63
+ "positive_samples": 0,
64
+ "zero_positive_samples": 20,
65
+ "iou_positive": 0.2,
66
+ "dice_positive": 0.2,
67
+ "f1_positive": 0.2,
68
+ "precision_positive": 0.2,
69
+ "recall_positive": 0.2
70
+ }
71
+ },
72
+ "per_dataset_metrics": {}
73
+ },
74
+ "per_label_metrics": {
75
+ "Label 0 (Real / Black Mask)": {
76
+ "iou": 0.2,
77
+ "iou_all": 0.2,
78
+ "iou_positive": 0.2,
79
+ "dice": 0.2,
80
+ "dice_all": 0.2,
81
+ "dice_positive": 0.2,
82
+ "f1": 0.2,
83
+ "f1_all": 0.2,
84
+ "f1_positive": 0.2,
85
+ "pixel_f1": 0.2,
86
+ "auroc": 1.0,
87
+ "pixel_auroc": 1.0,
88
+ "pixel_acc": 0.9088134765625,
89
+ "precision": 0.2,
90
+ "precision_all": 0.2,
91
+ "precision_positive": 0.2,
92
+ "recall": 0.2,
93
+ "recall_all": 0.2,
94
+ "recall_positive": 0.2,
95
+ "specificity": 0.9088134765625,
96
+ "samples": 20,
97
+ "positive_samples": 0
98
+ },
99
+ "Label 1 (Synthetic / White Mask)": {
100
+ "iou": 0.0,
101
+ "iou_all": 0.0,
102
+ "iou_positive": 0.0,
103
+ "dice": 0.0,
104
+ "dice_all": 0.0,
105
+ "dice_positive": 0.0,
106
+ "f1": 0.0,
107
+ "f1_all": 0.0,
108
+ "f1_positive": 0.0,
109
+ "pixel_f1": 0.0,
110
+ "auroc": 0.0,
111
+ "pixel_auroc": 0.0,
112
+ "pixel_acc": 0.0,
113
+ "precision": 0.0,
114
+ "precision_all": 0.0,
115
+ "precision_positive": 0.0,
116
+ "recall": 0.0,
117
+ "recall_all": 0.0,
118
+ "recall_positive": 0.0,
119
+ "specificity": 0.0,
120
+ "samples": 0,
121
+ "positive_samples": 0
122
+ },
123
+ "Label 2 (Tampered / Partial Mask)": {
124
+ "iou": 0.0,
125
+ "iou_all": 0.0,
126
+ "iou_positive": 0.0,
127
+ "dice": 0.0,
128
+ "dice_all": 0.0,
129
+ "dice_positive": 0.0,
130
+ "f1": 0.0,
131
+ "f1_all": 0.0,
132
+ "f1_positive": 0.0,
133
+ "pixel_f1": 0.0,
134
+ "auroc": 0.0,
135
+ "pixel_auroc": 0.0,
136
+ "pixel_acc": 0.0,
137
+ "precision": 0.0,
138
+ "precision_all": 0.0,
139
+ "precision_positive": 0.0,
140
+ "recall": 0.0,
141
+ "recall_all": 0.0,
142
+ "recall_positive": 0.0,
143
+ "specificity": 0.0,
144
+ "samples": 0,
145
+ "positive_samples": 0
146
+ }
147
+ },
148
+ "ablation_comparison": {
149
+ "Baseline (Raw UNet)": {
150
+ "iou": 0.15,
151
+ "dice": 0.15,
152
+ "f1": 0.15,
153
+ "pixel_f1": 0.15,
154
+ "auroc": 1.0,
155
+ "pixel_auroc": 1.0,
156
+ "pixel_acc": 0.908416748046875,
157
+ "precision": 0.15,
158
+ "recall": 0.15,
159
+ "specificity": 0.908416748046875,
160
+ "iou_all": 0.15,
161
+ "dice_all": 0.15,
162
+ "f1_all": 0.15,
163
+ "positive_samples": 0,
164
+ "zero_positive_samples": 20,
165
+ "iou_positive": 0.15,
166
+ "dice_positive": 0.15,
167
+ "f1_positive": 0.15,
168
+ "precision_positive": 0.15,
169
+ "recall_positive": 0.15
170
+ },
171
+ "+ Post-Processing": {
172
+ "iou": 0.2,
173
+ "dice": 0.2,
174
+ "f1": 0.2,
175
+ "pixel_f1": 0.2,
176
+ "auroc": 1.0,
177
+ "pixel_auroc": 1.0,
178
+ "pixel_acc": 0.9088134765625,
179
+ "precision": 0.2,
180
+ "recall": 0.2,
181
+ "specificity": 0.9088134765625,
182
+ "iou_all": 0.2,
183
+ "dice_all": 0.2,
184
+ "f1_all": 0.2,
185
+ "positive_samples": 0,
186
+ "zero_positive_samples": 20,
187
+ "iou_positive": 0.2,
188
+ "dice_positive": 0.2,
189
+ "f1_positive": 0.2,
190
+ "precision_positive": 0.2,
191
+ "recall_positive": 0.2
192
+ }
193
+ },
194
+ "illustration_paths": [
195
+ "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png"
196
+ ],
197
+ "configuration": {
198
+ "project": {
199
+ "name": "eval_casia_v2.0",
200
+ "seed": 42,
201
+ "device": "auto",
202
+ "output_dir": "outputs/cross_eval_results",
203
+ "hub_repo": null,
204
+ "resume_repo": null
205
+ },
206
+ "data": {
207
+ "dataset_name": "KhangTruong/Casia-v2.0",
208
+ "streaming": false,
209
+ "image_size": [
210
+ 256,
211
+ 256
212
+ ],
213
+ "batch_size": 4,
214
+ "num_workers": 0,
215
+ "pin_memory": true,
216
+ "shuffle_buffer_size": 1000,
217
+ "train_split": "train",
218
+ "val_split": "validation",
219
+ "test_split": "cross_test",
220
+ "eval_split": "cross_test",
221
+ "train_samples_per_epoch": -1,
222
+ "val_samples_per_epoch": -1,
223
+ "val_samples": -1,
224
+ "test_samples": -1,
225
+ "evaluate_on_test": true,
226
+ "augmentations": {
227
+ "horizontal_flip": 0.5,
228
+ "vertical_flip": 0.2,
229
+ "random_rotate90": 0.5,
230
+ "color_jitter": 0.0
231
+ }
232
+ },
233
+ "model": {
234
+ "name": "unet",
235
+ "in_channels": 3,
236
+ "out_channels": 1,
237
+ "features": [
238
+ 64,
239
+ 128,
240
+ 256,
241
+ 512
242
+ ],
243
+ "bilinear": true,
244
+ "dropout": 0.0,
245
+ "aux_classifier": false,
246
+ "num_classes": 3,
247
+ "gradient_checkpointing": false
248
+ },
249
+ "loss": {
250
+ "mask_loss_type": "combined",
251
+ "bce_weight": 0.5,
252
+ "dice_weight": 0.5,
253
+ "focal_weight": 0.5,
254
+ "focal_gamma": 2.0,
255
+ "focal_alpha": 0.25,
256
+ "aux_loss_type": "cross_entropy",
257
+ "aux_weight": 0.2
258
+ },
259
+ "training": {
260
+ "epochs": 10,
261
+ "learning_rate": 0.001,
262
+ "weight_decay": 0.0001,
263
+ "optimizer": "adamw",
264
+ "use_8bit_optimizer": false,
265
+ "check_8bit_compatibility": true,
266
+ "fallback_to_16bit": true,
267
+ "fallback_on_unsupported_8bit": true,
268
+ "scheduler": "cosine",
269
+ "resume_lr_mode": "auto",
270
+ "resume_lr": null,
271
+ "warmup_epochs": 1,
272
+ "min_lr": 1e-06,
273
+ "grad_clip_norm": 1.0,
274
+ "amp": true,
275
+ "gradient_accumulation_steps": 1,
276
+ "data_parallel": true,
277
+ "use_hard_mining": false,
278
+ "auto_batch_size": true,
279
+ "empty_cache_per_epoch": true,
280
+ "save_best": true,
281
+ "save_latest": false,
282
+ "checkpoint_period": 3600,
283
+ "eval_interval": 1,
284
+ "early_stopping_patience": 5,
285
+ "early_stopping_metric": "val_iou",
286
+ "early_stopping_mode": "max",
287
+ "push_to_hub": false,
288
+ "hub_repo": null,
289
+ "hub_version": "v1"
290
+ },
291
+ "hard_mining": {
292
+ "enabled": false,
293
+ "metric": "median",
294
+ "reset_epochs": 5
295
+ },
296
+ "bootstrapping": {
297
+ "enabled": false,
298
+ "run_bootstrap": false,
299
+ "epochs": 5,
300
+ "num_samples": 512,
301
+ "batch_size": null,
302
+ "learning_rate": null,
303
+ "freeze_strategy": "channel_stream",
304
+ "stream_ratio": 0.5,
305
+ "release_mode": "restore",
306
+ "freeze_modules": [],
307
+ "initialization": "kaiming_normal",
308
+ "target_score": 0.5,
309
+ "min_loss_drop": 0.15,
310
+ "early_stopping": false,
311
+ "patience": 3
312
+ },
313
+ "logging": {
314
+ "log_interval": 20,
315
+ "log_memory": true,
316
+ "measure_network": true
317
+ },
318
+ "post_processing": {
319
+ "enabled": true,
320
+ "min_area": 64,
321
+ "fill_holes": true,
322
+ "max_hole_size": 256,
323
+ "morphology": "open_close",
324
+ "morph_kernel_size": 3
325
+ }
326
+ },
327
+ "hyperparameters": {
328
+ "Project Name": "eval_casia_v2.0",
329
+ "Dataset": "KhangTruong/Casia-v2.0",
330
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
331
+ "Streaming": false,
332
+ "Image Size": "[256, 256]",
333
+ "Batch Size": 4,
334
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
335
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
336
+ "Aux Classifier": "False (classes=3)",
337
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
338
+ "Aux Loss": "cross_entropy (wt=0.2)",
339
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
340
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
341
+ "Scheduler": "cosine",
342
+ "Epochs": 10,
343
+ "AMP": true
344
+ }
345
+ }
versions/v1/cross_evaluation/eval_casia_v2.0_report.md ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SID-UNet Evaluation Report
2
+
3
+ ### Run Configuration & Hyperparameters
4
+
5
+ | Hyperparameter / Setting | Configured Value |
6
+ |----------------------------|----------------------------------------------------------|
7
+ | Project Name | eval_casia_v2.0 |
8
+ | Dataset | KhangTruong/Casia-v2.0 |
9
+ | Splits | Train: 'train' | Val: 'validation' | Test: 'cross_test' |
10
+ | Streaming | False |
11
+ | Image Size | [256, 256] |
12
+ | Batch Size | 4 |
13
+ | Epoch Limit / Steps | Full Dataset (until depletion) |
14
+ | Model | unet (features=[64, 128, 256, 512], bilinear=True) |
15
+ | Aux Classifier | False (classes=3) |
16
+ | Mask Loss | Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5) |
17
+ | Aux Loss | cross_entropy (wt=0.2) |
18
+ | Post-Processing | Enabled (min_area=64, fill_holes=True, morph=open_close) |
19
+ | Optimizer | adamw (lr=0.001, wd=0.0001) |
20
+ | Scheduler | cosine |
21
+ | Epochs | 10 |
22
+ | AMP | True |
23
+
24
+ ### 🔬 Pipeline Ablation & Mask Refinement Comparison
25
+
26
+ | Pipeline Stage / Variant | Mean IoU | Dice / F1 | Pixel AUROC | Pixel Acc | Precision | Recall |
27
+ |----------------------------|------------|-------------|---------------|-------------|-------------|----------|
28
+ | Baseline (Raw UNet) | 0.1500 | 0.1500 | 1.0000 | 0.9084 | 0.1500 | 0.1500 |
29
+ | + Post-Processing | 0.2000 | 0.2000 | 1.0000 | 0.9088 | 0.2000 | 0.2000 |
30
+
31
+
32
+ ### Overall Segmentation & Classification Metrics
33
+
34
+ | Metric | Value |
35
+ |-------------------------|------------|
36
+ | Eval Split | cross_test |
37
+ | Eval Total Loss | 0.6689 |
38
+ | Iou | 0.2000 |
39
+ | Dice | 0.2000 |
40
+ | F1 | 0.2000 |
41
+ | Pixel F1 | 0.2000 |
42
+ | Auroc | 1.0000 |
43
+ | Pixel Auroc | 1.0000 |
44
+ | Pixel Acc | 0.9088 |
45
+ | Precision | 0.2000 |
46
+ | Recall | 0.2000 |
47
+ | Specificity | 0.9088 |
48
+ | Iou All | 0.2000 |
49
+ | Dice All | 0.2000 |
50
+ | F1 All | 0.2000 |
51
+ | Positive Samples | 0 |
52
+ | Zero Positive Samples | 20 |
53
+ | Iou Positive | 0.2000 |
54
+ | Dice Positive | 0.2000 |
55
+ | F1 Positive | 0.2000 |
56
+ | Precision Positive | 0.2000 |
57
+ | Recall Positive | 0.2000 |
58
+ | Total Evaluated Samples | 20 |
59
+
60
+ ### 🖼️ Evaluation Illustrations & Sample Predictions
61
+
62
+ **Sample Predictions Casia V2.0**
63
+
64
+ ![Sample Predictions Casia V2.0](../../../cross_evaluation/illustrations/sample_predictions_casia_v2.0.png)
65
+
66
+ ### Per-Subset Metrics Breakdown
67
+
68
+ | Subset | Mean IoU | Dice / F1 | Pixel Accuracy | Sample Count |
69
+ |-----------------------------------|------------|-------------|------------------|----------------|
70
+ | Label 0 (Real / Black Mask) | 0.2 | 0.2 | 0.9088 | 20 |
71
+ | Label 1 (Synthetic / White Mask) | 0 | 0 | 0 | 0 |
72
+ | Label 2 (Tampered / Partial Mask) | 0 | 0 | 0 | 0 |
73
+
versions/v1/cross_evaluation/illustrations/cross_eval_auroc_heatmap.png ADDED
versions/v1/cross_evaluation/illustrations/cross_eval_f1_heatmap.png ADDED
versions/v1/cross_evaluation/illustrations/cross_eval_iou_heatmap.png ADDED
versions/v1/cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png ADDED
versions/v1/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png ADDED

Git LFS Details

  • SHA256: a952f61081c70d15081e98b8c91f883fc671f0e07bde9e2eeae4c4bc3a5997b1
  • Pointer size: 132 Bytes
  • Size of remote file: 2.53 MB
versions/v1/cross_evaluation/master_cross_evaluation_report.json ADDED
@@ -0,0 +1,1352 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "total_checkpoints": 2,
3
+ "total_configs": 1,
4
+ "total_runs": 2,
5
+ "checkpoints": [
6
+ "sid_focal_hrnet",
7
+ "sid_pim_swin"
8
+ ],
9
+ "configs": [
10
+ "eval_casia_v2.0"
11
+ ],
12
+ "matrices": {
13
+ "iou": {
14
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
15
+ "sid_pim_swin:eval_casia_v2.0": 0.2
16
+ },
17
+ "dice_f1": {
18
+ "sid_focal_hrnet:eval_casia_v2.0": 0.0,
19
+ "sid_pim_swin:eval_casia_v2.0": 0.2
20
+ },
21
+ "auroc": {
22
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
23
+ "sid_pim_swin:eval_casia_v2.0": 1.0
24
+ },
25
+ "pixel_acc": {
26
+ "sid_focal_hrnet:eval_casia_v2.0": 0.006264495849609375,
27
+ "sid_pim_swin:eval_casia_v2.0": 0.9088134765625
28
+ },
29
+ "loss": {
30
+ "sid_focal_hrnet:eval_casia_v2.0": 1.415394163131714,
31
+ "sid_pim_swin:eval_casia_v2.0": 0.6688968300819397
32
+ }
33
+ },
34
+ "cross_results": [
35
+ {
36
+ "checkpoint_path": "outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt",
37
+ "checkpoint_name": "sid_focal_hrnet",
38
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
39
+ "config_stem": "casia_v2.0",
40
+ "config_name": "eval_casia_v2.0",
41
+ "dataset_name": "KhangTruong/Casia-v2.0",
42
+ "eval_split": "cross_test",
43
+ "total_evaluated_samples": 20,
44
+ "metrics": {
45
+ "eval_split": "cross_test",
46
+ "eval_total_loss": 1.415394163131714,
47
+ "iou": 0.0,
48
+ "dice": 0.0,
49
+ "f1": 0.0,
50
+ "pixel_f1": 0.0,
51
+ "auroc": 0.006264495849609375,
52
+ "pixel_auroc": 0.006264495849609375,
53
+ "pixel_acc": 0.006264495849609375,
54
+ "precision": 0.0,
55
+ "recall": 0.0,
56
+ "specificity": 0.006264495849609375,
57
+ "iou_all": 0.0,
58
+ "dice_all": 0.0,
59
+ "f1_all": 0.0,
60
+ "positive_samples": 0,
61
+ "zero_positive_samples": 20,
62
+ "iou_positive": 0.0,
63
+ "dice_positive": 0.0,
64
+ "f1_positive": 0.0,
65
+ "precision_positive": 0.0,
66
+ "recall_positive": 0.0,
67
+ "total_evaluated_samples": 20,
68
+ "ablation_summary": {
69
+ "Baseline (Raw UNet)": {
70
+ "iou": 0.0,
71
+ "dice": 0.0,
72
+ "f1": 0.0,
73
+ "pixel_f1": 0.0,
74
+ "auroc": 0.2930259943008423,
75
+ "pixel_auroc": 0.2930259943008423,
76
+ "pixel_acc": 0.00025634765625,
77
+ "precision": 0.0,
78
+ "recall": 0.0,
79
+ "specificity": 0.00025634765625,
80
+ "iou_all": 0.0,
81
+ "dice_all": 0.0,
82
+ "f1_all": 0.0,
83
+ "positive_samples": 0,
84
+ "zero_positive_samples": 20,
85
+ "iou_positive": 0.0,
86
+ "dice_positive": 0.0,
87
+ "f1_positive": 0.0,
88
+ "precision_positive": 0.0,
89
+ "recall_positive": 0.0
90
+ },
91
+ "+ Post-Processing": {
92
+ "iou": 0.0,
93
+ "dice": 0.0,
94
+ "f1": 0.0,
95
+ "pixel_f1": 0.0,
96
+ "auroc": 0.006264495849609375,
97
+ "pixel_auroc": 0.006264495849609375,
98
+ "pixel_acc": 0.006264495849609375,
99
+ "precision": 0.0,
100
+ "recall": 0.0,
101
+ "specificity": 0.006264495849609375,
102
+ "iou_all": 0.0,
103
+ "dice_all": 0.0,
104
+ "f1_all": 0.0,
105
+ "positive_samples": 0,
106
+ "zero_positive_samples": 20,
107
+ "iou_positive": 0.0,
108
+ "dice_positive": 0.0,
109
+ "f1_positive": 0.0,
110
+ "precision_positive": 0.0,
111
+ "recall_positive": 0.0
112
+ }
113
+ },
114
+ "per_dataset_metrics": {}
115
+ },
116
+ "overall_metrics": {
117
+ "eval_split": "cross_test",
118
+ "eval_total_loss": 1.415394163131714,
119
+ "iou": 0.0,
120
+ "dice": 0.0,
121
+ "f1": 0.0,
122
+ "pixel_f1": 0.0,
123
+ "auroc": 0.006264495849609375,
124
+ "pixel_auroc": 0.006264495849609375,
125
+ "pixel_acc": 0.006264495849609375,
126
+ "precision": 0.0,
127
+ "recall": 0.0,
128
+ "specificity": 0.006264495849609375,
129
+ "iou_all": 0.0,
130
+ "dice_all": 0.0,
131
+ "f1_all": 0.0,
132
+ "positive_samples": 0,
133
+ "zero_positive_samples": 20,
134
+ "iou_positive": 0.0,
135
+ "dice_positive": 0.0,
136
+ "f1_positive": 0.0,
137
+ "precision_positive": 0.0,
138
+ "recall_positive": 0.0,
139
+ "total_evaluated_samples": 20,
140
+ "ablation_summary": {
141
+ "Baseline (Raw UNet)": {
142
+ "iou": 0.0,
143
+ "dice": 0.0,
144
+ "f1": 0.0,
145
+ "pixel_f1": 0.0,
146
+ "auroc": 0.2930259943008423,
147
+ "pixel_auroc": 0.2930259943008423,
148
+ "pixel_acc": 0.00025634765625,
149
+ "precision": 0.0,
150
+ "recall": 0.0,
151
+ "specificity": 0.00025634765625,
152
+ "iou_all": 0.0,
153
+ "dice_all": 0.0,
154
+ "f1_all": 0.0,
155
+ "positive_samples": 0,
156
+ "zero_positive_samples": 20,
157
+ "iou_positive": 0.0,
158
+ "dice_positive": 0.0,
159
+ "f1_positive": 0.0,
160
+ "precision_positive": 0.0,
161
+ "recall_positive": 0.0
162
+ },
163
+ "+ Post-Processing": {
164
+ "iou": 0.0,
165
+ "dice": 0.0,
166
+ "f1": 0.0,
167
+ "pixel_f1": 0.0,
168
+ "auroc": 0.006264495849609375,
169
+ "pixel_auroc": 0.006264495849609375,
170
+ "pixel_acc": 0.006264495849609375,
171
+ "precision": 0.0,
172
+ "recall": 0.0,
173
+ "specificity": 0.006264495849609375,
174
+ "iou_all": 0.0,
175
+ "dice_all": 0.0,
176
+ "f1_all": 0.0,
177
+ "positive_samples": 0,
178
+ "zero_positive_samples": 20,
179
+ "iou_positive": 0.0,
180
+ "dice_positive": 0.0,
181
+ "f1_positive": 0.0,
182
+ "precision_positive": 0.0,
183
+ "recall_positive": 0.0
184
+ }
185
+ },
186
+ "per_dataset_metrics": {}
187
+ },
188
+ "ablation_metrics": {
189
+ "Baseline (Raw UNet)": {
190
+ "iou": 0.0,
191
+ "dice": 0.0,
192
+ "f1": 0.0,
193
+ "pixel_f1": 0.0,
194
+ "auroc": 0.2930259943008423,
195
+ "pixel_auroc": 0.2930259943008423,
196
+ "pixel_acc": 0.00025634765625,
197
+ "precision": 0.0,
198
+ "recall": 0.0,
199
+ "specificity": 0.00025634765625,
200
+ "iou_all": 0.0,
201
+ "dice_all": 0.0,
202
+ "f1_all": 0.0,
203
+ "positive_samples": 0,
204
+ "zero_positive_samples": 20,
205
+ "iou_positive": 0.0,
206
+ "dice_positive": 0.0,
207
+ "f1_positive": 0.0,
208
+ "precision_positive": 0.0,
209
+ "recall_positive": 0.0
210
+ },
211
+ "+ Post-Processing": {
212
+ "iou": 0.0,
213
+ "dice": 0.0,
214
+ "f1": 0.0,
215
+ "pixel_f1": 0.0,
216
+ "auroc": 0.006264495849609375,
217
+ "pixel_auroc": 0.006264495849609375,
218
+ "pixel_acc": 0.006264495849609375,
219
+ "precision": 0.0,
220
+ "recall": 0.0,
221
+ "specificity": 0.006264495849609375,
222
+ "iou_all": 0.0,
223
+ "dice_all": 0.0,
224
+ "f1_all": 0.0,
225
+ "positive_samples": 0,
226
+ "zero_positive_samples": 20,
227
+ "iou_positive": 0.0,
228
+ "dice_positive": 0.0,
229
+ "f1_positive": 0.0,
230
+ "precision_positive": 0.0,
231
+ "recall_positive": 0.0
232
+ }
233
+ },
234
+ "per_label_metrics": {
235
+ "0": {
236
+ "iou": 0.0,
237
+ "iou_all": 0.0,
238
+ "iou_positive": 0.0,
239
+ "dice": 0.0,
240
+ "dice_all": 0.0,
241
+ "dice_positive": 0.0,
242
+ "f1": 0.0,
243
+ "f1_all": 0.0,
244
+ "f1_positive": 0.0,
245
+ "pixel_f1": 0.0,
246
+ "auroc": 0.006264495849609375,
247
+ "pixel_auroc": 0.006264495849609375,
248
+ "pixel_acc": 0.006264495849609375,
249
+ "precision": 0.0,
250
+ "precision_all": 0.0,
251
+ "precision_positive": 0.0,
252
+ "recall": 0.0,
253
+ "recall_all": 0.0,
254
+ "recall_positive": 0.0,
255
+ "specificity": 0.006264495849609375,
256
+ "samples": 20,
257
+ "positive_samples": 0
258
+ },
259
+ "1": {
260
+ "iou": 0.0,
261
+ "iou_all": 0.0,
262
+ "iou_positive": 0.0,
263
+ "dice": 0.0,
264
+ "dice_all": 0.0,
265
+ "dice_positive": 0.0,
266
+ "f1": 0.0,
267
+ "f1_all": 0.0,
268
+ "f1_positive": 0.0,
269
+ "pixel_f1": 0.0,
270
+ "auroc": 0.0,
271
+ "pixel_auroc": 0.0,
272
+ "pixel_acc": 0.0,
273
+ "precision": 0.0,
274
+ "precision_all": 0.0,
275
+ "precision_positive": 0.0,
276
+ "recall": 0.0,
277
+ "recall_all": 0.0,
278
+ "recall_positive": 0.0,
279
+ "specificity": 0.0,
280
+ "samples": 0,
281
+ "positive_samples": 0
282
+ },
283
+ "2": {
284
+ "iou": 0.0,
285
+ "iou_all": 0.0,
286
+ "iou_positive": 0.0,
287
+ "dice": 0.0,
288
+ "dice_all": 0.0,
289
+ "dice_positive": 0.0,
290
+ "f1": 0.0,
291
+ "f1_all": 0.0,
292
+ "f1_positive": 0.0,
293
+ "pixel_f1": 0.0,
294
+ "auroc": 0.0,
295
+ "pixel_auroc": 0.0,
296
+ "pixel_acc": 0.0,
297
+ "precision": 0.0,
298
+ "precision_all": 0.0,
299
+ "precision_positive": 0.0,
300
+ "recall": 0.0,
301
+ "recall_all": 0.0,
302
+ "recall_positive": 0.0,
303
+ "specificity": 0.0,
304
+ "samples": 0,
305
+ "positive_samples": 0
306
+ }
307
+ },
308
+ "raw_per_label_metrics": {
309
+ "0": {
310
+ "iou": 0.0,
311
+ "iou_all": 0.0,
312
+ "iou_positive": 0.0,
313
+ "dice": 0.0,
314
+ "dice_all": 0.0,
315
+ "dice_positive": 0.0,
316
+ "f1": 0.0,
317
+ "f1_all": 0.0,
318
+ "f1_positive": 0.0,
319
+ "pixel_f1": 0.0,
320
+ "auroc": 0.2930259943008423,
321
+ "pixel_auroc": 0.2930259943008423,
322
+ "pixel_acc": 0.00025634765625,
323
+ "precision": 0.0,
324
+ "precision_all": 0.0,
325
+ "precision_positive": 0.0,
326
+ "recall": 0.0,
327
+ "recall_all": 0.0,
328
+ "recall_positive": 0.0,
329
+ "specificity": 0.00025634765625,
330
+ "samples": 20,
331
+ "positive_samples": 0
332
+ },
333
+ "1": {
334
+ "iou": 0.0,
335
+ "iou_all": 0.0,
336
+ "iou_positive": 0.0,
337
+ "dice": 0.0,
338
+ "dice_all": 0.0,
339
+ "dice_positive": 0.0,
340
+ "f1": 0.0,
341
+ "f1_all": 0.0,
342
+ "f1_positive": 0.0,
343
+ "pixel_f1": 0.0,
344
+ "auroc": 0.0,
345
+ "pixel_auroc": 0.0,
346
+ "pixel_acc": 0.0,
347
+ "precision": 0.0,
348
+ "precision_all": 0.0,
349
+ "precision_positive": 0.0,
350
+ "recall": 0.0,
351
+ "recall_all": 0.0,
352
+ "recall_positive": 0.0,
353
+ "specificity": 0.0,
354
+ "samples": 0,
355
+ "positive_samples": 0
356
+ },
357
+ "2": {
358
+ "iou": 0.0,
359
+ "iou_all": 0.0,
360
+ "iou_positive": 0.0,
361
+ "dice": 0.0,
362
+ "dice_all": 0.0,
363
+ "dice_positive": 0.0,
364
+ "f1": 0.0,
365
+ "f1_all": 0.0,
366
+ "f1_positive": 0.0,
367
+ "pixel_f1": 0.0,
368
+ "auroc": 0.0,
369
+ "pixel_auroc": 0.0,
370
+ "pixel_acc": 0.0,
371
+ "precision": 0.0,
372
+ "precision_all": 0.0,
373
+ "precision_positive": 0.0,
374
+ "recall": 0.0,
375
+ "recall_all": 0.0,
376
+ "recall_positive": 0.0,
377
+ "specificity": 0.0,
378
+ "samples": 0,
379
+ "positive_samples": 0
380
+ }
381
+ },
382
+ "per_dataset_metrics": {},
383
+ "raw_per_dataset_metrics": {},
384
+ "confusion_matrix": null,
385
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
386
+ "checkpoint_config": {
387
+ "project": {
388
+ "name": "sid_focal_hrnet",
389
+ "seed": 42,
390
+ "device": "auto",
391
+ "output_dir": "outputs/RUN/focal_hrnet",
392
+ "hub_repo": null,
393
+ "resume_repo": null
394
+ },
395
+ "data": {
396
+ "dataset_name": "KhangTruong/IMD2020",
397
+ "streaming": false,
398
+ "image_size": [
399
+ 256,
400
+ 256
401
+ ],
402
+ "batch_size": 16,
403
+ "num_workers": -1,
404
+ "pin_memory": true,
405
+ "shuffle_buffer_size": 1000,
406
+ "train_split": "train",
407
+ "val_split": "validation",
408
+ "test_split": "test",
409
+ "eval_split": "test",
410
+ "train_samples_per_epoch": -1,
411
+ "val_samples_per_epoch": -1,
412
+ "val_samples": -1,
413
+ "test_samples": -1,
414
+ "evaluate_on_test": true,
415
+ "augmentations": {
416
+ "horizontal_flip": 0.5,
417
+ "vertical_flip": 0.2,
418
+ "random_rotate90": 0.5,
419
+ "color_jitter": 0.0
420
+ }
421
+ },
422
+ "model": {
423
+ "name": "focal",
424
+ "in_channels": 3,
425
+ "out_channels": 1,
426
+ "features": [
427
+ 64,
428
+ 128,
429
+ 256,
430
+ 512
431
+ ],
432
+ "bilinear": true,
433
+ "dropout": 0.1,
434
+ "aux_classifier": false,
435
+ "num_classes": 2,
436
+ "gradient_checkpointing": false,
437
+ "backbone": "hrnet",
438
+ "pretrained": true,
439
+ "hf_repo": "KhangTruong/FOCAL"
440
+ },
441
+ "loss": {
442
+ "mask_loss_type": "combined",
443
+ "bce_weight": 0.5,
444
+ "dice_weight": 0.5,
445
+ "focal_weight": 0.5,
446
+ "focal_gamma": 2.0,
447
+ "focal_alpha": 0.25,
448
+ "aux_loss_type": "cross_entropy",
449
+ "aux_weight": 0.2
450
+ },
451
+ "training": {
452
+ "epochs": 10,
453
+ "learning_rate": 0.0001,
454
+ "weight_decay": 0.0001,
455
+ "optimizer": "adamw",
456
+ "use_8bit_optimizer": false,
457
+ "check_8bit_compatibility": true,
458
+ "fallback_to_16bit": true,
459
+ "fallback_on_unsupported_8bit": true,
460
+ "scheduler": "cosine",
461
+ "resume_lr_mode": "auto",
462
+ "resume_lr": null,
463
+ "warmup_epochs": 1,
464
+ "min_lr": 1e-06,
465
+ "grad_clip_norm": 1.0,
466
+ "amp": true,
467
+ "gradient_accumulation_steps": 1,
468
+ "data_parallel": true,
469
+ "use_hard_mining": false,
470
+ "auto_batch_size": true,
471
+ "empty_cache_per_epoch": true,
472
+ "save_best": true,
473
+ "save_latest": false,
474
+ "checkpoint_period": 3600,
475
+ "eval_interval": 1,
476
+ "early_stopping_patience": 5,
477
+ "early_stopping_metric": "val_iou",
478
+ "early_stopping_mode": "max",
479
+ "push_to_hub": false,
480
+ "hub_repo": null,
481
+ "hub_version": "v1"
482
+ },
483
+ "hard_mining": {
484
+ "enabled": false,
485
+ "metric": "median",
486
+ "reset_epochs": 5
487
+ },
488
+ "bootstrapping": {
489
+ "enabled": false,
490
+ "run_bootstrap": false,
491
+ "epochs": 5,
492
+ "num_samples": 512,
493
+ "batch_size": null,
494
+ "learning_rate": null,
495
+ "freeze_strategy": "channel_stream",
496
+ "stream_ratio": 0.5,
497
+ "release_mode": "restore",
498
+ "freeze_modules": [],
499
+ "initialization": "kaiming_normal",
500
+ "target_score": 0.5,
501
+ "min_loss_drop": 0.15,
502
+ "early_stopping": false,
503
+ "patience": 3
504
+ },
505
+ "logging": {
506
+ "log_interval": 20,
507
+ "log_memory": true,
508
+ "measure_network": true
509
+ },
510
+ "post_processing": {
511
+ "enabled": true,
512
+ "min_area": 64,
513
+ "fill_holes": true,
514
+ "max_hole_size": 256,
515
+ "morphology": "open_close",
516
+ "morph_kernel_size": 3
517
+ }
518
+ },
519
+ "eval_config": {
520
+ "project": {
521
+ "name": "eval_casia_v2.0",
522
+ "seed": 42,
523
+ "device": "auto",
524
+ "output_dir": "outputs/cross_eval_results",
525
+ "hub_repo": null,
526
+ "resume_repo": null
527
+ },
528
+ "data": {
529
+ "dataset_name": "KhangTruong/Casia-v2.0",
530
+ "streaming": false,
531
+ "image_size": [
532
+ 256,
533
+ 256
534
+ ],
535
+ "batch_size": 4,
536
+ "num_workers": 0,
537
+ "pin_memory": true,
538
+ "shuffle_buffer_size": 1000,
539
+ "train_split": "train",
540
+ "val_split": "validation",
541
+ "test_split": "cross_test",
542
+ "eval_split": "cross_test",
543
+ "train_samples_per_epoch": -1,
544
+ "val_samples_per_epoch": -1,
545
+ "val_samples": -1,
546
+ "test_samples": -1,
547
+ "evaluate_on_test": true,
548
+ "augmentations": {
549
+ "horizontal_flip": 0.5,
550
+ "vertical_flip": 0.2,
551
+ "random_rotate90": 0.5,
552
+ "color_jitter": 0.0
553
+ }
554
+ },
555
+ "model": {
556
+ "name": "unet",
557
+ "in_channels": 3,
558
+ "out_channels": 1,
559
+ "features": [
560
+ 64,
561
+ 128,
562
+ 256,
563
+ 512
564
+ ],
565
+ "bilinear": true,
566
+ "dropout": 0.0,
567
+ "aux_classifier": false,
568
+ "num_classes": 3,
569
+ "gradient_checkpointing": false
570
+ },
571
+ "loss": {
572
+ "mask_loss_type": "combined",
573
+ "bce_weight": 0.5,
574
+ "dice_weight": 0.5,
575
+ "focal_weight": 0.5,
576
+ "focal_gamma": 2.0,
577
+ "focal_alpha": 0.25,
578
+ "aux_loss_type": "cross_entropy",
579
+ "aux_weight": 0.2
580
+ },
581
+ "training": {
582
+ "epochs": 10,
583
+ "learning_rate": 0.001,
584
+ "weight_decay": 0.0001,
585
+ "optimizer": "adamw",
586
+ "use_8bit_optimizer": false,
587
+ "check_8bit_compatibility": true,
588
+ "fallback_to_16bit": true,
589
+ "fallback_on_unsupported_8bit": true,
590
+ "scheduler": "cosine",
591
+ "resume_lr_mode": "auto",
592
+ "resume_lr": null,
593
+ "warmup_epochs": 1,
594
+ "min_lr": 1e-06,
595
+ "grad_clip_norm": 1.0,
596
+ "amp": true,
597
+ "gradient_accumulation_steps": 1,
598
+ "data_parallel": true,
599
+ "use_hard_mining": false,
600
+ "auto_batch_size": true,
601
+ "empty_cache_per_epoch": true,
602
+ "save_best": true,
603
+ "save_latest": false,
604
+ "checkpoint_period": 3600,
605
+ "eval_interval": 1,
606
+ "early_stopping_patience": 5,
607
+ "early_stopping_metric": "val_iou",
608
+ "early_stopping_mode": "max",
609
+ "push_to_hub": false,
610
+ "hub_repo": null,
611
+ "hub_version": "v1"
612
+ },
613
+ "hard_mining": {
614
+ "enabled": false,
615
+ "metric": "median",
616
+ "reset_epochs": 5
617
+ },
618
+ "bootstrapping": {
619
+ "enabled": false,
620
+ "run_bootstrap": false,
621
+ "epochs": 5,
622
+ "num_samples": 512,
623
+ "batch_size": null,
624
+ "learning_rate": null,
625
+ "freeze_strategy": "channel_stream",
626
+ "stream_ratio": 0.5,
627
+ "release_mode": "restore",
628
+ "freeze_modules": [],
629
+ "initialization": "kaiming_normal",
630
+ "target_score": 0.5,
631
+ "min_loss_drop": 0.15,
632
+ "early_stopping": false,
633
+ "patience": 3
634
+ },
635
+ "logging": {
636
+ "log_interval": 20,
637
+ "log_memory": true,
638
+ "measure_network": true
639
+ },
640
+ "post_processing": {
641
+ "enabled": true,
642
+ "min_area": 64,
643
+ "fill_holes": true,
644
+ "max_hole_size": 256,
645
+ "morphology": "open_close",
646
+ "morph_kernel_size": 3
647
+ }
648
+ },
649
+ "checkpoint_hyperparameters": {
650
+ "Project Name": "sid_focal_hrnet",
651
+ "Dataset": "KhangTruong/IMD2020",
652
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
653
+ "Streaming": false,
654
+ "Image Size": "[256, 256]",
655
+ "Batch Size": 16,
656
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
657
+ "Model": "focal (features=[64, 128, 256, 512], bilinear=True)",
658
+ "Aux Classifier": "False (classes=2)",
659
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
660
+ "Aux Loss": "cross_entropy (wt=0.2)",
661
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
662
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
663
+ "Scheduler": "cosine",
664
+ "Epochs": 10,
665
+ "AMP": true
666
+ },
667
+ "eval_hyperparameters": {
668
+ "Project Name": "eval_casia_v2.0",
669
+ "Dataset": "KhangTruong/Casia-v2.0",
670
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
671
+ "Streaming": false,
672
+ "Image Size": "[256, 256]",
673
+ "Batch Size": 4,
674
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
675
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
676
+ "Aux Classifier": "False (classes=3)",
677
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
678
+ "Aux Loss": "cross_entropy (wt=0.2)",
679
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
680
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
681
+ "Scheduler": "cosine",
682
+ "Epochs": 10,
683
+ "AMP": true
684
+ },
685
+ "devices": [
686
+ "cuda"
687
+ ],
688
+ "gpu_utilization": {}
689
+ },
690
+ {
691
+ "checkpoint_path": "outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt",
692
+ "checkpoint_name": "sid_pim_swin",
693
+ "config_path": "configs/cross-eval/casia_v2.0.yaml",
694
+ "config_stem": "casia_v2.0",
695
+ "config_name": "eval_casia_v2.0",
696
+ "dataset_name": "KhangTruong/Casia-v2.0",
697
+ "eval_split": "cross_test",
698
+ "total_evaluated_samples": 20,
699
+ "metrics": {
700
+ "eval_split": "cross_test",
701
+ "eval_total_loss": 0.6688968300819397,
702
+ "iou": 0.2,
703
+ "dice": 0.2,
704
+ "f1": 0.2,
705
+ "pixel_f1": 0.2,
706
+ "auroc": 1.0,
707
+ "pixel_auroc": 1.0,
708
+ "pixel_acc": 0.9088134765625,
709
+ "precision": 0.2,
710
+ "recall": 0.2,
711
+ "specificity": 0.9088134765625,
712
+ "iou_all": 0.2,
713
+ "dice_all": 0.2,
714
+ "f1_all": 0.2,
715
+ "positive_samples": 0,
716
+ "zero_positive_samples": 20,
717
+ "iou_positive": 0.2,
718
+ "dice_positive": 0.2,
719
+ "f1_positive": 0.2,
720
+ "precision_positive": 0.2,
721
+ "recall_positive": 0.2,
722
+ "total_evaluated_samples": 20,
723
+ "ablation_summary": {
724
+ "Baseline (Raw UNet)": {
725
+ "iou": 0.15,
726
+ "dice": 0.15,
727
+ "f1": 0.15,
728
+ "pixel_f1": 0.15,
729
+ "auroc": 1.0,
730
+ "pixel_auroc": 1.0,
731
+ "pixel_acc": 0.908416748046875,
732
+ "precision": 0.15,
733
+ "recall": 0.15,
734
+ "specificity": 0.908416748046875,
735
+ "iou_all": 0.15,
736
+ "dice_all": 0.15,
737
+ "f1_all": 0.15,
738
+ "positive_samples": 0,
739
+ "zero_positive_samples": 20,
740
+ "iou_positive": 0.15,
741
+ "dice_positive": 0.15,
742
+ "f1_positive": 0.15,
743
+ "precision_positive": 0.15,
744
+ "recall_positive": 0.15
745
+ },
746
+ "+ Post-Processing": {
747
+ "iou": 0.2,
748
+ "dice": 0.2,
749
+ "f1": 0.2,
750
+ "pixel_f1": 0.2,
751
+ "auroc": 1.0,
752
+ "pixel_auroc": 1.0,
753
+ "pixel_acc": 0.9088134765625,
754
+ "precision": 0.2,
755
+ "recall": 0.2,
756
+ "specificity": 0.9088134765625,
757
+ "iou_all": 0.2,
758
+ "dice_all": 0.2,
759
+ "f1_all": 0.2,
760
+ "positive_samples": 0,
761
+ "zero_positive_samples": 20,
762
+ "iou_positive": 0.2,
763
+ "dice_positive": 0.2,
764
+ "f1_positive": 0.2,
765
+ "precision_positive": 0.2,
766
+ "recall_positive": 0.2
767
+ }
768
+ },
769
+ "per_dataset_metrics": {}
770
+ },
771
+ "overall_metrics": {
772
+ "eval_split": "cross_test",
773
+ "eval_total_loss": 0.6688968300819397,
774
+ "iou": 0.2,
775
+ "dice": 0.2,
776
+ "f1": 0.2,
777
+ "pixel_f1": 0.2,
778
+ "auroc": 1.0,
779
+ "pixel_auroc": 1.0,
780
+ "pixel_acc": 0.9088134765625,
781
+ "precision": 0.2,
782
+ "recall": 0.2,
783
+ "specificity": 0.9088134765625,
784
+ "iou_all": 0.2,
785
+ "dice_all": 0.2,
786
+ "f1_all": 0.2,
787
+ "positive_samples": 0,
788
+ "zero_positive_samples": 20,
789
+ "iou_positive": 0.2,
790
+ "dice_positive": 0.2,
791
+ "f1_positive": 0.2,
792
+ "precision_positive": 0.2,
793
+ "recall_positive": 0.2,
794
+ "total_evaluated_samples": 20,
795
+ "ablation_summary": {
796
+ "Baseline (Raw UNet)": {
797
+ "iou": 0.15,
798
+ "dice": 0.15,
799
+ "f1": 0.15,
800
+ "pixel_f1": 0.15,
801
+ "auroc": 1.0,
802
+ "pixel_auroc": 1.0,
803
+ "pixel_acc": 0.908416748046875,
804
+ "precision": 0.15,
805
+ "recall": 0.15,
806
+ "specificity": 0.908416748046875,
807
+ "iou_all": 0.15,
808
+ "dice_all": 0.15,
809
+ "f1_all": 0.15,
810
+ "positive_samples": 0,
811
+ "zero_positive_samples": 20,
812
+ "iou_positive": 0.15,
813
+ "dice_positive": 0.15,
814
+ "f1_positive": 0.15,
815
+ "precision_positive": 0.15,
816
+ "recall_positive": 0.15
817
+ },
818
+ "+ Post-Processing": {
819
+ "iou": 0.2,
820
+ "dice": 0.2,
821
+ "f1": 0.2,
822
+ "pixel_f1": 0.2,
823
+ "auroc": 1.0,
824
+ "pixel_auroc": 1.0,
825
+ "pixel_acc": 0.9088134765625,
826
+ "precision": 0.2,
827
+ "recall": 0.2,
828
+ "specificity": 0.9088134765625,
829
+ "iou_all": 0.2,
830
+ "dice_all": 0.2,
831
+ "f1_all": 0.2,
832
+ "positive_samples": 0,
833
+ "zero_positive_samples": 20,
834
+ "iou_positive": 0.2,
835
+ "dice_positive": 0.2,
836
+ "f1_positive": 0.2,
837
+ "precision_positive": 0.2,
838
+ "recall_positive": 0.2
839
+ }
840
+ },
841
+ "per_dataset_metrics": {}
842
+ },
843
+ "ablation_metrics": {
844
+ "Baseline (Raw UNet)": {
845
+ "iou": 0.15,
846
+ "dice": 0.15,
847
+ "f1": 0.15,
848
+ "pixel_f1": 0.15,
849
+ "auroc": 1.0,
850
+ "pixel_auroc": 1.0,
851
+ "pixel_acc": 0.908416748046875,
852
+ "precision": 0.15,
853
+ "recall": 0.15,
854
+ "specificity": 0.908416748046875,
855
+ "iou_all": 0.15,
856
+ "dice_all": 0.15,
857
+ "f1_all": 0.15,
858
+ "positive_samples": 0,
859
+ "zero_positive_samples": 20,
860
+ "iou_positive": 0.15,
861
+ "dice_positive": 0.15,
862
+ "f1_positive": 0.15,
863
+ "precision_positive": 0.15,
864
+ "recall_positive": 0.15
865
+ },
866
+ "+ Post-Processing": {
867
+ "iou": 0.2,
868
+ "dice": 0.2,
869
+ "f1": 0.2,
870
+ "pixel_f1": 0.2,
871
+ "auroc": 1.0,
872
+ "pixel_auroc": 1.0,
873
+ "pixel_acc": 0.9088134765625,
874
+ "precision": 0.2,
875
+ "recall": 0.2,
876
+ "specificity": 0.9088134765625,
877
+ "iou_all": 0.2,
878
+ "dice_all": 0.2,
879
+ "f1_all": 0.2,
880
+ "positive_samples": 0,
881
+ "zero_positive_samples": 20,
882
+ "iou_positive": 0.2,
883
+ "dice_positive": 0.2,
884
+ "f1_positive": 0.2,
885
+ "precision_positive": 0.2,
886
+ "recall_positive": 0.2
887
+ }
888
+ },
889
+ "per_label_metrics": {
890
+ "0": {
891
+ "iou": 0.2,
892
+ "iou_all": 0.2,
893
+ "iou_positive": 0.2,
894
+ "dice": 0.2,
895
+ "dice_all": 0.2,
896
+ "dice_positive": 0.2,
897
+ "f1": 0.2,
898
+ "f1_all": 0.2,
899
+ "f1_positive": 0.2,
900
+ "pixel_f1": 0.2,
901
+ "auroc": 1.0,
902
+ "pixel_auroc": 1.0,
903
+ "pixel_acc": 0.9088134765625,
904
+ "precision": 0.2,
905
+ "precision_all": 0.2,
906
+ "precision_positive": 0.2,
907
+ "recall": 0.2,
908
+ "recall_all": 0.2,
909
+ "recall_positive": 0.2,
910
+ "specificity": 0.9088134765625,
911
+ "samples": 20,
912
+ "positive_samples": 0
913
+ },
914
+ "1": {
915
+ "iou": 0.0,
916
+ "iou_all": 0.0,
917
+ "iou_positive": 0.0,
918
+ "dice": 0.0,
919
+ "dice_all": 0.0,
920
+ "dice_positive": 0.0,
921
+ "f1": 0.0,
922
+ "f1_all": 0.0,
923
+ "f1_positive": 0.0,
924
+ "pixel_f1": 0.0,
925
+ "auroc": 0.0,
926
+ "pixel_auroc": 0.0,
927
+ "pixel_acc": 0.0,
928
+ "precision": 0.0,
929
+ "precision_all": 0.0,
930
+ "precision_positive": 0.0,
931
+ "recall": 0.0,
932
+ "recall_all": 0.0,
933
+ "recall_positive": 0.0,
934
+ "specificity": 0.0,
935
+ "samples": 0,
936
+ "positive_samples": 0
937
+ },
938
+ "2": {
939
+ "iou": 0.0,
940
+ "iou_all": 0.0,
941
+ "iou_positive": 0.0,
942
+ "dice": 0.0,
943
+ "dice_all": 0.0,
944
+ "dice_positive": 0.0,
945
+ "f1": 0.0,
946
+ "f1_all": 0.0,
947
+ "f1_positive": 0.0,
948
+ "pixel_f1": 0.0,
949
+ "auroc": 0.0,
950
+ "pixel_auroc": 0.0,
951
+ "pixel_acc": 0.0,
952
+ "precision": 0.0,
953
+ "precision_all": 0.0,
954
+ "precision_positive": 0.0,
955
+ "recall": 0.0,
956
+ "recall_all": 0.0,
957
+ "recall_positive": 0.0,
958
+ "specificity": 0.0,
959
+ "samples": 0,
960
+ "positive_samples": 0
961
+ }
962
+ },
963
+ "raw_per_label_metrics": {
964
+ "0": {
965
+ "iou": 0.15,
966
+ "iou_all": 0.15,
967
+ "iou_positive": 0.15,
968
+ "dice": 0.15,
969
+ "dice_all": 0.15,
970
+ "dice_positive": 0.15,
971
+ "f1": 0.15,
972
+ "f1_all": 0.15,
973
+ "f1_positive": 0.15,
974
+ "pixel_f1": 0.15,
975
+ "auroc": 1.0,
976
+ "pixel_auroc": 1.0,
977
+ "pixel_acc": 0.908416748046875,
978
+ "precision": 0.15,
979
+ "precision_all": 0.15,
980
+ "precision_positive": 0.15,
981
+ "recall": 0.15,
982
+ "recall_all": 0.15,
983
+ "recall_positive": 0.15,
984
+ "specificity": 0.908416748046875,
985
+ "samples": 20,
986
+ "positive_samples": 0
987
+ },
988
+ "1": {
989
+ "iou": 0.0,
990
+ "iou_all": 0.0,
991
+ "iou_positive": 0.0,
992
+ "dice": 0.0,
993
+ "dice_all": 0.0,
994
+ "dice_positive": 0.0,
995
+ "f1": 0.0,
996
+ "f1_all": 0.0,
997
+ "f1_positive": 0.0,
998
+ "pixel_f1": 0.0,
999
+ "auroc": 0.0,
1000
+ "pixel_auroc": 0.0,
1001
+ "pixel_acc": 0.0,
1002
+ "precision": 0.0,
1003
+ "precision_all": 0.0,
1004
+ "precision_positive": 0.0,
1005
+ "recall": 0.0,
1006
+ "recall_all": 0.0,
1007
+ "recall_positive": 0.0,
1008
+ "specificity": 0.0,
1009
+ "samples": 0,
1010
+ "positive_samples": 0
1011
+ },
1012
+ "2": {
1013
+ "iou": 0.0,
1014
+ "iou_all": 0.0,
1015
+ "iou_positive": 0.0,
1016
+ "dice": 0.0,
1017
+ "dice_all": 0.0,
1018
+ "dice_positive": 0.0,
1019
+ "f1": 0.0,
1020
+ "f1_all": 0.0,
1021
+ "f1_positive": 0.0,
1022
+ "pixel_f1": 0.0,
1023
+ "auroc": 0.0,
1024
+ "pixel_auroc": 0.0,
1025
+ "pixel_acc": 0.0,
1026
+ "precision": 0.0,
1027
+ "precision_all": 0.0,
1028
+ "precision_positive": 0.0,
1029
+ "recall": 0.0,
1030
+ "recall_all": 0.0,
1031
+ "recall_positive": 0.0,
1032
+ "specificity": 0.0,
1033
+ "samples": 0,
1034
+ "positive_samples": 0
1035
+ }
1036
+ },
1037
+ "per_dataset_metrics": {},
1038
+ "raw_per_dataset_metrics": {},
1039
+ "confusion_matrix": null,
1040
+ "dataset_illustration_path": "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png",
1041
+ "checkpoint_config": {
1042
+ "project": {
1043
+ "name": "sid_pim_swin",
1044
+ "seed": 42,
1045
+ "device": "auto",
1046
+ "output_dir": "outputs/RUN/pim_swin",
1047
+ "hub_repo": null,
1048
+ "resume_repo": null
1049
+ },
1050
+ "data": {
1051
+ "dataset_name": "KhangTruong/IMD2020",
1052
+ "streaming": false,
1053
+ "image_size": [
1054
+ 256,
1055
+ 256
1056
+ ],
1057
+ "batch_size": 8,
1058
+ "num_workers": -1,
1059
+ "pin_memory": true,
1060
+ "shuffle_buffer_size": 1000,
1061
+ "train_split": "train",
1062
+ "val_split": "validation",
1063
+ "test_split": "test",
1064
+ "eval_split": "test",
1065
+ "train_samples_per_epoch": -1,
1066
+ "val_samples_per_epoch": -1,
1067
+ "val_samples": -1,
1068
+ "test_samples": -1,
1069
+ "evaluate_on_test": true,
1070
+ "augmentations": {
1071
+ "horizontal_flip": 0.5,
1072
+ "vertical_flip": 0.2,
1073
+ "random_rotate90": 0.5,
1074
+ "color_jitter": 0.0
1075
+ }
1076
+ },
1077
+ "model": {
1078
+ "name": "pim",
1079
+ "in_channels": 3,
1080
+ "out_channels": 1,
1081
+ "features": [
1082
+ 64,
1083
+ 128,
1084
+ 256,
1085
+ 512
1086
+ ],
1087
+ "bilinear": true,
1088
+ "dropout": 0.1,
1089
+ "aux_classifier": false,
1090
+ "num_classes": 2,
1091
+ "gradient_checkpointing": false,
1092
+ "pretrained": true,
1093
+ "hf_repo": "KhangTruong/PIM"
1094
+ },
1095
+ "loss": {
1096
+ "mask_loss_type": "combined",
1097
+ "bce_weight": 0.5,
1098
+ "dice_weight": 0.5,
1099
+ "focal_weight": 0.5,
1100
+ "focal_gamma": 2.0,
1101
+ "focal_alpha": 0.25,
1102
+ "aux_loss_type": "cross_entropy",
1103
+ "aux_weight": 0.2
1104
+ },
1105
+ "training": {
1106
+ "epochs": 10,
1107
+ "learning_rate": 0.0001,
1108
+ "weight_decay": 0.0001,
1109
+ "optimizer": "adamw",
1110
+ "use_8bit_optimizer": false,
1111
+ "check_8bit_compatibility": true,
1112
+ "fallback_to_16bit": true,
1113
+ "fallback_on_unsupported_8bit": true,
1114
+ "scheduler": "cosine",
1115
+ "resume_lr_mode": "auto",
1116
+ "resume_lr": null,
1117
+ "warmup_epochs": 1,
1118
+ "min_lr": 1e-06,
1119
+ "grad_clip_norm": 1.0,
1120
+ "amp": true,
1121
+ "gradient_accumulation_steps": 1,
1122
+ "data_parallel": true,
1123
+ "use_hard_mining": false,
1124
+ "auto_batch_size": true,
1125
+ "empty_cache_per_epoch": true,
1126
+ "save_best": true,
1127
+ "save_latest": false,
1128
+ "checkpoint_period": 3600,
1129
+ "eval_interval": 1,
1130
+ "early_stopping_patience": 5,
1131
+ "early_stopping_metric": "val_iou",
1132
+ "early_stopping_mode": "max",
1133
+ "push_to_hub": false,
1134
+ "hub_repo": null,
1135
+ "hub_version": "v1"
1136
+ },
1137
+ "hard_mining": {
1138
+ "enabled": false,
1139
+ "metric": "median",
1140
+ "reset_epochs": 5
1141
+ },
1142
+ "bootstrapping": {
1143
+ "enabled": false,
1144
+ "run_bootstrap": false,
1145
+ "epochs": 5,
1146
+ "num_samples": 512,
1147
+ "batch_size": null,
1148
+ "learning_rate": null,
1149
+ "freeze_strategy": "channel_stream",
1150
+ "stream_ratio": 0.5,
1151
+ "release_mode": "restore",
1152
+ "freeze_modules": [],
1153
+ "initialization": "kaiming_normal",
1154
+ "target_score": 0.5,
1155
+ "min_loss_drop": 0.15,
1156
+ "early_stopping": false,
1157
+ "patience": 3
1158
+ },
1159
+ "logging": {
1160
+ "log_interval": 20,
1161
+ "log_memory": true,
1162
+ "measure_network": true
1163
+ },
1164
+ "post_processing": {
1165
+ "enabled": true,
1166
+ "min_area": 64,
1167
+ "fill_holes": true,
1168
+ "max_hole_size": 256,
1169
+ "morphology": "open_close",
1170
+ "morph_kernel_size": 3
1171
+ }
1172
+ },
1173
+ "eval_config": {
1174
+ "project": {
1175
+ "name": "eval_casia_v2.0",
1176
+ "seed": 42,
1177
+ "device": "auto",
1178
+ "output_dir": "outputs/cross_eval_results",
1179
+ "hub_repo": null,
1180
+ "resume_repo": null
1181
+ },
1182
+ "data": {
1183
+ "dataset_name": "KhangTruong/Casia-v2.0",
1184
+ "streaming": false,
1185
+ "image_size": [
1186
+ 256,
1187
+ 256
1188
+ ],
1189
+ "batch_size": 4,
1190
+ "num_workers": 0,
1191
+ "pin_memory": true,
1192
+ "shuffle_buffer_size": 1000,
1193
+ "train_split": "train",
1194
+ "val_split": "validation",
1195
+ "test_split": "cross_test",
1196
+ "eval_split": "cross_test",
1197
+ "train_samples_per_epoch": -1,
1198
+ "val_samples_per_epoch": -1,
1199
+ "val_samples": -1,
1200
+ "test_samples": -1,
1201
+ "evaluate_on_test": true,
1202
+ "augmentations": {
1203
+ "horizontal_flip": 0.5,
1204
+ "vertical_flip": 0.2,
1205
+ "random_rotate90": 0.5,
1206
+ "color_jitter": 0.0
1207
+ }
1208
+ },
1209
+ "model": {
1210
+ "name": "unet",
1211
+ "in_channels": 3,
1212
+ "out_channels": 1,
1213
+ "features": [
1214
+ 64,
1215
+ 128,
1216
+ 256,
1217
+ 512
1218
+ ],
1219
+ "bilinear": true,
1220
+ "dropout": 0.0,
1221
+ "aux_classifier": false,
1222
+ "num_classes": 3,
1223
+ "gradient_checkpointing": false
1224
+ },
1225
+ "loss": {
1226
+ "mask_loss_type": "combined",
1227
+ "bce_weight": 0.5,
1228
+ "dice_weight": 0.5,
1229
+ "focal_weight": 0.5,
1230
+ "focal_gamma": 2.0,
1231
+ "focal_alpha": 0.25,
1232
+ "aux_loss_type": "cross_entropy",
1233
+ "aux_weight": 0.2
1234
+ },
1235
+ "training": {
1236
+ "epochs": 10,
1237
+ "learning_rate": 0.001,
1238
+ "weight_decay": 0.0001,
1239
+ "optimizer": "adamw",
1240
+ "use_8bit_optimizer": false,
1241
+ "check_8bit_compatibility": true,
1242
+ "fallback_to_16bit": true,
1243
+ "fallback_on_unsupported_8bit": true,
1244
+ "scheduler": "cosine",
1245
+ "resume_lr_mode": "auto",
1246
+ "resume_lr": null,
1247
+ "warmup_epochs": 1,
1248
+ "min_lr": 1e-06,
1249
+ "grad_clip_norm": 1.0,
1250
+ "amp": true,
1251
+ "gradient_accumulation_steps": 1,
1252
+ "data_parallel": true,
1253
+ "use_hard_mining": false,
1254
+ "auto_batch_size": true,
1255
+ "empty_cache_per_epoch": true,
1256
+ "save_best": true,
1257
+ "save_latest": false,
1258
+ "checkpoint_period": 3600,
1259
+ "eval_interval": 1,
1260
+ "early_stopping_patience": 5,
1261
+ "early_stopping_metric": "val_iou",
1262
+ "early_stopping_mode": "max",
1263
+ "push_to_hub": false,
1264
+ "hub_repo": null,
1265
+ "hub_version": "v1"
1266
+ },
1267
+ "hard_mining": {
1268
+ "enabled": false,
1269
+ "metric": "median",
1270
+ "reset_epochs": 5
1271
+ },
1272
+ "bootstrapping": {
1273
+ "enabled": false,
1274
+ "run_bootstrap": false,
1275
+ "epochs": 5,
1276
+ "num_samples": 512,
1277
+ "batch_size": null,
1278
+ "learning_rate": null,
1279
+ "freeze_strategy": "channel_stream",
1280
+ "stream_ratio": 0.5,
1281
+ "release_mode": "restore",
1282
+ "freeze_modules": [],
1283
+ "initialization": "kaiming_normal",
1284
+ "target_score": 0.5,
1285
+ "min_loss_drop": 0.15,
1286
+ "early_stopping": false,
1287
+ "patience": 3
1288
+ },
1289
+ "logging": {
1290
+ "log_interval": 20,
1291
+ "log_memory": true,
1292
+ "measure_network": true
1293
+ },
1294
+ "post_processing": {
1295
+ "enabled": true,
1296
+ "min_area": 64,
1297
+ "fill_holes": true,
1298
+ "max_hole_size": 256,
1299
+ "morphology": "open_close",
1300
+ "morph_kernel_size": 3
1301
+ }
1302
+ },
1303
+ "checkpoint_hyperparameters": {
1304
+ "Project Name": "sid_pim_swin",
1305
+ "Dataset": "KhangTruong/IMD2020",
1306
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'test'",
1307
+ "Streaming": false,
1308
+ "Image Size": "[256, 256]",
1309
+ "Batch Size": 8,
1310
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
1311
+ "Model": "pim (features=[64, 128, 256, 512], bilinear=True)",
1312
+ "Aux Classifier": "False (classes=2)",
1313
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
1314
+ "Aux Loss": "cross_entropy (wt=0.2)",
1315
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
1316
+ "Optimizer": "adamw (lr=0.0001, wd=0.0001)",
1317
+ "Scheduler": "cosine",
1318
+ "Epochs": 10,
1319
+ "AMP": true
1320
+ },
1321
+ "eval_hyperparameters": {
1322
+ "Project Name": "eval_casia_v2.0",
1323
+ "Dataset": "KhangTruong/Casia-v2.0",
1324
+ "Splits": "Train: 'train' | Val: 'validation' | Test: 'cross_test'",
1325
+ "Streaming": false,
1326
+ "Image Size": "[256, 256]",
1327
+ "Batch Size": 4,
1328
+ "Epoch Limit / Steps": "Full Dataset (until depletion)",
1329
+ "Model": "unet (features=[64, 128, 256, 512], bilinear=True)",
1330
+ "Aux Classifier": "False (classes=3)",
1331
+ "Mask Loss": "Combined (BCE: 0.5, Dice: 0.5, Focal: 0.5)",
1332
+ "Aux Loss": "cross_entropy (wt=0.2)",
1333
+ "Post-Processing": "Enabled (min_area=64, fill_holes=True, morph=open_close)",
1334
+ "Optimizer": "adamw (lr=0.001, wd=0.0001)",
1335
+ "Scheduler": "cosine",
1336
+ "Epochs": 10,
1337
+ "AMP": true
1338
+ },
1339
+ "devices": [
1340
+ "cuda"
1341
+ ],
1342
+ "gpu_utilization": {}
1343
+ }
1344
+ ],
1345
+ "illustration_paths": [
1346
+ "outputs/cross_evaluation/illustrations/cross_eval_iou_heatmap.png",
1347
+ "outputs/cross_evaluation/illustrations/cross_eval_f1_heatmap.png",
1348
+ "outputs/cross_evaluation/illustrations/cross_eval_pixel_acc_heatmap.png",
1349
+ "outputs/cross_evaluation/illustrations/cross_eval_auroc_heatmap.png",
1350
+ "outputs/cross_evaluation/illustrations/sample_predictions_casia_v2.0.png"
1351
+ ]
1352
+ }
versions/v1/cross_evaluation/master_cross_evaluation_report.md ADDED
@@ -0,0 +1,110 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # SID-UNet Master Cross-Evaluation Report
2
+
3
+ ## Overview
4
+
5
+ - **Evaluated Checkpoints**: 2
6
+ - **Evaluated Configurations**: 1
7
+ - **Total Cross-Evaluation Runs**: 2
8
+
9
+ ## Cross-Evaluation Matrices
10
+
11
+ ### Mean IoU (Intersection over Union) Cross-Evaluation Matrix
12
+
13
+ | Checkpoint \ Config | eval_casia_v2.0 |
14
+ |-----------------------|-------------------|
15
+ | sid_focal_hrnet | 0.0000 |
16
+ | sid_pim_swin | 0.2000 |
17
+
18
+ ### Dice / Pixel F1 Score Cross-Evaluation Matrix
19
+
20
+ | Checkpoint \ Config | eval_casia_v2.0 |
21
+ |-----------------------|-------------------|
22
+ | sid_focal_hrnet | 0.0000 |
23
+ | sid_pim_swin | 0.2000 |
24
+
25
+ ### AUROC (Pixel-level ROC-AUC) Cross-Evaluation Matrix
26
+
27
+ | Checkpoint \ Config | eval_casia_v2.0 |
28
+ |-----------------------|-------------------|
29
+ | sid_focal_hrnet | 0.0063 |
30
+ | sid_pim_swin | 1.0000 |
31
+
32
+ ### Pixel Accuracy Cross-Evaluation Matrix
33
+
34
+ | Checkpoint \ Config | eval_casia_v2.0 |
35
+ |-----------------------|-------------------|
36
+ | sid_focal_hrnet | 0.0063 |
37
+ | sid_pim_swin | 0.9088 |
38
+
39
+ ### Evaluation Total Loss Cross-Evaluation Matrix
40
+
41
+ | Checkpoint \ Config | eval_casia_v2.0 |
42
+ |-----------------------|-------------------|
43
+ | sid_focal_hrnet | 1.4154 |
44
+ | sid_pim_swin | 0.6689 |
45
+
46
+ ## Master Consolidated Ranking
47
+
48
+ | Checkpoint | Config (Dataset) | Split | Samples | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Precision | Recall | Specificity | Class Acc |
49
+ |-----------------|------------------------------------------|------------|-----------|--------|--------|-------------|---------|-------------|-------------|----------|---------------|-------------|
50
+ | sid_pim_swin | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 0.6689 | 0.2000 | 0.2000 | 1.0000 | 0.9088 | 0.2000 | 0.2000 | 0.9088 | N/A |
51
+ | sid_focal_hrnet | eval_casia_v2.0 (KhangTruong/Casia-v2.0) | cross_test | 20 | 1.4154 | 0.0000 | 0.0000 | 0.0063 | 0.0063 | 0.0000 | 0.0000 | 0.0063 | N/A |
52
+
53
+
54
+ ## 🖼️ Cross-Evaluation Heatmaps & Visual Illustrations
55
+
56
+ ### Cross Eval Iou Heatmap
57
+
58
+ ![Cross Eval Iou Heatmap](illustrations/cross_eval_iou_heatmap.png)
59
+
60
+ ### Cross Eval F1 Heatmap
61
+
62
+ ![Cross Eval F1 Heatmap](illustrations/cross_eval_f1_heatmap.png)
63
+
64
+ ### Cross Eval Pixel Acc Heatmap
65
+
66
+ ![Cross Eval Pixel Acc Heatmap](illustrations/cross_eval_pixel_acc_heatmap.png)
67
+
68
+ ### Cross Eval Auroc Heatmap
69
+
70
+ ![Cross Eval Auroc Heatmap](illustrations/cross_eval_auroc_heatmap.png)
71
+
72
+ ### Sample Predictions Casia V2.0
73
+
74
+ ![Sample Predictions Casia V2.0](illustrations/sample_predictions_casia_v2.0.png)
75
+
76
+ ---
77
+
78
+ ## Per-Checkpoint Deep-Dive
79
+
80
+ ### Checkpoint: sid_focal_hrnet
81
+
82
+ - **Path**: `outputs/RUN/focal_hrnet/checkpoints/checkpoint_best.pt`
83
+
84
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
85
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
86
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
87
+
88
+
89
+ ### Checkpoint: sid_pim_swin
90
+
91
+ - **Path**: `outputs/RUN/pim_swin/checkpoints/checkpoint_best.pt`
92
+
93
+ | Config | Dataset | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
94
+ |-----------------|------------------------|--------|-------|-------------|---------|-------------|-----------|
95
+ | eval_casia_v2.0 | KhangTruong/Casia-v2.0 | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
96
+
97
+
98
+ ---
99
+
100
+ ## Per-Configuration / Dataset Deep-Dive
101
+
102
+ ### Configuration: eval_casia_v2.0 (Dataset: KhangTruong/Casia-v2.0)
103
+
104
+ - **Config Path**: `configs/cross-eval/casia_v2.0.yaml`
105
+
106
+ | Checkpoint | Loss | IoU | Dice / F1 | AUROC | Pixel Acc | Samples |
107
+ |-----------------|--------|-------|-------------|---------|-------------|-----------|
108
+ | sid_focal_hrnet | 1.4154 | 0 | 0 | 0.0063 | 0.0063 | 20 |
109
+ | sid_pim_swin | 0.6689 | 0.2 | 0.2 | 1 | 0.9088 | 20 |
110
+
versions/v1/manifest.json CHANGED
@@ -9,7 +9,7 @@
9
  "best_epoch": "10",
10
  "total_epochs": "10",
11
  "step": "1000",
12
- "updated_at": "2026-11-12T06:51:00.954724+00:00",
13
  "files": [
14
  "checkpoint_best.pt",
15
  "checkpoint_latest.pt"
 
9
  "best_epoch": "10",
10
  "total_epochs": "10",
11
  "step": "1000",
12
+ "updated_at": "2026-11-12T06:56:08.289820+00:00",
13
  "files": [
14
  "checkpoint_best.pt",
15
  "checkpoint_latest.pt"