arudaev commited on
Commit
105dcfb
Β·
verified Β·
1 Parent(s): fd78522

docs(card): correct CheXNet benchmark comparison to actual per-model AUC

Browse files
Files changed (1) hide show
  1. README.md +167 -167
README.md CHANGED
@@ -1,167 +1,167 @@
1
- ---
2
- license: mit
3
- language:
4
- - en
5
- library_name: pytorch
6
- pipeline_tag: image-classification
7
- tags:
8
- - chexvision
9
- - medical-imaging
10
- - chest-xray
11
- - radiology
12
- - pytorch
13
- - multi-label-classification
14
- datasets:
15
- - arudaev/chest-xray-14-320
16
- ---
17
-
18
- # CheXVision-ResNet
19
-
20
- > **CheXVision** β€” Deep Learning & Big Data university project.
21
- > 14-class chest X-ray pathology detection + binary normal/abnormal classification
22
- > on the NIH Chest X-ray14 dataset (112,120 images).
23
-
24
- ## Project Resources
25
-
26
- - [GitHub repository](https://github.com/arudaev/chexvision)
27
- - [Presentation deck](https://arudaev.github.io/chexvision/)
28
- - [Live demo](https://huggingface.co/spaces/arudaev/chexvision-demo)
29
- - [Dataset](https://huggingface.co/datasets/arudaev/chest-xray-14-320)
30
-
31
- ## Architecture
32
-
33
- ```mermaid
34
- graph LR
35
- IN["Input
36
- 3 Γ— 320 Γ— 320"] --> STEM["Stem
37
- 7Γ—7 Conv Β· BN Β· ReLU
38
- 3β†’64ch Β· MaxPool Γ·2"]
39
- STEM --> S1["Stage 1
40
- 3Γ— SE-ResBlock
41
- 64ch"]
42
- S1 --> S2["Stage 2 ↓½
43
- 4Γ— SE-ResBlock
44
- 128ch"]
45
- S2 --> S3["Stage 3 ↓½
46
- 6Γ— SE-ResBlock
47
- 256ch"]
48
- S3 --> S4["Stage 4 ↓½
49
- 3Γ— SE-ResBlock
50
- 512ch"]
51
- S4 --> GAP["Global Avg Pool
52
- Dropout(0.5)
53
- 512-dim"]
54
- GAP --> MLH["Multilabel Head
55
- Linear 512β†’14
56
- sigmoid Β· 14 pathologies"]
57
- GAP --> BH["Binary Head
58
- Linear 512β†’1
59
- sigmoid Β· Normal/Abnormal"]
60
- style MLH fill:#2e7d32,color:#fff
61
- style BH fill:#1565c0,color:#fff
62
- style IN fill:#37474f,color:#fff
63
- ```
64
-
65
- ## Training Pipeline
66
-
67
- ```mermaid
68
- flowchart TD
69
- DS[("πŸ—„οΈ arudaev/chest-xray-14-320
70
- 112,120 images Β· 36 shards Β· ~7.97 GB")]
71
- DS -->|snapshot_download| PREP["πŸ“‚ data/images Β· data/labels.csv
72
- train 78,468 Β· val 11,210 Β· test 22,442"]
73
- PREP --> AUG["Augmentation Pipeline
74
- HFlip Β· RotateΒ±15Β° Β· RandomAffine
75
- ColorJitter Β· GaussianBlur Β· RandomErasing"]
76
- AUG --> FWD["⚑ Model Forward Pass
77
- torch.cuda.amp.autocast Β· fp16"]
78
- FWD --> ML["multilabel_logits BΓ—14
79
- WeightedBCE + pos_weight Β· 14 classes"]
80
- FWD --> BIN["binary_logits BΓ—1
81
- BCE Β· Normal vs. Abnormal"]
82
- ML --> LOSS["Combined Loss
83
- 1.0 Γ— multilabel + 0.5 Γ— binary"]
84
- BIN --> LOSS
85
- LOSS --> BACK["Backward Β· Grad Clip 1.0
86
- Gradient Accumulation Γ—4 Β· eff. batch 96"]
87
- BACK --> OPT["AdamW Β· CosineAnnealingLR
88
- early stop patience = 15"]
89
- OPT -->|"↑ best val AUC-ROC"| BEST["πŸ’Ύ Best Checkpoint
90
- model_state Β· best_val_metrics Β· config"]
91
- BEST -->|upload_model_artifacts| HUB["πŸ€— HF Hub
92
- checkpoint Β· history.json Β· model card"]
93
- ```
94
-
95
- ## Training Metrics
96
-
97
- - Best validation macro AUC-ROC: `0.8141`
98
- - Best validation binary AUC-ROC: `0.7739`
99
- - Best validation binary F1: `0.6587`
100
- - Best checkpoint epoch: `41`
101
-
102
-
103
- ## Per-Class AUC-ROC at Best Epoch
104
-
105
- | Pathology | AUC-ROC | Visual |
106
- |----------------------|----------|---------------|
107
- | Atelectasis | `0.8022` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
108
- | Cardiomegaly | `0.9059` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
109
- | Effusion | `0.8831` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
110
- | Infiltration | `0.7060` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘` |
111
- | Mass | `0.8596` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
112
- | Nodule | `0.7525` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
113
- | Pneumonia | `0.7298` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘` |
114
- | Pneumothorax | `0.8329` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
115
- | Consolidation | `0.8080` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
116
- | Edema | `0.9122` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
117
- | Emphysema | `0.8545` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
118
- | Fibrosis | `0.7622` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
119
- | Pleural_Thickening | `0.7782` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
120
- | Hernia | `0.8101` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
121
-
122
- ## Training Configuration
123
-
124
- - Repository: `arudaev/chexvision-scratch`
125
- - Dataset: [arudaev/chest-xray-14-320](https://huggingface.co/datasets/arudaev/chest-xray-14-320) Β· revision `44443e6ee968b3c6094b63f14a27698c40b50680`
126
- - Architecture: Custom residual CNN with Squeeze-Excitation channel attention (depth [3, 4, 6, 3]) trained from scratch with shared features and dual classification heads.
127
- - Platform: Kaggle GPU kernel (NVIDIA T4 / P100)
128
- - Batch size: `24` Γ— grad_accum `4` = **effective batch `96`**
129
- - AMP (fp16): `enabled`
130
- - CLAHE preprocessing: `enabled`
131
- - Label smoothing: `0.1`
132
- - Optimizer: AdamW Β· Scheduler: CosineAnnealingLR
133
- - Epochs configured: `100` Β· Early stop patience: `15`
134
-
135
- ## Intended Use
136
-
137
- This model is intended for research and educational work on automated chest X-ray pathology detection.
138
- It outputs two predictions per image:
139
- 1. **Multi-label scores** β€” independent sigmoid probability for each of 14 NIH pathologies
140
- 2. **Binary score** β€” sigmoid probability of any abnormality (Normal vs. Abnormal)
141
-
142
- ## Limitations
143
-
144
- - Not validated for clinical use. Predictions must not substitute professional medical judgment.
145
- - Trained on NIH Chest X-ray14, which contains noisy radiologist annotations (patient-level labels, not lesion-level).
146
- - Performance degrades on images from equipment, patient populations, or preprocessing pipelines
147
- that differ from the NIH training distribution.
148
- - Reported AUC metrics are on the validation split, not the held-out test set.
149
-
150
- ## CheXNet Benchmark Context
151
-
152
- CheXNet (Rajpurkar et al., 2017) β€” the seminal paper establishing DenseNet-121 for chest X-ray
153
- classification β€” reported **0.841 macro AUC-ROC** on a comparable split of this dataset.
154
- CheXVision-DenseNet matches this benchmark. See the
155
- [CheXVision demo](https://huggingface.co/spaces/arudaev/chexvision-demo) for live inference, or the
156
- [presentation deck](https://arudaev.github.io/chexvision/) for the project walkthrough.
157
-
158
- ## Citation
159
-
160
- ```bibtex
161
- @misc{chexvision2026,
162
- title={CheXVision: Dual-Task Chest X-ray Classification with Custom CNN and DenseNet-121},
163
- author={BIG D(ATA) Team},
164
- year={2026},
165
- howpublished={\url{https://huggingface.co/arudaev/chexvision-scratch}}
166
- }
167
- ```
 
1
+ ---
2
+ license: mit
3
+ language:
4
+ - en
5
+ library_name: pytorch
6
+ pipeline_tag: image-classification
7
+ tags:
8
+ - chexvision
9
+ - medical-imaging
10
+ - chest-xray
11
+ - radiology
12
+ - pytorch
13
+ - multi-label-classification
14
+ datasets:
15
+ - arudaev/chest-xray-14-320
16
+ ---
17
+
18
+ # CheXVision-ResNet
19
+
20
+ > **CheXVision** β€” Deep Learning & Big Data university project.
21
+ > 14-class chest X-ray pathology detection + binary normal/abnormal classification
22
+ > on the NIH Chest X-ray14 dataset (112,120 images).
23
+
24
+ ## Project Resources
25
+
26
+ - [GitHub repository](https://github.com/arudaev/chexvision)
27
+ - [Presentation deck](https://arudaev.github.io/chexvision/)
28
+ - [Live demo](https://huggingface.co/spaces/arudaev/chexvision-demo)
29
+ - [Dataset](https://huggingface.co/datasets/arudaev/chest-xray-14-320)
30
+
31
+ ## Architecture
32
+
33
+ ```mermaid
34
+ graph LR
35
+ IN["Input
36
+ 3 Γ— 320 Γ— 320"] --> STEM["Stem
37
+ 7Γ—7 Conv Β· BN Β· ReLU
38
+ 3β†’64ch Β· MaxPool Γ·2"]
39
+ STEM --> S1["Stage 1
40
+ 3Γ— SE-ResBlock
41
+ 64ch"]
42
+ S1 --> S2["Stage 2 ↓½
43
+ 4Γ— SE-ResBlock
44
+ 128ch"]
45
+ S2 --> S3["Stage 3 ↓½
46
+ 6Γ— SE-ResBlock
47
+ 256ch"]
48
+ S3 --> S4["Stage 4 ↓½
49
+ 3Γ— SE-ResBlock
50
+ 512ch"]
51
+ S4 --> GAP["Global Avg Pool
52
+ Dropout(0.5)
53
+ 512-dim"]
54
+ GAP --> MLH["Multilabel Head
55
+ Linear 512β†’14
56
+ sigmoid Β· 14 pathologies"]
57
+ GAP --> BH["Binary Head
58
+ Linear 512β†’1
59
+ sigmoid Β· Normal/Abnormal"]
60
+ style MLH fill:#2e7d32,color:#fff
61
+ style BH fill:#1565c0,color:#fff
62
+ style IN fill:#37474f,color:#fff
63
+ ```
64
+
65
+ ## Training Pipeline
66
+
67
+ ```mermaid
68
+ flowchart TD
69
+ DS[("πŸ—„οΈ arudaev/chest-xray-14-320
70
+ 112,120 images Β· 36 shards Β· ~7.97 GB")]
71
+ DS -->|snapshot_download| PREP["πŸ“‚ data/images Β· data/labels.csv
72
+ train 78,468 Β· val 11,210 Β· test 22,442"]
73
+ PREP --> AUG["Augmentation Pipeline
74
+ HFlip Β· RotateΒ±15Β° Β· RandomAffine
75
+ ColorJitter Β· GaussianBlur Β· RandomErasing"]
76
+ AUG --> FWD["⚑ Model Forward Pass
77
+ torch.cuda.amp.autocast Β· fp16"]
78
+ FWD --> ML["multilabel_logits BΓ—14
79
+ WeightedBCE + pos_weight Β· 14 classes"]
80
+ FWD --> BIN["binary_logits BΓ—1
81
+ BCE Β· Normal vs. Abnormal"]
82
+ ML --> LOSS["Combined Loss
83
+ 1.0 Γ— multilabel + 0.5 Γ— binary"]
84
+ BIN --> LOSS
85
+ LOSS --> BACK["Backward Β· Grad Clip 1.0
86
+ Gradient Accumulation Γ—4 Β· eff. batch 96"]
87
+ BACK --> OPT["AdamW Β· CosineAnnealingLR
88
+ early stop patience = 15"]
89
+ OPT -->|"↑ best val AUC-ROC"| BEST["πŸ’Ύ Best Checkpoint
90
+ model_state Β· best_val_metrics Β· config"]
91
+ BEST -->|upload_model_artifacts| HUB["πŸ€— HF Hub
92
+ checkpoint Β· history.json Β· model card"]
93
+ ```
94
+
95
+ ## Training Metrics
96
+
97
+ - Best validation macro AUC-ROC: `0.8141`
98
+ - Best validation binary AUC-ROC: `0.7739`
99
+ - Best validation binary F1: `0.6587`
100
+ - Best checkpoint epoch: `41`
101
+
102
+
103
+ ## Per-Class AUC-ROC at Best Epoch
104
+
105
+ | Pathology | AUC-ROC | Visual |
106
+ |----------------------|----------|---------------|
107
+ | Atelectasis | `0.8022` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
108
+ | Cardiomegaly | `0.9059` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
109
+ | Effusion | `0.8831` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
110
+ | Infiltration | `0.7060` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘` |
111
+ | Mass | `0.8596` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
112
+ | Nodule | `0.7525` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
113
+ | Pneumonia | `0.7298` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘β–‘` |
114
+ | Pneumothorax | `0.8329` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
115
+ | Consolidation | `0.8080` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
116
+ | Edema | `0.9122` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
117
+ | Emphysema | `0.8545` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘` |
118
+ | Fibrosis | `0.7622` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
119
+ | Pleural_Thickening | `0.7782` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
120
+ | Hernia | `0.8101` | `β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–‘β–‘` |
121
+
122
+ ## Training Configuration
123
+
124
+ - Repository: `arudaev/chexvision-scratch`
125
+ - Dataset: [arudaev/chest-xray-14-320](https://huggingface.co/datasets/arudaev/chest-xray-14-320) Β· revision `44443e6ee968b3c6094b63f14a27698c40b50680`
126
+ - Architecture: Custom residual CNN with Squeeze-Excitation channel attention (depth [3, 4, 6, 3]) trained from scratch with shared features and dual classification heads.
127
+ - Platform: Kaggle GPU kernel (NVIDIA T4 / P100)
128
+ - Batch size: `24` Γ— grad_accum `4` = **effective batch `96`**
129
+ - AMP (fp16): `enabled`
130
+ - CLAHE preprocessing: `enabled`
131
+ - Label smoothing: `0.1`
132
+ - Optimizer: AdamW Β· Scheduler: CosineAnnealingLR
133
+ - Epochs configured: `100` Β· Early stop patience: `15`
134
+
135
+ ## Intended Use
136
+
137
+ This model is intended for research and educational work on automated chest X-ray pathology detection.
138
+ It outputs two predictions per image:
139
+ 1. **Multi-label scores** β€” independent sigmoid probability for each of 14 NIH pathologies
140
+ 2. **Binary score** β€” sigmoid probability of any abnormality (Normal vs. Abnormal)
141
+
142
+ ## Limitations
143
+
144
+ - Not validated for clinical use. Predictions must not substitute professional medical judgment.
145
+ - Trained on NIH Chest X-ray14, which contains noisy radiologist annotations (patient-level labels, not lesion-level).
146
+ - Performance degrades on images from equipment, patient populations, or preprocessing pipelines
147
+ that differ from the NIH training distribution.
148
+ - Reported AUC metrics are on the validation split, not the held-out test set.
149
+
150
+ ## CheXNet Benchmark Context
151
+
152
+ CheXNet (Rajpurkar et al., 2017) β€” the seminal paper establishing DenseNet-121 for chest X-ray
153
+ classification β€” reported **0.841 macro AUC-ROC** on a comparable split of this dataset.
154
+ CheXVision-ResNet, a custom residual SE-network trained **from scratch with no pretrained weights**, reaches **0.8141** macro AUC-ROC β€” within **0.027** of this benchmark. See the
155
+ [CheXVision demo](https://huggingface.co/spaces/arudaev/chexvision-demo) for live inference, or the
156
+ [presentation deck](https://arudaev.github.io/chexvision/) for the project walkthrough.
157
+
158
+ ## Citation
159
+
160
+ ```bibtex
161
+ @misc{chexvision2026,
162
+ title={CheXVision: Dual-Task Chest X-ray Classification with Custom CNN and DenseNet-121},
163
+ author={BIG D(ATA) Team},
164
+ year={2026},
165
+ howpublished={\url{https://huggingface.co/arudaev/chexvision-scratch}}
166
+ }
167
+ ```