Make every command runnable from the repo root; fix the folder-limit and gate wording
Browse files
DATA.md
CHANGED
|
@@ -62,18 +62,20 @@ Every dataset ships the same four companion files, and all four are needed.
|
|
| 62 |
|
| 63 |
## graphs/
|
| 64 |
|
| 65 |
-
Large datasets ship as tars because Hugging Face
|
| 66 |
|
| 67 |
### `graphs/tcga_brca/`: 11,149,499 graphs, 652 GB
|
| 68 |
The **pre-training corpus**. 324 × ~2.0 GB tars under `tcga_brca/data/`, plus the four
|
| 69 |
companion files (its `sample_labels_rich.csv` is 1.94 GB). Also the in-domain cohort for
|
| 70 |
slide-level evaluation.
|
| 71 |
|
|
|
|
|
|
|
| 72 |
```bash
|
| 73 |
-
mkdir -p tcga_brca/
|
| 74 |
-
for t in tcga_brca/data/tcga-brca-*.tar; do tar -xf "$t" -C tcga_brca/
|
| 75 |
-
python modeling/rebase_graph_paths.py --csv tcga_brca/sample_labels_rich.csv \
|
| 76 |
-
--mode absolute --root "$PWD/tcga_brca/
|
| 77 |
```
|
| 78 |
|
| 79 |
### `graphs/bach/`: 14,341 graphs
|
|
@@ -82,7 +84,9 @@ BACH (ICIAR 2018), 224 px @ 20×, 4-class. One `graphs.tar` (flat: the `.pt` fil
|
|
| 82 |
create yourself:
|
| 83 |
|
| 84 |
```bash
|
| 85 |
-
mkdir -p bach/
|
|
|
|
|
|
|
| 86 |
```
|
| 87 |
|
| 88 |
### `graphs/bracs/`: 96,153 graphs
|
|
@@ -225,7 +229,7 @@ cells, columns `centroid_x_px,centroid_y_px,cell_type,type_prob`) plus a summary
|
|
| 225 |
`code/` and `paper_metrics/` hold the analysis code and its outputs.
|
| 226 |
|
| 227 |
The pseudo-labeller was validated on a held-out ground-truth split first: **ARI 0.560**
|
| 228 |
-
(
|
| 229 |
0.900 / 0.000. Three of four classes pass, and the fourth (`other_nucleus`) is untestable at
|
| 230 |
7 ground-truth instances. Re-running that gate needs the external CellViT++ repository and
|
| 231 |
SAM-H weights, which are not included here.
|
|
@@ -238,8 +242,7 @@ conclusion changes, but it needs a minimum-degree guard.
|
|
| 238 |
### `studies/adaptergnn/`: parameter-efficient fine-tuning
|
| 239 |
|
| 240 |
Adapters inserted into a **frozen GrapHist v1** encoder for cell-type identification. Test
|
| 241 |
-
macro-F1 %,
|
| 242 |
-
not comparable across columns):
|
| 243 |
|
| 244 |
| Model | P20 Breast | P20 PanCancer | P40 Breast | P40 PanCancer | NuCLS main | NuCLS super |
|
| 245 |
|---|:--:|:--:|:--:|:--:|:--:|:--:|
|
|
|
|
| 62 |
|
| 63 |
## graphs/
|
| 64 |
|
| 65 |
+
Large datasets ship as tars because Hugging Face documents a ceiling of 10,000 entries per folder.
|
| 66 |
|
| 67 |
### `graphs/tcga_brca/`: 11,149,499 graphs, 652 GB
|
| 68 |
The **pre-training corpus**. 324 × ~2.0 GB tars under `tcga_brca/data/`, plus the four
|
| 69 |
companion files (its `sample_labels_rich.csv` is 1.94 GB). Also the in-domain cohort for
|
| 70 |
slide-level evaluation.
|
| 71 |
|
| 72 |
+
Run from the repository root:
|
| 73 |
+
|
| 74 |
```bash
|
| 75 |
+
mkdir -p graphs/tcga_brca/extracted
|
| 76 |
+
for t in graphs/tcga_brca/data/tcga-brca-*.tar; do tar -xf "$t" -C graphs/tcga_brca/extracted; done
|
| 77 |
+
python modeling/rebase_graph_paths.py --csv graphs/tcga_brca/sample_labels_rich.csv \
|
| 78 |
+
--mode absolute --root "$PWD/graphs/tcga_brca/extracted"
|
| 79 |
```
|
| 80 |
|
| 81 |
### `graphs/bach/`: 14,341 graphs
|
|
|
|
| 84 |
create yourself:
|
| 85 |
|
| 86 |
```bash
|
| 87 |
+
mkdir -p graphs/bach/extracted && tar -xf graphs/bach/graphs.tar -C graphs/bach/extracted
|
| 88 |
+
python modeling/rebase_graph_paths.py --csv graphs/bach/extracted/sample_labels_rich.csv \
|
| 89 |
+
--mode absolute --root "$PWD/graphs/bach/extracted"
|
| 90 |
```
|
| 91 |
|
| 92 |
### `graphs/bracs/`: 96,153 graphs
|
|
|
|
| 229 |
`code/` and `paper_metrics/` hold the analysis code and its outputs.
|
| 230 |
|
| 231 |
The pseudo-labeller was validated on a held-out ground-truth split first: **ARI 0.560**
|
| 232 |
+
(threshold 0.5), Cohen κ 0.709, per-class confusion diagonal 0.864 / 0.562 /
|
| 233 |
0.900 / 0.000. Three of four classes pass, and the fourth (`other_nucleus`) is untestable at
|
| 234 |
7 ground-truth instances. Re-running that gate needs the external CellViT++ repository and
|
| 235 |
SAM-H weights, which are not included here.
|
|
|
|
| 242 |
### `studies/adaptergnn/`: parameter-efficient fine-tuning
|
| 243 |
|
| 244 |
Adapters inserted into a **frozen GrapHist v1** encoder for cell-type identification. Test
|
| 245 |
+
macro-F1 %, each the mean over cross-validation folds (2 to 5 folds depending on the column):
|
|
|
|
| 246 |
|
| 247 |
| Model | P20 Breast | P20 PanCancer | P40 Breast | P40 PanCancer | NuCLS main | NuCLS super |
|
| 248 |
|---|:--:|:--:|:--:|:--:|:--:|:--:|
|