Image Classification
Transformers
Safetensors
bone_age
feature-extraction
radiology
medical_imaging
x_ray
custom_code
Instructions to use Tata24/bone-age with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Tata24/bone-age with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-classification", model="Tata24/bone-age", trust_remote_code=True) pipe("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/hub/parrots.png")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Tata24/bone-age", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Duplicate from ianpan/bone-age
Browse filesCo-authored-by: Ian Pan <ianpan@users.noreply.huggingface.co>
- .gitattributes +35 -0
- README.md +122 -0
- config.json +18 -0
- configuration.py +23 -0
- model.safetensors +3 -0
- modeling.py +133 -0
- ref_img.png +0 -0
.gitattributes
ADDED
|
@@ -0,0 +1,35 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
| 2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
| 3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
| 4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
| 5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
| 6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
| 7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
| 8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
| 9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
| 10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
| 11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
| 12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
| 13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
| 14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
| 15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
| 16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
| 17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
| 18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
| 19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
| 20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
| 21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
| 22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
| 23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
| 24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
| 25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
| 26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
| 27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
| 28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
| 29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
| 30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
| 31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
| 32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
| 33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
README.md
ADDED
|
@@ -0,0 +1,122 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
library_name: transformers
|
| 3 |
+
tags:
|
| 4 |
+
- radiology
|
| 5 |
+
- medical_imaging
|
| 6 |
+
- bone_age
|
| 7 |
+
- x_ray
|
| 8 |
+
license: apache-2.0
|
| 9 |
+
base_model:
|
| 10 |
+
- timm/convnextv2_tiny.fcmae_ft_in22k_in1k
|
| 11 |
+
pipeline_tag: image-classification
|
| 12 |
+
---
|
| 13 |
+
|
| 14 |
+
**Training code:** [bone-age-prediction](https://github.com/i-pan/bone-age-prediction) — data preparation, training, evaluation, and Hugging Face export.
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
This model has been trained and validated on 14,036 pediatric hand radiographs from the [RSNA Pediatric Bone Age Challenge](https://www.rsna.org/rsnai/ai-image-challenge/rsna-pediatric-bone-age-challenge-2017) dataset, which is publicly available.
|
| 18 |
+
It can be loaded using:
|
| 19 |
+
```
|
| 20 |
+
from transformers import AutoModel
|
| 21 |
+
|
| 22 |
+
model = AutoModel.from_pretrained("ianpan/bone-age", trust_remote_code=True)
|
| 23 |
+
```
|
| 24 |
+
The model is a 3-fold ensemble utilizing the `convnextv2_tiny` backbone.
|
| 25 |
+
The individual single-fold models can be accessed through `model.net0`, `model.net1`, `model.net2`. Each of these models was trained over 20,000 iterations using a batch size of 64 across 2 NVIDIA RTX 3090 GPUs.
|
| 26 |
+
Originally, it was trained with both a regression and classification head.
|
| 27 |
+
However, this model only loads the classification head, as stand-alone performance was slightly better. The classification head also generates better GradCAMs.
|
| 28 |
+
The softmax function is applied to the output logits and multiplied by the corresponding class indices, then summed.
|
| 29 |
+
This outputs a scalar float value representing the predicted bone age in units of months.
|
| 30 |
+
|
| 31 |
+
In addition to standard data augmentation, additional augmentations were also applied:
|
| 32 |
+
- Using a cropped radiograph (from the model <https://huggingface.co/ianpan/bone-age-crop>) with probability 0.5
|
| 33 |
+
- Histogram matching with a reference image (available in this repo under Files, `ref_img.png`) with probability 0.5
|
| 34 |
+
|
| 35 |
+
Note that both of the above augmentations could be applied simultaneously and in conjunction with standard data augamentations. Thus, the model accommodates a large range of variability in the appearance of a hand radiograph.
|
| 36 |
+
|
| 37 |
+
On the original challenge test set comprising 200 multi-annotated pediatric hand radiographs, this model achieves a **mean absolute error of 4.16 months** (when applying both cropping and histogram matching to the input radiograph), which surpasses the [top solutions](https://pubs.rsna.org/doi/10.1148/radiol.2018180736) from the original challenge.
|
| 38 |
+
Specific results as follows, with single model performance using `model.net0` in brackets:
|
| 39 |
+
```
|
| 40 |
+
Crop (-) / Histogram Matching (-): 4.42 [4.67] months
|
| 41 |
+
Crop (+) / Histogram Matching (-): 4.47 [4.84] months
|
| 42 |
+
Crop (-) / Histogram Matching (+): 4.34 [4.59] months
|
| 43 |
+
Crop (+) / Histogram Matching (+): 4.16 [4.45] months
|
| 44 |
+
```
|
| 45 |
+
|
| 46 |
+
Thus it is preferable to both crop and histogram match the image to obtain the optimal results. See <https://huggingface.co/ianpan/bone-age-crop> for how to crop a bone age radiograph with a pretrained model.
|
| 47 |
+
To histogram match with a reference image:
|
| 48 |
+
```
|
| 49 |
+
import cv2
|
| 50 |
+
from skimage.exposure import match_histograms
|
| 51 |
+
|
| 52 |
+
x = cv2.imread("target_radiograph.png", 0)
|
| 53 |
+
ref = cv2.imread("ref_img.png", 0) # download ref_img.png from this repo
|
| 54 |
+
x = match_histograms(x, ref)
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
Patient sex is an important variable affecting the model's prediction. This is passed to the model's `forward()` function using the `female` argument:
|
| 58 |
+
```
|
| 59 |
+
# 1 indicates female, 0 male
|
| 60 |
+
model(x, female=torch.tensor([1, 0, 1, 0])) # assuming batch size of 4
|
| 61 |
+
```
|
| 62 |
+
|
| 63 |
+
Example usage for a single image:
|
| 64 |
+
```
|
| 65 |
+
import cv2
|
| 66 |
+
import torch
|
| 67 |
+
from skimage.exposure import match_histograms
|
| 68 |
+
from transformers import AutoModel
|
| 69 |
+
|
| 70 |
+
device = "cuda" if torch.cuda.is_available() else "cpu"
|
| 71 |
+
|
| 72 |
+
crop_model = AutoModel.from_pretrained("ianpan/bone-age-crop", trust_remote_code=True)
|
| 73 |
+
crop_model = crop_model.eval().to(device)
|
| 74 |
+
img = cv2.imread(..., 0)
|
| 75 |
+
img_shape = torch.tensor([img.shape[:2]])
|
| 76 |
+
x = crop_model.preprocess(img) # only takes single image as input
|
| 77 |
+
x = torch.from_numpy(x).unsqueeze(0).unsqueeze(0) # add channel, batch dims
|
| 78 |
+
x = x.float()
|
| 79 |
+
|
| 80 |
+
# if you do not provide img_shape
|
| 81 |
+
# model will return normalized coordinates
|
| 82 |
+
with torch.inference_mode():
|
| 83 |
+
coords = model(x.to(device), img_shape.to(device))
|
| 84 |
+
|
| 85 |
+
# only 1 sample in batch
|
| 86 |
+
coords = coords[0].cpu().numpy()
|
| 87 |
+
x, y, w, h = coords
|
| 88 |
+
# coords already rescaled with img_shape
|
| 89 |
+
cropped_img = img[y: y + h, x: x + w]
|
| 90 |
+
|
| 91 |
+
# histogram matching
|
| 92 |
+
ref = cv2.imread("ref_img.png", 0) # download ref_img.png from this repo
|
| 93 |
+
cropped_img = match_histograms(cropped_img, ref)
|
| 94 |
+
|
| 95 |
+
model = AutoModel.from_pretrained("ianpan/bone-age", trust_remote_code=True)
|
| 96 |
+
model = model.eval().to(device)
|
| 97 |
+
x = model.preprocess(cropped_img)
|
| 98 |
+
x = torch.from_numpy(x).unsqueeze(0).unsqueeze(0)
|
| 99 |
+
x = x.float()
|
| 100 |
+
female = torch.tensor([1])
|
| 101 |
+
|
| 102 |
+
with torch.inference_mode():
|
| 103 |
+
bone_age = model(x.to(device), female.to(device))
|
| 104 |
+
```
|
| 105 |
+
|
| 106 |
+
If you want the raw logits (class `i` = `i` months), you can pass `return_logits=True` to `forward()`:
|
| 107 |
+
```
|
| 108 |
+
bone_age_logits = model(x, female, return_logits=True)
|
| 109 |
+
```
|
| 110 |
+
|
| 111 |
+
To run single model inference, simply access one of the nets:
|
| 112 |
+
```
|
| 113 |
+
bone_age = model.net0(x, female)
|
| 114 |
+
```
|
| 115 |
+
|
| 116 |
+
If you have `pydicom` installed, you can also load a DICOM image directly:
|
| 117 |
+
```
|
| 118 |
+
img = model.load_image_from_dicom(path_to_dicom)
|
| 119 |
+
```
|
| 120 |
+
|
| 121 |
+
This model is for demonstration and research purposes only and has NOT been approved by any regulatory agency for clinical use.
|
| 122 |
+
The user assumes any and all responsibility regarding their own use of this model and its outputs.
|
config.json
ADDED
|
@@ -0,0 +1,18 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"architectures": [
|
| 3 |
+
"BoneAgeEnsembleModel"
|
| 4 |
+
],
|
| 5 |
+
"auto_map": {
|
| 6 |
+
"AutoConfig": "configuration.BoneAgeConfig",
|
| 7 |
+
"AutoModel": "modeling.BoneAgeEnsembleModel"
|
| 8 |
+
},
|
| 9 |
+
"backbone": "convnextv2_tiny",
|
| 10 |
+
"dropout": 0.1,
|
| 11 |
+
"feature_dim": 768,
|
| 12 |
+
"in_chans": 2,
|
| 13 |
+
"model_type": "bone_age",
|
| 14 |
+
"num_classes": 240,
|
| 15 |
+
"num_models": 3,
|
| 16 |
+
"torch_dtype": "float32",
|
| 17 |
+
"transformers_version": "4.47.0"
|
| 18 |
+
}
|
configuration.py
ADDED
|
@@ -0,0 +1,23 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from transformers import PretrainedConfig
|
| 2 |
+
|
| 3 |
+
|
| 4 |
+
class BoneAgeConfig(PretrainedConfig):
|
| 5 |
+
model_type = "bone_age"
|
| 6 |
+
|
| 7 |
+
def __init__(
|
| 8 |
+
self,
|
| 9 |
+
backbone="convnextv2_tiny",
|
| 10 |
+
feature_dim=768,
|
| 11 |
+
dropout=0.1,
|
| 12 |
+
num_classes=240,
|
| 13 |
+
in_chans=2,
|
| 14 |
+
num_models=3,
|
| 15 |
+
**kwargs
|
| 16 |
+
):
|
| 17 |
+
self.backbone = backbone
|
| 18 |
+
self.feature_dim = feature_dim
|
| 19 |
+
self.dropout = dropout
|
| 20 |
+
self.num_classes = num_classes
|
| 21 |
+
self.in_chans = in_chans
|
| 22 |
+
self.num_models = num_models
|
| 23 |
+
super().__init__(**kwargs)
|
model.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2ac6cd24258802e23a9233f824f4670de4aece4523681511a03d0841a06cc2fc
|
| 3 |
+
size 336662916
|
modeling.py
ADDED
|
@@ -0,0 +1,133 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import albumentations as A
|
| 2 |
+
import cv2
|
| 3 |
+
import torch
|
| 4 |
+
import torch.nn as nn
|
| 5 |
+
import torch.nn.functional as F
|
| 6 |
+
|
| 7 |
+
from numpy.typing import NDArray
|
| 8 |
+
from transformers import PreTrainedModel
|
| 9 |
+
from timm import create_model
|
| 10 |
+
from typing import Optional
|
| 11 |
+
from .configuration import BoneAgeConfig
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class GeM(nn.Module):
|
| 15 |
+
def __init__(
|
| 16 |
+
self, p: int = 3, eps: float = 1e-6, dim: int = 2, flatten: bool = True
|
| 17 |
+
):
|
| 18 |
+
super().__init__()
|
| 19 |
+
self.p = nn.Parameter(torch.ones(1) * p)
|
| 20 |
+
self.eps = eps
|
| 21 |
+
assert dim in {2, 3}, f"dim must be one of [2, 3], not {dim}"
|
| 22 |
+
self.dim = dim
|
| 23 |
+
if self.dim == 2:
|
| 24 |
+
self.func = F.adaptive_avg_pool2d
|
| 25 |
+
elif self.dim == 3:
|
| 26 |
+
self.func = F.adaptive_avg_pool3d
|
| 27 |
+
self.flatten = nn.Flatten(1) if flatten else nn.Identity()
|
| 28 |
+
|
| 29 |
+
def forward(self, x: torch.Tensor) -> torch.Tensor:
|
| 30 |
+
# assumes x.shape is (n, c, [t], h, w)
|
| 31 |
+
x = self.func(x.clamp(min=self.eps).pow(self.p), output_size=1).pow(
|
| 32 |
+
1.0 / self.p
|
| 33 |
+
)
|
| 34 |
+
return self.flatten(x)
|
| 35 |
+
|
| 36 |
+
|
| 37 |
+
class BoneAgeModel(nn.Module):
|
| 38 |
+
def __init__(
|
| 39 |
+
self, backbone, feature_dim=768, dropout=0.1, num_classes=240, in_chans=2
|
| 40 |
+
):
|
| 41 |
+
super().__init__()
|
| 42 |
+
self.backbone = create_model(
|
| 43 |
+
model_name=backbone,
|
| 44 |
+
pretrained=False,
|
| 45 |
+
num_classes=0,
|
| 46 |
+
global_pool="",
|
| 47 |
+
features_only=False,
|
| 48 |
+
in_chans=in_chans,
|
| 49 |
+
)
|
| 50 |
+
self.pooling = GeM(p=3, dim=2)
|
| 51 |
+
self.dropout = nn.Dropout(p=dropout)
|
| 52 |
+
self.linear = nn.Linear(feature_dim, num_classes)
|
| 53 |
+
|
| 54 |
+
def normalize(self, x: torch.Tensor) -> torch.Tensor:
|
| 55 |
+
# [0, 255] -> [-1, 1]
|
| 56 |
+
mini, maxi = 0.0, 255.0
|
| 57 |
+
x = (x - mini) / (maxi - mini)
|
| 58 |
+
x = (x - 0.5) * 2.0
|
| 59 |
+
return x
|
| 60 |
+
|
| 61 |
+
def forward(
|
| 62 |
+
self, x: torch.Tensor, female: torch.Tensor, return_logits: bool = False
|
| 63 |
+
) -> torch.Tensor:
|
| 64 |
+
assert x.size(0) == female.size(
|
| 65 |
+
0
|
| 66 |
+
), f"x.size(0) [{x.size(0)}] must equal female.size(0) [{female.size(0)}]"
|
| 67 |
+
female_ch = torch.zeros_like(x).to(x.device)
|
| 68 |
+
female_ch[female.bool()] = 255.0
|
| 69 |
+
x = torch.cat([x, female_ch], dim=1)
|
| 70 |
+
x = self.normalize(x)
|
| 71 |
+
features = self.pooling(self.backbone(x))
|
| 72 |
+
logits = self.linear(features)
|
| 73 |
+
if return_logits:
|
| 74 |
+
return logits
|
| 75 |
+
out = (logits.softmax(1) * torch.arange(logits.size(1)).to(logits.device)).sum(
|
| 76 |
+
1
|
| 77 |
+
)
|
| 78 |
+
return out
|
| 79 |
+
|
| 80 |
+
|
| 81 |
+
class BoneAgeEnsembleModel(PreTrainedModel):
|
| 82 |
+
config_class = BoneAgeConfig
|
| 83 |
+
|
| 84 |
+
def __init__(self, config):
|
| 85 |
+
super().__init__(config)
|
| 86 |
+
self.num_models = config.num_models
|
| 87 |
+
for i in range(self.num_models):
|
| 88 |
+
setattr(
|
| 89 |
+
self,
|
| 90 |
+
f"net{i}",
|
| 91 |
+
BoneAgeModel(
|
| 92 |
+
config.backbone,
|
| 93 |
+
config.feature_dim,
|
| 94 |
+
config.dropout,
|
| 95 |
+
config.num_classes,
|
| 96 |
+
config.in_chans,
|
| 97 |
+
),
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
@staticmethod
|
| 101 |
+
def load_image_from_dicom(path: str) -> Optional[NDArray]:
|
| 102 |
+
try:
|
| 103 |
+
from pydicom import dcmread
|
| 104 |
+
from pydicom.pixels import apply_voi_lut
|
| 105 |
+
except ModuleNotFoundError:
|
| 106 |
+
print("`pydicom` is not installed, returning None ...")
|
| 107 |
+
return None
|
| 108 |
+
dicom = dcmread(path)
|
| 109 |
+
arr = apply_voi_lut(dicom.pixel_array, dicom)
|
| 110 |
+
if dicom.PhotometricInterpretation == "MONOCHROME1":
|
| 111 |
+
arr = arr.max() - arr
|
| 112 |
+
|
| 113 |
+
arr = arr - arr.min()
|
| 114 |
+
arr = arr / arr.max()
|
| 115 |
+
arr = (arr * 255).astype("uint8")
|
| 116 |
+
return arr
|
| 117 |
+
|
| 118 |
+
@staticmethod
|
| 119 |
+
def preprocess(x: NDArray) -> NDArray:
|
| 120 |
+
x = A.LongestMaxSize(max_size=512, p=1)(image=x)["image"]
|
| 121 |
+
x = A.PadIfNeeded(512, 512, border_mode=cv2.BORDER_CONSTANT, p=1)(image=x)[
|
| 122 |
+
"image"
|
| 123 |
+
]
|
| 124 |
+
return x
|
| 125 |
+
|
| 126 |
+
def forward(
|
| 127 |
+
self, x: torch.Tensor, female: torch.Tensor, return_logits: bool = False
|
| 128 |
+
) -> torch.Tensor:
|
| 129 |
+
out = []
|
| 130 |
+
for i in range(self.num_models):
|
| 131 |
+
model = getattr(self, f"net{i}")
|
| 132 |
+
out.append(model(x, female, return_logits))
|
| 133 |
+
return torch.stack(out).mean(0)
|
ref_img.png
ADDED
|