Image Segmentation
FleXray
ONNX
Safetensors
PyTorch
medical-image-segmentation
x-ray
radiograph
anatomy
Instructions to use VictorButoi/flexray with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- FleXray
How to use VictorButoi/flexray with FleXray:
pip install flexray
from fxr.inference import FleXraySegmenter segmenter = FleXraySegmenter.from_pretrained("VictorButoi/flexray") prediction = segmenter.predict("image.png", threshold=0.5) masks = prediction.masks - Notebooks
- Google Colab
- Kaggle
Correct public tutorial and inference documentation
Browse files
README.md
CHANGED
|
@@ -18,7 +18,7 @@ tags:
|
|
| 18 |
- Website and in-browser demo: [flexray.csail.mit.edu](https://flexray.csail.mit.edu/)
|
| 19 |
- Code: [github.com/VictorButoi/FleXray](https://github.com/VictorButoi/FleXray)
|
| 20 |
- Data: [`VictorButoi/flexray-data`](https://huggingface.co/datasets/VictorButoi/flexray-data)
|
| 21 |
-
- Tutorial: [Colab notebook](https://colab.research.google.com/drive/
|
| 22 |
- Paper: *FleXray: Universal Clinical X-ray Segmentation* (coming soon)
|
| 23 |
|
| 24 |
FleXray is a single 2D UNet that segments anatomy from standard radiographs
|
|
@@ -43,13 +43,15 @@ from fxr.inference import FleXraySegmenter
|
|
| 43 |
|
| 44 |
segmenter = FleXraySegmenter.from_pretrained("VictorButoi/flexray")
|
| 45 |
prediction = segmenter.predict("./image.png", threshold=0.5)
|
| 46 |
-
prediction.masks # uint8,
|
| 47 |
-
prediction.probabilities # float32,
|
| 48 |
-
prediction.logits # float32,
|
| 49 |
```
|
| 50 |
|
| 51 |
`flexify` writes `<name>_masks.npy`, `<name>_probabilities.npy`, and
|
| 52 |
-
`<name>_logits.npy` per image
|
|
|
|
|
|
|
| 53 |
Pass `--binary LABEL` (for example `--binary femurs`) to write one label. See
|
| 54 |
[docs/inference.md](https://github.com/VictorButoi/FleXray/blob/main/docs/inference.md)
|
| 55 |
for the full CLI and Python API.
|
|
@@ -87,39 +89,48 @@ member = FleXraySegmenter.from_pretrained(
|
|
| 87 |
```
|
| 88 |
|
| 89 |
The website demo exposes the same choices as quality modes: **Low** runs the
|
| 90 |
-
flagship once, **Normal** runs the flagship with
|
| 91 |
-
five-model ensemble once, and **X-High** runs the ensemble with
|
| 92 |
The members are also listed in
|
| 93 |
[MODEL_ZOO.md](https://github.com/VictorButoi/FleXray/blob/main/MODEL_ZOO.md).
|
| 94 |
|
| 95 |
## Test-time augmentation
|
| 96 |
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
|
|
|
|
| 100 |
|
| 101 |
`tta_samples=N` runs one un-augmented pass plus `N - 1` randomly augmented
|
| 102 |
-
passes and averages
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
and the browser demo mirrors it exactly:
|
| 106 |
|
| 107 |
-
| Transform | Probability |
|
| 108 |
| --- | --- | --- |
|
| 109 |
-
| Horizontal flip
|
| 110 |
-
|
|
| 111 |
-
|
|
| 112 |
-
|
|
| 113 |
-
|
|
| 114 |
-
|
|
| 115 |
-
|
|
| 116 |
-
|
| 117 |
-
The flip is the only geometric transform; intensity transforms
|
| 118 |
-
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
| 122 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 123 |
|
| 124 |
## Input contract
|
| 125 |
|
|
@@ -183,8 +194,8 @@ Every dataset's license, redistribution status, and download pointer is
|
|
| 183 |
documented in the
|
| 184 |
[`VictorButoi/flexray-data`](https://huggingface.co/datasets/VictorButoi/flexray-data)
|
| 185 |
card. That repository ships the real X-ray sources whose licenses permit
|
| 186 |
-
redistribution
|
| 187 |
-
the FluXray database.
|
| 188 |
|
| 189 |
## Evaluation
|
| 190 |
|
|
@@ -222,7 +233,8 @@ clinical diagnosis, treatment planning, or patient-care decisions.
|
|
| 222 |
- `members/<name>/checksums.json`: SHA256 checksums of the bundle files.
|
| 223 |
- `members/<name>/onnx/flexray-<name>-256-fp16.onnx`: fp16 ONNX export
|
| 224 |
(opset 18, sigmoid baked in) used by the in-browser demo; parity-checked
|
| 225 |
-
against the PyTorch weights by `
|
|
|
|
| 226 |
|
| 227 |
## Licenses
|
| 228 |
|
|
|
|
| 18 |
- Website and in-browser demo: [flexray.csail.mit.edu](https://flexray.csail.mit.edu/)
|
| 19 |
- Code: [github.com/VictorButoi/FleXray](https://github.com/VictorButoi/FleXray)
|
| 20 |
- Data: [`VictorButoi/flexray-data`](https://huggingface.co/datasets/VictorButoi/flexray-data)
|
| 21 |
+
- Tutorial: [Colab notebook](https://colab.research.google.com/drive/1jMBoOyV8PkRThHi3i6QIMjolmNoRE0cD)
|
| 22 |
- Paper: *FleXray: Universal Clinical X-ray Segmentation* (coming soon)
|
| 23 |
|
| 24 |
FleXray is a single 2D UNet that segments anatomy from standard radiographs
|
|
|
|
| 43 |
|
| 44 |
segmenter = FleXraySegmenter.from_pretrained("VictorButoi/flexray")
|
| 45 |
prediction = segmenter.predict("./image.png", threshold=0.5)
|
| 46 |
+
prediction.masks # uint8, BxCxHxW thresholded masks
|
| 47 |
+
prediction.probabilities # float32, BxCxHxW sigmoid probabilities
|
| 48 |
+
prediction.logits # float32, BxCxHxW raw scores
|
| 49 |
```
|
| 50 |
|
| 51 |
`flexify` writes `<name>_masks.npy`, `<name>_probabilities.npy`, and
|
| 52 |
+
`<name>_logits.npy` per image, each shaped `CxHxW`. The Python API keeps
|
| 53 |
+
the batch dimension (`B=1` for a single image). Channel order follows
|
| 54 |
+
`label_schema.json`.
|
| 55 |
Pass `--binary LABEL` (for example `--binary femurs`) to write one label. See
|
| 56 |
[docs/inference.md](https://github.com/VictorButoi/FleXray/blob/main/docs/inference.md)
|
| 57 |
for the full CLI and Python API.
|
|
|
|
| 89 |
```
|
| 90 |
|
| 91 |
The website demo exposes the same choices as quality modes: **Low** runs the
|
| 92 |
+
flagship once, **Normal** runs the flagship with 8-pass TTA, **High** runs the
|
| 93 |
+
five-model ensemble once, and **X-High** runs the ensemble with 8-pass TTA.
|
| 94 |
The members are also listed in
|
| 95 |
[MODEL_ZOO.md](https://github.com/VictorButoi/FleXray/blob/main/MODEL_ZOO.md).
|
| 96 |
|
| 97 |
## Test-time augmentation
|
| 98 |
|
| 99 |
+
The reported results use 16 passes per model (`--tta-samples 16` or
|
| 100 |
+
`predict(..., tta_samples=16)`). The browser demo uses 8 passes per model in
|
| 101 |
+
Normal and X-High modes; its current settings are published in the
|
| 102 |
+
[demo manifest](https://flexray.csail.mit.edu/demo/demo_manifest.json).
|
| 103 |
|
| 104 |
`tta_samples=N` runs one un-augmented pass plus `N - 1` randomly augmented
|
| 105 |
+
passes and averages their sigmoid probabilities, then converts that mean
|
| 106 |
+
back to logits. The package and browser implement the released `tta_v3`
|
| 107 |
+
chain in this order:
|
|
|
|
| 108 |
|
| 109 |
+
| Transform | Probability | Parameters |
|
| 110 |
| --- | --- | --- |
|
| 111 |
+
| Horizontal flip | 0.5 | Exactly inverted on the prediction before averaging |
|
| 112 |
+
| Invert intensities | 0.5 | `1 - image` |
|
| 113 |
+
| CLAHE | 0.1 | Clip limit 1.0-2.0; 8 x 8 grid |
|
| 114 |
+
| Gamma | 0.25 | Gamma 0.9-1.1; gain 0.9-1.1; mutually exclusive with CLAHE |
|
| 115 |
+
| Contrast | 0.25 | Multiply intensities by 0.7-1.3 and clamp to [0, 1] |
|
| 116 |
+
| Sharpness | 0.5 | Factor 0.7-1.3 |
|
| 117 |
+
| Gaussian noise | 0.25 | Standard deviation 0.01 |
|
| 118 |
+
|
| 119 |
+
The flip is the only geometric transform; intensity transforms are not
|
| 120 |
+
inverted. The CLAHE/gamma branch leaves the image unchanged with probability
|
| 121 |
+
0.65. See the
|
| 122 |
+
[Python implementation](https://github.com/VictorButoi/FleXray/blob/main/src/fxr/inference/tta.py)
|
| 123 |
+
and [browser implementation](https://flexray.csail.mit.edu/demo/tta.js).
|
| 124 |
+
|
| 125 |
+
Use `predict(..., tta_samples=16, seed=42)` to reproduce the Python
|
| 126 |
+
augmentation draws without changing the global torch RNG. With no seed,
|
| 127 |
+
draws use the global torch RNG. The browser uses its own random-number
|
| 128 |
+
source, so matching augmentation settings do not imply identical random views.
|
| 129 |
+
|
| 130 |
+
With an ensemble, each view is drawn once and run through every member.
|
| 131 |
+
`M` models and `N` passes therefore require `M x N` forward passes: 80 for
|
| 132 |
+
the five-model ensemble at N=16, or 40 for the browser's X-High mode at N=8.
|
| 133 |
+
`tta_samples<=1` disables augmentation.
|
| 134 |
|
| 135 |
## Input contract
|
| 136 |
|
|
|
|
| 194 |
documented in the
|
| 195 |
[`VictorButoi/flexray-data`](https://huggingface.co/datasets/VictorButoi/flexray-data)
|
| 196 |
card. That repository ships the real X-ray sources whose licenses permit
|
| 197 |
+
redistribution as image/mask pairs with packaging manifests, the MURA masks,
|
| 198 |
+
and the FluXray database.
|
| 199 |
|
| 200 |
## Evaluation
|
| 201 |
|
|
|
|
| 233 |
- `members/<name>/checksums.json`: SHA256 checksums of the bundle files.
|
| 234 |
- `members/<name>/onnx/flexray-<name>-256-fp16.onnx`: fp16 ONNX export
|
| 235 |
(opset 18, sigmoid baked in) used by the in-browser demo; parity-checked
|
| 236 |
+
against the PyTorch weights by `scripts.release.export_web_demo` in the
|
| 237 |
+
release tooling.
|
| 238 |
|
| 239 |
## Licenses
|
| 240 |
|