|
Download README.md from Ahus-AIM/echo-mae-2d: direct link, hf CLI and curl.
- Browser
- Download file 3.56 kB
-
https://huggingface.co/Ahus-AIM/echo-mae-2d/resolve/main/README.md
- Command line
-
hf download hf://Ahus-AIM/echo-mae-2d/README.md
-
curl -L -o README.md https://huggingface.co/Ahus-AIM/echo-mae-2d/resolve/main/README.md
3.56 kB
| license: cc-by-nc-sa-4.0 | |
| datasets: [Ahus-AIM/EchoXFlow] | |
| library_name: pytorch | |
| tags: [echocardiography, ultrasound, masked-autoencoder] | |
| # 2D echocardiography masked autoencoder | |
| Convolutional masked autoencoder for single 2D echocardiography frames in native beamspace | |
| (256 x 128, depth by lateral direction, values in [0, 1]). It reconstructs the full frame from a frame in | |
| which 16 x 16 pixel blocks are hidden, and its encoder gives multi-scale feature maps (`model.encode`). | |
|  | |
| ## Usage | |
| Requires `torch` and `matplotlib`. Download the repository and run the code below from inside it | |
| (it is also saved as `example.py`). It loads the model and the sample, hides part of the input, and | |
| saves the figure above. | |
| ```python | |
| import matplotlib.pyplot as plt | |
| import numpy as np | |
| import torch | |
| from model import EchoMAE2D | |
| model = EchoMAE2D() | |
| model.load_state_dict(torch.load("model.pt", weights_only=True)) | |
| model.eval() | |
| frame = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255 # 1 x 1 x 256 x 128 | |
| torch.manual_seed(0) | |
| mask = (torch.rand(1, 1, 16, 8) < 0.5).float() # hide half of the 16 x 16 pixel blocks | |
| mask = mask.repeat_interleave(16, 2).repeat_interleave(16, 3) | |
| with torch.no_grad(): | |
| reconstruction = model(frame * (1 - mask)) | |
| panels = {"Input": frame, "Masked": frame * (1 - mask), "Reconstruction": reconstruction} | |
| figure, axes = plt.subplots(1, 3, figsize=(6, 4.4)) | |
| for axis, (title, image) in zip(axes, panels.items()): | |
| axis.imshow(image[0, 0], cmap="gray", vmin=0, vmax=1) | |
| axis.set_title(title) | |
| axis.axis("off") | |
| figure.tight_layout() | |
| figure.savefig("reconstruction.png", dpi=150) | |
| ``` | |
| ## Model | |
| - Residual encoder with block counts [3, 4, 6, 3] and base width 320 after 8 x 8 input patchification, | |
| batch normalization, and a bilinear U-Net decoder with skip connections; 671 million parameters. | |
| - Pretrained for 1,000 epochs with full-image L1 reconstruction and 50% block masking, then fine-tuned | |
| for 30 epochs together with auxiliary tasks (view, sector geometry, probe rotation, examination | |
| identity, ECG phase). The weights are the exponential moving average at the best validation step. | |
| - The auxiliary prediction heads are not included. | |
| ## Files | |
| | File | Content | | |
| |---|---| | |
| | `model.pt` | Model weights (state dict, 2.7 GB) | | |
| | `model.py` | Model definition | | |
| | `sample.npy` | One beamspace frame, `uint8`, 256 x 128 | | |
| | `example.py` | The code above | | |
| ## Training data and citation | |
| Trained on [EchoXFlow](https://huggingface.co/datasets/Ahus-AIM/EchoXFlow), a clinical echocardiography | |
| dataset in native beamspace. `sample.npy` is one apical four-chamber frame from its validation fold. | |
| - Paper: [EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function](https://arxiv.org/abs/2605.05447) | |
| - Dataset: https://huggingface.co/datasets/Ahus-AIM/EchoXFlow | |
| - Tooling: https://github.com/Ahus-AIM/EchoXFlow | |
| ```bibtex | |
| @misc{stenhede2026echoxflow, | |
| title = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function}, | |
| author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian}, | |
| year = {2026}, | |
| eprint = {2605.05447}, | |
| archivePrefix = {arXiv}, | |
| primaryClass = {cs.CV} | |
| } | |
| ``` | |
| ## License | |
| CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived. | |
| The model definition is adapted from the MAE in [lambertae/drifting](https://github.com/lambertae/drifting) (MIT license). | |