PyTorch
echocardiography
ultrasound
masked-autoencoder
echo-mae-2d / README.md
eliasstenhede's picture
Model weights, code, sample and model card
b080b8b verified
|
Raw History Blame Contribute Delete
3.56 kB
---
license: cc-by-nc-sa-4.0
datasets: [Ahus-AIM/EchoXFlow]
library_name: pytorch
tags: [echocardiography, ultrasound, masked-autoencoder]
---
# 2D echocardiography masked autoencoder
Convolutional masked autoencoder for single 2D echocardiography frames in native beamspace
(256 x 128, depth by lateral direction, values in [0, 1]). It reconstructs the full frame from a frame in
which 16 x 16 pixel blocks are hidden, and its encoder gives multi-scale feature maps (`model.encode`).
![Input, masked input and reconstruction](reconstruction.png)
## Usage
Requires `torch` and `matplotlib`. Download the repository and run the code below from inside it
(it is also saved as `example.py`). It loads the model and the sample, hides part of the input, and
saves the figure above.
```python
import matplotlib.pyplot as plt
import numpy as np
import torch
from model import EchoMAE2D
model = EchoMAE2D()
model.load_state_dict(torch.load("model.pt", weights_only=True))
model.eval()
frame = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255 # 1 x 1 x 256 x 128
torch.manual_seed(0)
mask = (torch.rand(1, 1, 16, 8) < 0.5).float() # hide half of the 16 x 16 pixel blocks
mask = mask.repeat_interleave(16, 2).repeat_interleave(16, 3)
with torch.no_grad():
reconstruction = model(frame * (1 - mask))
panels = {"Input": frame, "Masked": frame * (1 - mask), "Reconstruction": reconstruction}
figure, axes = plt.subplots(1, 3, figsize=(6, 4.4))
for axis, (title, image) in zip(axes, panels.items()):
axis.imshow(image[0, 0], cmap="gray", vmin=0, vmax=1)
axis.set_title(title)
axis.axis("off")
figure.tight_layout()
figure.savefig("reconstruction.png", dpi=150)
```
## Model
- Residual encoder with block counts [3, 4, 6, 3] and base width 320 after 8 x 8 input patchification,
batch normalization, and a bilinear U-Net decoder with skip connections; 671 million parameters.
- Pretrained for 1,000 epochs with full-image L1 reconstruction and 50% block masking, then fine-tuned
for 30 epochs together with auxiliary tasks (view, sector geometry, probe rotation, examination
identity, ECG phase). The weights are the exponential moving average at the best validation step.
- The auxiliary prediction heads are not included.
## Files
| File | Content |
|---|---|
| `model.pt` | Model weights (state dict, 2.7 GB) |
| `model.py` | Model definition |
| `sample.npy` | One beamspace frame, `uint8`, 256 x 128 |
| `example.py` | The code above |
## Training data and citation
Trained on [EchoXFlow](https://huggingface.co/datasets/Ahus-AIM/EchoXFlow), a clinical echocardiography
dataset in native beamspace. `sample.npy` is one apical four-chamber frame from its validation fold.
- Paper: [EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function](https://arxiv.org/abs/2605.05447)
- Dataset: https://huggingface.co/datasets/Ahus-AIM/EchoXFlow
- Tooling: https://github.com/Ahus-AIM/EchoXFlow
```bibtex
@misc{stenhede2026echoxflow,
title = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function},
author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian},
year = {2026},
eprint = {2605.05447},
archivePrefix = {arXiv},
primaryClass = {cs.CV}
}
```
## License
CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived.
The model definition is adapted from the MAE in [lambertae/drifting](https://github.com/lambertae/drifting) (MIT license).