PyTorch
echocardiography
ultrasound
masked-autoencoder
echo-mae-2d / README.md
eliasstenhede's picture
Model weights, code, sample and model card
b080b8b verified
|
Raw History Blame Contribute Delete
3.56 kB
metadata
license: cc-by-nc-sa-4.0
datasets:
  - Ahus-AIM/EchoXFlow
library_name: pytorch
tags:
  - echocardiography
  - ultrasound
  - masked-autoencoder

2D echocardiography masked autoencoder

Convolutional masked autoencoder for single 2D echocardiography frames in native beamspace (256 x 128, depth by lateral direction, values in [0, 1]). It reconstructs the full frame from a frame in which 16 x 16 pixel blocks are hidden, and its encoder gives multi-scale feature maps (model.encode).

Input, masked input and reconstruction

Usage

Requires torch and matplotlib. Download the repository and run the code below from inside it (it is also saved as example.py). It loads the model and the sample, hides part of the input, and saves the figure above.

import matplotlib.pyplot as plt
import numpy as np
import torch

from model import EchoMAE2D

model = EchoMAE2D()
model.load_state_dict(torch.load("model.pt", weights_only=True))
model.eval()

frame = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255  # 1 x 1 x 256 x 128
torch.manual_seed(0)
mask = (torch.rand(1, 1, 16, 8) < 0.5).float()  # hide half of the 16 x 16 pixel blocks
mask = mask.repeat_interleave(16, 2).repeat_interleave(16, 3)
with torch.no_grad():
    reconstruction = model(frame * (1 - mask))

panels = {"Input": frame, "Masked": frame * (1 - mask), "Reconstruction": reconstruction}
figure, axes = plt.subplots(1, 3, figsize=(6, 4.4))
for axis, (title, image) in zip(axes, panels.items()):
    axis.imshow(image[0, 0], cmap="gray", vmin=0, vmax=1)
    axis.set_title(title)
    axis.axis("off")
figure.tight_layout()
figure.savefig("reconstruction.png", dpi=150)

Model

  • Residual encoder with block counts [3, 4, 6, 3] and base width 320 after 8 x 8 input patchification, batch normalization, and a bilinear U-Net decoder with skip connections; 671 million parameters.
  • Pretrained for 1,000 epochs with full-image L1 reconstruction and 50% block masking, then fine-tuned for 30 epochs together with auxiliary tasks (view, sector geometry, probe rotation, examination identity, ECG phase). The weights are the exponential moving average at the best validation step.
  • The auxiliary prediction heads are not included.

Files

File Content
model.pt Model weights (state dict, 2.7 GB)
model.py Model definition
sample.npy One beamspace frame, uint8, 256 x 128
example.py The code above

Training data and citation

Trained on EchoXFlow, a clinical echocardiography dataset in native beamspace. sample.npy is one apical four-chamber frame from its validation fold.

@misc{stenhede2026echoxflow,
  title  = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function},
  author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian},
  year   = {2026},
  eprint = {2605.05447},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

License

CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived. The model definition is adapted from the MAE in lambertae/drifting (MIT license).