PyTorch
echocardiography
ultrasound
masked-autoencoder
3d

3D echocardiography masked autoencoder

Transformer masked autoencoder for single 3D echocardiography volumes in native beamspace (radius x azimuth x elevation = 256 x 64 x 64, values in [0, 1]). It reconstructs the volume from the 25% of its 16 x 8 x 8 voxel patches that stay visible; the figure shows the central elevation slice.

Input, masked input and reconstruction

Usage

Requires torch and matplotlib. Download the repository and run the code below from inside it (it is also saved as example.py). It loads the model and the sample, hides part of the input, and saves the figure above.

import matplotlib.pyplot as plt
import numpy as np
import torch

from model import EchoMAE3D

model = EchoMAE3D()
model.load_state_dict(torch.load("model.pt", weights_only=True))
model.eval()

volume = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255  # 1 x 1 x 256 x 64 x 64
torch.manual_seed(0)
with torch.no_grad():
    reconstruction, mask = model(volume)  # hides 75% of the 16 x 8 x 8 voxel patches

panels = {"Input": volume, "Masked": volume * (1 - mask), "Reconstruction": reconstruction}
figure, axes = plt.subplots(1, 3, figsize=(6, 4.4))
for axis, (title, image) in zip(axes, panels.items()):
    axis.imshow(image[0, 0, :, :, 32], cmap="gray", vmin=0, vmax=1, aspect=0.5)  # central elevation slice
    axis.set_title(title)
    axis.axis("off")
figure.tight_layout()
figure.savefig("reconstruction.png", dpi=150)

Model

  • 17.2 million parameters; 1,024 patch tokens, 75% random masking.
  • Encoder: 8 blocks, width 384, 6 heads. Decoder: 4 blocks, width 192, 6 heads.
  • Trained with L1 reconstruction over all voxels (AdamW). The weights are the checkpoint with the lowest validation L1 (0.063).
  • model.encode(volume, keep) returns the latent tokens of the selected patches.

Files

File Content
model.pt Model weights (state dict, 69 MB)
model.py Model definition
sample.npy One beamspace volume, uint8, 256 x 64 x 64
example.py The code above

Training data and citation

Trained on EchoXFlow, a clinical echocardiography dataset in native beamspace. sample.npy is one volume from its validation fold.

@misc{stenhede2026echoxflow,
  title  = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function},
  author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian},
  year   = {2026},
  eprint = {2605.05447},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

License

CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Dataset used to train Ahus-AIM/echo-mae-3d

Paper for Ahus-AIM/echo-mae-3d