PyTorch
echocardiography
ultrasound
masked-autoencoder
3d
echo-mae-3d / README.md
eliasstenhede's picture
Model weights, code, sample and model card
c9e97e7 verified
|
Raw History Blame Contribute Delete
3.17 kB
metadata
license: cc-by-nc-sa-4.0
datasets:
  - Ahus-AIM/EchoXFlow
library_name: pytorch
tags:
  - echocardiography
  - ultrasound
  - masked-autoencoder
  - 3d

3D echocardiography masked autoencoder

Transformer masked autoencoder for single 3D echocardiography volumes in native beamspace (radius x azimuth x elevation = 256 x 64 x 64, values in [0, 1]). It reconstructs the volume from the 25% of its 16 x 8 x 8 voxel patches that stay visible; the figure shows the central elevation slice.

Input, masked input and reconstruction

Usage

Requires torch and matplotlib. Download the repository and run the code below from inside it (it is also saved as example.py). It loads the model and the sample, hides part of the input, and saves the figure above.

import matplotlib.pyplot as plt
import numpy as np
import torch

from model import EchoMAE3D

model = EchoMAE3D()
model.load_state_dict(torch.load("model.pt", weights_only=True))
model.eval()

volume = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255  # 1 x 1 x 256 x 64 x 64
torch.manual_seed(0)
with torch.no_grad():
    reconstruction, mask = model(volume)  # hides 75% of the 16 x 8 x 8 voxel patches

panels = {"Input": volume, "Masked": volume * (1 - mask), "Reconstruction": reconstruction}
figure, axes = plt.subplots(1, 3, figsize=(6, 4.4))
for axis, (title, image) in zip(axes, panels.items()):
    axis.imshow(image[0, 0, :, :, 32], cmap="gray", vmin=0, vmax=1, aspect=0.5)  # central elevation slice
    axis.set_title(title)
    axis.axis("off")
figure.tight_layout()
figure.savefig("reconstruction.png", dpi=150)

Model

  • 17.2 million parameters; 1,024 patch tokens, 75% random masking.
  • Encoder: 8 blocks, width 384, 6 heads. Decoder: 4 blocks, width 192, 6 heads.
  • Trained with L1 reconstruction over all voxels (AdamW). The weights are the checkpoint with the lowest validation L1 (0.063).
  • model.encode(volume, keep) returns the latent tokens of the selected patches.

Files

File Content
model.pt Model weights (state dict, 69 MB)
model.py Model definition
sample.npy One beamspace volume, uint8, 256 x 64 x 64
example.py The code above

Training data and citation

Trained on EchoXFlow, a clinical echocardiography dataset in native beamspace. sample.npy is one volume from its validation fold.

@misc{stenhede2026echoxflow,
  title  = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function},
  author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian},
  year   = {2026},
  eprint = {2605.05447},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV}
}

License

CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived.