--- license: cc-by-nc-sa-4.0 datasets: [Ahus-AIM/EchoXFlow] library_name: pytorch tags: [echocardiography, ultrasound, masked-autoencoder, 3d] --- # 3D echocardiography masked autoencoder Transformer masked autoencoder for single 3D echocardiography volumes in native beamspace (radius x azimuth x elevation = 256 x 64 x 64, values in [0, 1]). It reconstructs the volume from the 25% of its 16 x 8 x 8 voxel patches that stay visible; the figure shows the central elevation slice. ![Input, masked input and reconstruction](reconstruction.png) ## Usage Requires `torch` and `matplotlib`. Download the repository and run the code below from inside it (it is also saved as `example.py`). It loads the model and the sample, hides part of the input, and saves the figure above. ```python import matplotlib.pyplot as plt import numpy as np import torch from model import EchoMAE3D model = EchoMAE3D() model.load_state_dict(torch.load("model.pt", weights_only=True)) model.eval() volume = torch.from_numpy(np.load("sample.npy")).float()[None, None] / 255 # 1 x 1 x 256 x 64 x 64 torch.manual_seed(0) with torch.no_grad(): reconstruction, mask = model(volume) # hides 75% of the 16 x 8 x 8 voxel patches panels = {"Input": volume, "Masked": volume * (1 - mask), "Reconstruction": reconstruction} figure, axes = plt.subplots(1, 3, figsize=(6, 4.4)) for axis, (title, image) in zip(axes, panels.items()): axis.imshow(image[0, 0, :, :, 32], cmap="gray", vmin=0, vmax=1, aspect=0.5) # central elevation slice axis.set_title(title) axis.axis("off") figure.tight_layout() figure.savefig("reconstruction.png", dpi=150) ``` ## Model - 17.2 million parameters; 1,024 patch tokens, 75% random masking. - Encoder: 8 blocks, width 384, 6 heads. Decoder: 4 blocks, width 192, 6 heads. - Trained with L1 reconstruction over all voxels (AdamW). The weights are the checkpoint with the lowest validation L1 (0.063). - `model.encode(volume, keep)` returns the latent tokens of the selected patches. ## Files | File | Content | |---|---| | `model.pt` | Model weights (state dict, 69 MB) | | `model.py` | Model definition | | `sample.npy` | One beamspace volume, `uint8`, 256 x 64 x 64 | | `example.py` | The code above | ## Training data and citation Trained on [EchoXFlow](https://huggingface.co/datasets/Ahus-AIM/EchoXFlow), a clinical echocardiography dataset in native beamspace. `sample.npy` is one volume from its validation fold. - Paper: [EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function](https://arxiv.org/abs/2605.05447) - Dataset: https://huggingface.co/datasets/Ahus-AIM/EchoXFlow - Tooling: https://github.com/Ahus-AIM/EchoXFlow ```bibtex @misc{stenhede2026echoxflow, title = {EchoXFlow: A Beamspace Echocardiography Dataset for Cardiac Motion, Flow, and Function}, author = {Stenhede, Elias and Sulkowska, Joanna and Orstad, Eivind Bj{\o}rkan and Schirmer, Henrik and Ranjbar, Arian}, year = {2026}, eprint = {2605.05447}, archivePrefix = {arXiv}, primaryClass = {cs.CV} } ``` ## License CC BY-NC-SA 4.0, the same license as EchoXFlow, from which the weights and the sample are derived.