Walrus-Heart

Walrus (1.3 B-parameter physics foundation model, Polymathic AI) fine-tuned to predict electrical waves in heart tissue: normal beats, premature beats and the spiral-wave re-entry behind ventricular tachycardia, on healthy, fibrotic and scarred tissue. Trained on Ethgar/cardiac-ep-2d.

▶ Interactive physics playground: Ethgar/heart-waves · 💻 Code: github.com/edgarmoreaualix/heart-waves

Checkpoints

folder init training runs test one-step VRMSE (voltage / recovery / mean)
pretrained_n1600/ Walrus 1,600 0.181 / 0.126 / 0.154
pretrained_n200/ Walrus 200 0.163 / 0.127 / 0.145
scratch_n1600/ random (same architecture) 1,600 0.164 / 0.196 / 0.180
persistence (repeat last frame) 0.592 / 0.535 / 0.563

Each folder holds model.safetensors, model_config.yaml (the Walrus Hydra config), field_to_index_map.json, metrics.json (full training/eval log), and test rollouts (rollouts_per_traj.json, rollouts_video.npz).

Training

  • Inputs: 6 past frames of voltage, recovery (predicted) + conductivity (constant tissue map, 0 = scar), 128 × 128, padded to 3D with a singleton z (walls in x, y). Three new field embeddings are added to Walrus' field index map (67, 68, 69).
  • Walrus' own recipe: RMS sample-wise reversible normalisation, delta prediction, causal in time, loss = 100 × MAE, AdamW (lr 1e-4, wd 1e-4), linear warmup + cosine, grad clip 10, learnable RoPE, bf16.
  • 1,200 steps, batch 2, one A100 80 GB, ~53 min of training + ~37 min of evaluation per run.
  • Windows whose targets contain an external stimulus (S1/S2 pacing) are excluded: they are not predictable from the past.
  • Script: walrus/walrus_heart.py.

Results and limitations

  • Pretraining transfers. Walrus beats the same architecture trained from scratch, even with 8× less heart data (200 vs 1,600 runs). The gain is on the slow recovery variable, not on the sharp voltage fronts.
  • Long rollouts fail. Fed its own predictions, every checkpoint drifts into high-frequency stripes after 10–20 steps (0.4–0.8 s of heart time); voltage RMSE at 1 / 5 / 20 steps: 0.14 / 0.26 / 0.41 (Walrus 1,600). Whether an arrhythmia sustains itself after 6 s is called at chance level: Walrus 52 % (it predicts re-entry almost always: 24 TP, 23 FP, 1 TN, 0 FN; n = 48), from scratch 42 %.
  • These are short v0 runs (1,200 steps). Longer training, multi-step / noise-injection training and physical clamping at rollout are the obvious next steps.
  • 2D isotropic sheet, phenomenological Aliev–Panfilov model, simulated data only. Research and teaching use, not for clinical use.

Loading

# pip install "walrus @ git+https://github.com/PolymathicAI/walrus@73f2cf6b4c5871c8f5434802725b55666c66b5ab" huggingface_hub safetensors
import json
from huggingface_hub import hf_hub_download
from hydra.utils import instantiate
from omegaconf import OmegaConf
from safetensors.torch import load_file

run = "pretrained_n200"
cfg = OmegaConf.load(hf_hub_download("Ethgar/walrus-heart", f"{run}/model_config.yaml"))
fmap = json.load(open(hf_hub_download("Ethgar/walrus-heart", f"{run}/field_to_index_map.json")))
model = instantiate(cfg.model, n_states=max(fmap.values()) + 1)
model.add_ft_options({"learnable_rope": True, "rope_per_axis": True})
model.load_state_dict(load_file(hf_hub_download("Ethgar/walrus-heart", f"{run}/model.safetensors")))

See walrus_heart.py for the exact input formatting (Walrus formatter, normalisation and rollout loop).

License

MIT, like the base model polymathic-ai/walrus. Dataset: CC BY 4.0.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for edgarmoreau/walrus-heart

Finetuned
(1)
this model

Dataset used to train edgarmoreau/walrus-heart

Space using edgarmoreau/walrus-heart 1