Model card: PhysioWave-T name, training curves, links to the series

89a1369 verified 2 days ago

6.45 kB

---
license: apache-2.0
tags:
  - physiowave
  - physiowave-t
  - multimodal-fusion
  - multimodal
  - eeg
  - ecg
  - emg
  - seizure-detection
  - sensor-fusion
  - biosignal
library_name: pytorch
---
# PhysioWave-T-RALF: multimodal seizure detection on SeizeIT2 (EEG + ECG + EMG)

The **PhysioWave-T** multimodal model, built on the three released encoders ([EEG](https://huggingface.co/ForeverBlue/EEG), [ECG](https://huggingface.co/ForeverBlue/ECG), [sEMG](https://huggingface.co/ForeverBlue/EMG)). Code: <https://github.com/ForeverBlue816/PhysioWave>.

Three independently pretrained C1 encoders -- EEG, ECG and sEMG -- fused by
**RALF (Reliability-Aware Latent Fusion)** for
seizure detection on SeizeIT2 (behind-the-ear EEG, ECG and deltoid EMG).
Each encoder hands the fusion a few summary tokens; RALF predicts a
reliability per modality and sample, gates the cross-attention with it, masks
absent modalities out, and adds a residual of reliability-weighted unimodal
logits.

Encoders **frozen** (the pretrained cores fixed; montage adapters, pools,
fusion core and heads trained), 69.4 M parameters in all.
Run `ralf_frozen_szo`, selected on validation auprc at epoch 5
counted from 0 (validation auprc 0.380).
Classes: ['background', 'seizure'].

## Test results

SeizeIT2's official split: patients sub-097 to sub-125 are the test set and
were never seen before the final evaluation; 12% of the training patients
were held out for checkpoint selection.

Continuous test recordings (778 recordings, 2,912 h, 181 seizures), the SeizeIT2 paper's scoring (any-overlap, means over recordings):

| metric | value |
|---|---|
| sensitivity at <= 2 false alarms/h | 32.0 % |
| sensitivity at <= 6.22 false alarms/h | 58.1 % |
| sensitivity at <= 11 false alarms/h | 74.4 % |
| AUROC (the dataset paper's epoch-based area) | 0.741 |
| AUPR (the dataset paper's epoch-based area) | 0.0121 |

Window-level test set (4 s windows; background subsampled to 30 per hour):

| modalities present | AUPRC | AUROC |
|---|---|---|
| EEG + ECG + EMG | 0.399 | 0.773 |
| EEG + ECG | 0.394 | 0.768 |
| EEG + EMG | 0.265 | 0.691 |
| ECG + EMG | 0.337 | 0.746 |
| EEG | 0.252 | 0.692 |
| ECG | 0.313 | 0.746 |
| EMG | 0.192 | 0.615 |

Under corruption of one modality at severity 0.7,
on top of the complete set, with the reliability the model assigned to each
modality:

| condition | AUPRC | reliability EEG | ECG | EMG |
|---|---|---|---|---|
| clean | 0.399 | 0.91 | 0.85 | 0.91 |
| EEG noise | 0.368 | 0.63 | 0.85 | 0.91 |
| EEG bad channel | 0.358 | 0.29 | 0.85 | 0.91 |
| EEG drift | 0.332 | 0.32 | 0.85 | 0.91 |
| ECG saturation | 0.367 | 0.91 | 0.33 | 0.91 |
| ECG noise | 0.395 | 0.91 | 0.65 | 0.91 |
| EMG dropout | 0.403 | 0.91 | 0.85 | 0.29 |
| EMG noise | 0.398 | 0.91 | 0.85 | 0.24 |

A corrupted modality's reliability drops and the other two stay where they
were; that is the mechanism the fusion is built around.

## Training curves

![Checkpoint selection and loss terms](training_curves.png)

*(a) Validation AUPRC per epoch for the released run (bold) and for the five frozen-encoder RALF variants of the paper's schedule sweep; the dot on each curve is its selected epoch. The released checkpoint is epoch 5 of the original configuration, selected on validation AUPRC. (b) The released run's loss terms. The reliability loss holds at 0.04 through epoch 7, where the six-epoch warm-up reaches its peak learning rate; after that the reliability head drifts and the task loss follows, which is why selection stopped at epoch 5 and why the paper reports RALF as the band over all six runs rather than one curve. The variants at lr 1e-4 train stably for their full 15 epochs and select at epochs 2 to 5 with validation AUPRC 0.376 to 0.401.*

## Files

| file | what it is |
|---|---|
| `ralf_seizeit2_best.pth` | the model: encoders, adapters, fusion core and heads (epoch 5) |
| `results.json` | the test matrix: every modality subset and corruption |
| `events_cross.json` | event-level scoring on the continuous test recordings |
| `history.json` | training and validation per epoch |
| `encoders.json` | which pretrained keys each encoder took |
| `config_resolved.yaml` | the run's config; encoders point at the released files |

## The encoders it was built on

| modality | file | repository |
|---|---|---|
| EEG | `pretrain_eeg_c1_moe_best.pth` | ForeverBlue/EEG |
| ECG | `pretrain_ecg_c1_moe_best.pth` | ForeverBlue/ECG |
| EMG | `pretrain_emg_c1_moe_epoch38.pth` | ForeverBlue/EMG |

The weights here already contain all three encoders as they were loaded, so
they are not needed to RUN the model. They are needed to BUILD it:
`build_model` constructs each encoder from its pretraining checkpoint before
the fusion weights are loaded on top. Download the three files into one
directory and point `encoders.<m>` at them (the released config already
names them).

## Loading

The model's montage adapters and heads are sized by the task's data, so it is
rebuilt from the prepared windows (`Fusion/seizeit2_prep.py` in the code
repository) and then loaded strictly:

```python
import torch
from physiowave.config import load_config
from physiowave.fusion.train import build_model, load_tasks
cfg = load_config("config_resolved.yaml")
cfg["data"]["tasks"]["seizeit2"]["dir"] = "path/to/seizeit2"     # train.h5 / val.h5 / test.h5
cfg["encoders"] = {m: f"path/to/encoders/{f}" for m, f in cfg["encoders"].items()
                   if m in ("eeg", "ecg", "emg") and f}
tasks = load_tasks(cfg, list(cfg["data"]["modalities"]))
model, _ = build_model(cfg, tasks, out_dir="encoders_built")
ck = torch.load("ralf_seizeit2_best.pth", map_location="cpu", weights_only=False)
model.load_state_dict(ck["model"])     # strict: every key must match
model.eval()
```

Only `train.h5` is read to size the model; nothing is trained.

## Training

```
fusion      ralf, width 256, 8 fusion tokens, depth 2, heads 8
summary     4 tokens per modality
encoders    frozen
lr          0.0003, batch 32, epochs 60 (patience 10), bf16
loss        task CE + 0.2 x unimodal CE + 1.0 x reliability MSE + 1.0 x subset consistency
augment     each modality corrupted with p 0.5; modalities dropped with p 0.3
seed        42
```

One run, one seed. The numbers above are that run's, not a mean.

Code: <https://github.com/ForeverBlue816/PhysioWave> (`docs/fusion.md`)

---
Prepared by `scripts/prepare_hf_fusion_release.py` from `ralf_frozen_szo` on
2026-10-29.

	---
	license: apache-2.0
	tags:
	- physiowave
	- physiowave-t
	- multimodal-fusion
	- multimodal
	- eeg
	- ecg
	- emg
	- seizure-detection
	- sensor-fusion
	- biosignal
	library_name: pytorch
	---

	# PhysioWave-T-RALF: multimodal seizure detection on SeizeIT2 (EEG + ECG + EMG)

	The PhysioWave-T multimodal model, built on the three released encoders ([EEG](https://huggingface.co/ForeverBlue/EEG), [ECG](https://huggingface.co/ForeverBlue/ECG), [sEMG](https://huggingface.co/ForeverBlue/EMG)). Code: <https://github.com/ForeverBlue816/PhysioWave>.

	Three independently pretrained C1 encoders -- EEG, ECG and sEMG -- fused by
	RALF (Reliability-Aware Latent Fusion) for
	seizure detection on SeizeIT2 (behind-the-ear EEG, ECG and deltoid EMG).
	Each encoder hands the fusion a few summary tokens; RALF predicts a
	reliability per modality and sample, gates the cross-attention with it, masks
	absent modalities out, and adds a residual of reliability-weighted unimodal
	logits.

	Encoders frozen (the pretrained cores fixed; montage adapters, pools,
	fusion core and heads trained), 69.4 M parameters in all.
	Run `ralf_frozen_szo`, selected on validation auprc at epoch 5
	counted from 0 (validation auprc 0.380).
	Classes: ['background', 'seizure'].

	## Test results

	SeizeIT2's official split: patients sub-097 to sub-125 are the test set and
	were never seen before the final evaluation; 12% of the training patients
	were held out for checkpoint selection.

	Continuous test recordings (778 recordings, 2,912 h, 181 seizures), the SeizeIT2 paper's scoring (any-overlap, means over recordings):

	\| metric \| value \|
	\|---\|---\|
	\| sensitivity at <= 2 false alarms/h \| 32.0 % \|
	\| sensitivity at <= 6.22 false alarms/h \| 58.1 % \|
	\| sensitivity at <= 11 false alarms/h \| 74.4 % \|
	\| AUROC (the dataset paper's epoch-based area) \| 0.741 \|
	\| AUPR (the dataset paper's epoch-based area) \| 0.0121 \|

	Window-level test set (4 s windows; background subsampled to 30 per hour):

	\| modalities present \| AUPRC \| AUROC \|
	\|---\|---\|---\|
	\| EEG + ECG + EMG \| 0.399 \| 0.773 \|
	\| EEG + ECG \| 0.394 \| 0.768 \|
	\| EEG + EMG \| 0.265 \| 0.691 \|
	\| ECG + EMG \| 0.337 \| 0.746 \|
	\| EEG \| 0.252 \| 0.692 \|
	\| ECG \| 0.313 \| 0.746 \|
	\| EMG \| 0.192 \| 0.615 \|

	Under corruption of one modality at severity 0.7,
	on top of the complete set, with the reliability the model assigned to each
	modality:

	\| condition \| AUPRC \| reliability EEG \| ECG \| EMG \|
	\|---\|---\|---\|---\|---\|
	\| clean \| 0.399 \| 0.91 \| 0.85 \| 0.91 \|
	\| EEG noise \| 0.368 \| 0.63 \| 0.85 \| 0.91 \|
	\| EEG bad channel \| 0.358 \| 0.29 \| 0.85 \| 0.91 \|
	\| EEG drift \| 0.332 \| 0.32 \| 0.85 \| 0.91 \|
	\| ECG saturation \| 0.367 \| 0.91 \| 0.33 \| 0.91 \|
	\| ECG noise \| 0.395 \| 0.91 \| 0.65 \| 0.91 \|
	\| EMG dropout \| 0.403 \| 0.91 \| 0.85 \| 0.29 \|
	\| EMG noise \| 0.398 \| 0.91 \| 0.85 \| 0.24 \|

	A corrupted modality's reliability drops and the other two stay where they
	were; that is the mechanism the fusion is built around.

	## Training curves

	![Checkpoint selection and loss terms](training_curves.png)

	(a) Validation AUPRC per epoch for the released run (bold) and for the five frozen-encoder RALF variants of the paper's schedule sweep; the dot on each curve is its selected epoch. The released checkpoint is epoch 5 of the original configuration, selected on validation AUPRC. (b) The released run's loss terms. The reliability loss holds at 0.04 through epoch 7, where the six-epoch warm-up reaches its peak learning rate; after that the reliability head drifts and the task loss follows, which is why selection stopped at epoch 5 and why the paper reports RALF as the band over all six runs rather than one curve. The variants at lr 1e-4 train stably for their full 15 epochs and select at epochs 2 to 5 with validation AUPRC 0.376 to 0.401.

	## Files

	\| file \| what it is \|
	\|---\|---\|
	\| `ralf_seizeit2_best.pth` \| the model: encoders, adapters, fusion core and heads (epoch 5) \|
	\| `results.json` \| the test matrix: every modality subset and corruption \|
	\| `events_cross.json` \| event-level scoring on the continuous test recordings \|
	\| `history.json` \| training and validation per epoch \|
	\| `encoders.json` \| which pretrained keys each encoder took \|
	\| `config_resolved.yaml` \| the run's config; encoders point at the released files \|

	## The encoders it was built on

	\| modality \| file \| repository \|
	\|---\|---\|---\|
	\| EEG \| `pretrain_eeg_c1_moe_best.pth` \| ForeverBlue/EEG \|
	\| ECG \| `pretrain_ecg_c1_moe_best.pth` \| ForeverBlue/ECG \|
	\| EMG \| `pretrain_emg_c1_moe_epoch38.pth` \| ForeverBlue/EMG \|

	The weights here already contain all three encoders as they were loaded, so
	they are not needed to RUN the model. They are needed to BUILD it:
	`build_model` constructs each encoder from its pretraining checkpoint before
	the fusion weights are loaded on top. Download the three files into one
	directory and point `encoders.<m>` at them (the released config already
	names them).

	## Loading

	The model's montage adapters and heads are sized by the task's data, so it is
	rebuilt from the prepared windows (`Fusion/seizeit2_prep.py` in the code
	repository) and then loaded strictly:

	```python
	import torch
	from physiowave.config import load_config
	from physiowave.fusion.train import build_model, load_tasks

	cfg = load_config("config_resolved.yaml")
	cfg["data"]["tasks"]["seizeit2"]["dir"] = "path/to/seizeit2" # train.h5 / val.h5 / test.h5
	cfg["encoders"] = {m: f"path/to/encoders/{f}" for m, f in cfg["encoders"].items()
	if m in ("eeg", "ecg", "emg") and f}

	tasks = load_tasks(cfg, list(cfg["data"]["modalities"]))
	model, _ = build_model(cfg, tasks, out_dir="encoders_built")
	ck = torch.load("ralf_seizeit2_best.pth", map_location="cpu", weights_only=False)
	model.load_state_dict(ck["model"]) # strict: every key must match
	model.eval()
	```

	Only `train.h5` is read to size the model; nothing is trained.

	## Training

	```
	fusion ralf, width 256, 8 fusion tokens, depth 2, heads 8
	summary 4 tokens per modality
	encoders frozen
	lr 0.0003, batch 32, epochs 60 (patience 10), bf16
	loss task CE + 0.2 x unimodal CE + 1.0 x reliability MSE + 1.0 x subset consistency
	augment each modality corrupted with p 0.5; modalities dropped with p 0.3
	seed 42
	```

	One run, one seed. The numbers above are that run's, not a mean.

	Code: <https://github.com/ForeverBlue816/PhysioWave> (`docs/fusion.md`)

	---
	Prepared by `scripts/prepare_hf_fusion_release.py` from `ralf_frozen_szo` on
	2026-10-29.