Upload folder using huggingface_hub

README.md

@@ -0,0 +1,213 @@
+---
+license: apache-2.0
+tags:
+  - ecg
+  - electrocardiogram
+  - biosignal
+  - self-supervised
+  - masked-autoencoder
+  - pretrained
+library_name: pytorch
+---
+
+# ECG C1 multi-route pretrained encoder
+
+Masked dual-reconstruction pretraining on 12-lead and single-lead ECG. One shared transformer serves two lead configurations through route-specific wavelet frontends, and lead identity enters as a learned embedding of the lead NAME rather than of its index.
+
+Trained for 57 epochs of 60 configured
+(57000 optimizer updates) on a mixture of public
+ECG corpora, 30.1 M parameters. The files
+here are built around `pretrain_ecg_c1_moe_best.pth`, epoch 33.
+
+## Objective
+
+```
+0.5*L_spec + 0.5*L_raw + 0.001*L_foldKL
+```
+
+Two heads at equal weight. One predicts folded wavelet patches, the other
+predicts the preprocessed ECG waveform of the same masked patches.
+Masking is applied BEFORE the wavelet frontend (True),
+at ratio 0.7, and the spec target is per-patch
+normalised (True).
+
+## Validation, at the released checkpoint (epoch 33)
+
+| metric | value | what 1.0 would mean |
+|---|---|---|
+| total loss | 0.1827 | — |
+| spec MSE | 0.2562 | — |
+| spec NMSE | 0.256 | predicting zero |
+| spec Pearson r | 0.851 | perfect |
+| raw SmoothL1 | 0.1092 | — |
+| raw NMSE | 0.255 | predicting zero |
+| raw Pearson r | 0.845 | perfect |
+| route-macro total | 0.2041 | — |
+
+NMSE is dimensionless: MSE(prediction, target) / MSE(0, target), so 1.0 is
+what predicting zero scores on that head's own target. The two losses are NOT
+comparable to each other -- one is an MSE on normalised wavelet coefficients,
+the other a SmoothL1 on the window-normalised waveform. The columns that compare the
+heads are the correlations and the NMSEs.
+
+The route-macro row weights the two routes equally. The plain rows weight by
+sample count, and validation is dominated by the largest corpora, so a model
+specialising toward the smaller ones moves the two rows in opposite
+directions.
+
+## Files
+
+### Which file do you want
+
+| your situation | download |
+|---|---|
+| fine-tune on any montage | `ecg_c1_encoder.pth` |
+| your montage IS one of the two routes below | `pretrain_ecg_c1_moe_best.pth`, then cut the route out (one command, below) |
+| continue pretraining, or take the model apart yourself | `pretrain_ecg_c1_moe_best.pth` |
+
+**`pretrain_ecg_c1_moe_best.pth` is the complete model** -- all two wavelet frontends, the
+patchers, both reconstruction decoders. Nothing is held back; only the
+optimiser state was stripped, which no downstream use needs. Take it if you
+want to keep or strip pieces yourself.
+
+`ecg_c1_encoder.pth` is the same weights minus everything route-bound and
+pretraining-only. It exists because it cannot be loaded wrong.
+
+| file | what it is |
+|---|---|
+| `ecg_c1_encoder.pth` | encoder alone: channel embedding, gate, transformer |
+| `pretrain_ecg_c1_moe_best.pth` | complete model, lowest validation **total** loss (epoch 33) |
+| `pretrain_ecg_c1_moe_final.pth` | complete model, last epoch (epoch 57) |
+| `pretrain_ecg_c1_moe_best_spec.pth` | complete model, lowest spec MSE (epoch 19) |
+| `pretrain_ecg_c1_moe_best_raw.pth` | complete model, lowest raw SmoothL1 (epoch 37) |
+| `pretrain_ecg_c1_moe_best_macro.pth` | complete model, lowest route-macro total (epoch 28) |
+| `metrics_epoch.jsonl` | the full training curve |
+| `config_resolved.yaml` | every resolved hyperparameter |
+
+### Why several checkpoints
+
+Each is the minimum of a different validation metric, and they are published
+together so a downstream comparison can ask which criterion transfers instead
+of assuming the one that names `best.pth`. Reconstruction loss cannot answer
+that question; a downstream task can.
+
+`best.pth` is selected on the TOTAL loss -- the loss that was trained (epoch
+32 counted from 0). It is not the best spec loss, and not
+the best raw loss.
+
+### What is in the complete model
+
+| component | parameters | share | |
+|---|---|---|---|
+| shared transformer | 28.38 M | 94.3% | encoder, always transfers |
+| channel-name embedding | 0.00 M | 0.0% | always transfers |
+| channel projection | 0.03 M | 0.1% | always transfers |
+| wavelet frontends (all two routes) | 0.03 M | 0.1% | route-bound; downstream usually builds its own |
+| patchers (both rates) | 0.20 M | 0.6% | route-bound |
+| spec decoder | 0.72 M | 2.4% | pretraining only |
+| raw decoder | 0.72 M | 2.4% | pretraining only |
+| **total** | **30.08 M** | 100% | |
+
+The shared transformer is 94% of the model and the two frontends together are 0%, which is why there is no per-route encoder file here: two of them would duplicate that transformer two times to deliver frontends that are a small fraction of it. Cut the one you want out of the complete checkpoint instead.
+
+### What transfers, and what does not
+
+`downstream.py` splits a checkpoint into two sets. The channel embedding, its
+gate and the shared transformer **always** load. The wavelet frontend and the
+patcher load **only when your montage is the route they were trained for** --
+same channel count, same patch length -- and are skipped otherwise rather
+than reshaped.
+
+A 12-lead recording at 500 Hz in 10 s windows IS the `L12_500` route, so on the most common ECG input the pretrained frontend and patcher transfer as well: cut that route out (below) and pass `route_id="L12_500"`. Any other lead set, rate or window length builds its own.
+
+The two routes the shared transformer was pretrained across:
+
+| route | channels | rate |
+|---|---|---|
+| `L12_500` | 12 | 500 Hz |
+| `L1_250` | 1 | 250 Hz |
+
+To fine-tune ON one of them exactly, take that route's frontend and patcher out
+of the complete checkpoint:
+
+```bash
+python scripts/export_eeg_pretrained_encoder.py \
+    --checkpoint pretrain_ecg_c1_moe_best.pth --route L12_500 \
+    --output encoder_L12_500.pth
+```
+
+The exporter reads the modality from the checkpoint; its name is historical.
+Omit `--route` and you get `ecg_c1_encoder.pth` again. Both pretraining decoders are
+dropped either way: fine-tuning against the pretext objective by accident is
+the failure that prevents.
+
+The released checkpoint and the last epoch differ by 2.6% in validation total loss (0.1827 against 0.1877). Which transfers better is a downstream question, not a reconstruction-loss question -- measure both.
+
+## The channel vocabulary travels with the weights
+
+An embedding row means whatever channel held that id when the row was learned.
+Every checkpoint here carries the vocabulary and its hash. Resolving
+lead names against a different vocabulary silently trains on relabelled
+channels, which is why the hash is checked rather than assumed.
+
+## Architecture
+
+```
+embed_dim   512      depth 9      heads 8
+norm        rmsnorm     ffn swiglu     qk_norm True
+wavelet     max_level 3, kernel 16, bank db4, db6, sym4, sym8, coif2
+channel     id embedding, injected at the token site
+mask        frequency_guided, ratio 0.7
+```
+
+## Loading
+
+`ECGC1Downstream` builds the frontend and patcher for your montage and loads this
+file into everything else. Nothing here is route-bound, so the montage, the
+window length and the patch length are yours to choose.
+
+```python
+import torch
+from physiowave.ecg_c1.downstream import ECGC1Downstream
+
+ENC = "ecg_c1_encoder.pth"
+ck = torch.load(ENC, map_location="cpu", weights_only=False)
+mc = ck["model_config"]                  # the width the weights were trained at
+CH = ["I", "II", "III", "aVR", "aVL", "aVF",
+      "V1", "V2", "V3", "V4", "V5", "V6"]   # your leads
+
+model = ECGC1Downstream(
+    in_channels=len(CH), window_samples=5000, sampling_rate=500.0,
+    patch_samples=250, num_classes=2, channel_names=CH,
+    embed_dim=mc["embed_dim"], depth=mc["depth"], num_heads=mc["num_heads"],
+    channel_embed_dim=mc.get("channel_embed_dim", 64),
+    channel_vocab_size=ck["channel_vocab_size"])
+
+report = model.load_pretrained(ENC)
+print(model.describe_transfer(report))
+# loaded: transformer N, channel encoder N, gate yes, frontend 0, patcher 0
+
+class Meta:                               # or your dataset's ChannelMeta
+    channel_names = CH
+    channel_mask = None
+
+out = model(torch.randn(2, len(CH), 5000), Meta())   # {"logits": [2, 2]}
+```
+
+`frontend 0, patcher 0` is the expected line, not a warning: those are the two
+things your montage supplies.
+
+**The frontend you build is not randomly initialised.** Its filters are seeded
+from real wavelet families -- `db4, db6, sym4, sym8, coif2` -- and learned from there during
+fine-tuning. What a fresh frontend costs you is the adaptation those filters
+underwent during pretraining, not the wavelet prior itself.
+
+Names, not indices, are what this encoder and your data agree on. Pass real
+lead names and check `channel_vocab_sha256` -- `load_pretrained` refuses a
+mismatch rather than silently training on relabelled channels.
+
+Code: <https://github.com/ForeverBlue816/PhysioWave-T>
+
+---
+Prepared by `scripts/prepare_hf_release.py` from `pretrain_ecg_c1_moe`
+on 2026-10-29.

config_resolved.yaml

@@ -0,0 +1,61 @@
+data:
+  manifest_train: <path redacted for release>
+  manifest_val: <path redacted for release>
+  modality: ecg
+  weights: temperature:0.5
+model:
+  channel_embed_dim: 64
+  channel_encoding: id
+  channel_injection: token
+  channel_token_gate_init: 0.0
+  depth: 9
+  dropout: 0.0
+  embed_dim: 512
+  ffn: swiglu
+  fold_gamma: 0.1
+  fold_synthesis: 3
+  importance_ratio: 0.6
+  lead_group_masking: true
+  mask_ratio: 0.7
+  masking_strategy: frequency_guided
+  max_level: 3
+  mlp_ratio: 4.0
+  norm: rmsnorm
+  num_heads: 8
+  qk_norm: true
+  scale_fold: dynamic
+  use_separate_channel: true
+  wave_init_mode: pad
+  wave_kernel_size: 16
+  wavelet_names:
+  - db4
+  - db6
+  - sym4
+  - sym8
+  - coif2
+objective:
+  fold_kl: 0.001
+  mask_before_frontend: true
+  normalize_spec_target: true
+  raw_beta: 0.5
+  raw_weight: 0.5
+  spec_weight: 0.5
+output_dir: <path redacted for release>
+run_name: pretrain_ecg_c1_moe
+seed: 42
+train:
+  batch_size_by_route:
+    L12_500: 96
+    L1_250: 1024
+  clip_grad_norm: 1.0
+  epochs: 60
+  grad_accumulation_steps: 1
+  lr: 0.0003
+  min_lr_ratio: 0.01
+  precision: bf16
+  steps_per_epoch: 1000
+  val_mask_seed: 1234
+  val_max_batches_per_dataset: 100
+  warmup_epochs: 2
+  weight_decay: 0.05
+trainer: ecg_c1_moe

ecg_c1_encoder.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6f8b22f458c2de48520e228d41a094c0498ac77850aa3ec450a901fca2a8edad
+size 113689120

pretrain_ecg_c1_moe_best.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8a8431de5d302277f9985e59b1c2c6c11bde73715279672cd6712ff127c3c408
+size 120442759

pretrain_ecg_c1_moe_best_macro.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9c758845d0367f170b825f9757c54045f08ee09f86ae035ece60fd9197040980
+size 120444633

pretrain_ecg_c1_moe_best_raw.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2e3dc6163d1a64fca29ccc1e7393b56165ddc7be5c915c1e9267525c3ac575a5
+size 120443923

pretrain_ecg_c1_moe_best_spec.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:86a5d10270f20abf7ead05ab297b05cf10fefed8cc012b2c63a74d8189398849
+size 120444214

pretrain_ecg_c1_moe_final.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5236c2f7a0f7db3d9fe1316e52f10198416f8f3b4738035fe3f5657ee8f78587
+size 120443050