Upload folder using huggingface_hub

README.md

@@ -0,0 +1,218 @@
+---
+license: apache-2.0
+tags:
+  - emg
+  - semg
+  - electromyography
+  - biosignal
+  - self-supervised
+  - masked-autoencoder
+  - pretrained
+library_name: pytorch
+---
+
+# sEMG C1 multi-route pretrained encoder
+
+Masked dual-reconstruction pretraining on surface EMG from wristbands, forearm arrays and high-density grids. One shared transformer serves three electrode layouts through route-specific wavelet frontends, and electrode identity enters as a learned embedding of the electrode NAME rather than of its index.
+
+Trained for 45 epochs of 60 configured; validation was logged for the first 39, which is what `metrics_epoch.jsonl` holds
+(the last logged validation at step 39000) on a mixture of public
+sEMG corpora, 12.1 M parameters. The files
+here are built around `pretrain_emg_c1_moe_epoch38.pth`, epoch 39.
+
+## Objective
+
+```
+0.5*L_spec + 0.5*L_raw + 0.001*L_foldKL
+```
+
+Two heads at equal weight. One predicts folded wavelet patches, the other
+predicts the preprocessed sEMG waveform of the same masked patches.
+Masking is applied BEFORE the wavelet frontend (True),
+at ratio 0.7, and the spec target is per-patch
+normalised (True).
+
+## Validation, at the released checkpoint (epoch 39)
+
+| metric | value | what 1.0 would mean |
+|---|---|---|
+| total loss | 0.2580 | — |
+| spec MSE | 0.3207 | — |
+| spec NMSE | 0.321 | predicting zero |
+| spec Pearson r | 0.822 | perfect |
+| raw SmoothL1 | 0.1953 | — |
+| raw NMSE | 0.381 | predicting zero |
+| raw Pearson r | 0.782 | perfect |
+| route-macro total | 0.2315 | — |
+
+NMSE is dimensionless: MSE(prediction, target) / MSE(0, target), so 1.0 is
+what predicting zero scores on that head's own target. The two losses are NOT
+comparable to each other -- one is an MSE on normalised wavelet coefficients,
+the other a SmoothL1 on the window-normalised waveform. The columns that compare the
+heads are the correlations and the NMSEs.
+
+The route-macro row weights the three routes equally. The plain rows weight by
+sample count, and validation is dominated by the largest corpora, so a model
+specialising toward the smaller ones moves the two rows in opposite
+directions.
+
+## Files
+
+### Which file do you want
+
+| your situation | download |
+|---|---|
+| fine-tune on any montage | `emg_c1_encoder.pth` |
+| your montage IS one of the three routes below | `pretrain_emg_c1_moe_epoch38.pth`, then cut the route out (one command, below) |
+| continue pretraining, or take the model apart yourself | `pretrain_emg_c1_moe_epoch38.pth` |
+
+**`pretrain_emg_c1_moe_epoch38.pth` is the complete model** -- all three wavelet frontends, the
+patchers, both reconstruction decoders. Nothing is held back; only the
+optimiser state was stripped, which no downstream use needs. Take it if you
+want to keep or strip pieces yourself.
+
+`emg_c1_encoder.pth` is the same weights minus everything route-bound and
+pretraining-only. It exists because it cannot be loaded wrong.
+
+| file | what it is |
+|---|---|
+| `emg_c1_encoder.pth` | encoder alone: channel embedding, gate, transformer |
+| `pretrain_emg_c1_moe_epoch38.pth` | complete model, the reported checkpoint (epoch 39) |
+| `pretrain_emg_c1_moe_final.pth` | complete model, last epoch (epoch 45) |
+| `pretrain_emg_c1_moe_best.pth` | complete model, lowest validation total loss (epoch 7) |
+| `pretrain_emg_c1_moe_best_macro.pth` | complete model, lowest route-macro total (epoch 25) |
+| `metrics_epoch.jsonl` | the full training curve |
+| `config_resolved.yaml` | every resolved hyperparameter |
+
+### Why several checkpoints
+
+Each is the minimum of a different validation metric, and they are published
+together so a downstream comparison can ask which criterion transfers instead
+of assuming the one that names `best.pth`. Reconstruction loss cannot answer
+that question; a downstream task can.
+
+`best.pth` is selected on the TOTAL loss -- the loss that was trained (epoch
+6 counted from 0). It is not the best spec loss, and not
+the best raw loss.
+
+**Why `pretrain_emg_c1_moe_epoch38.pth` is the main file.** It has the lowest validation raw loss; best.pth wins the total loss at epoch 7, before the raw head had converged (raw r 0.740 against 0.782). The paper's sEMG downstream and fusion results use this file.
+
+`pretrain_emg_c1_moe_best_spec.pth` would have been byte-identical to `pretrain_emg_c1_moe_best.pth` -- the same epoch won both bars -- so it is not published separately.
+`pretrain_emg_c1_moe_best_raw.pth` would have been byte-identical to `pretrain_emg_c1_moe_epoch38.pth` -- the same epoch won both bars -- so it is not published separately.
+
+### What is in the complete model
+
+| component | parameters | share | |
+|---|---|---|---|
+| shared transformer | 10.63 M | 88.2% | encoder, always transfers |
+| channel-name embedding | 0.02 M | 0.1% | always transfers |
+| channel projection | 0.02 M | 0.2% | always transfers |
+| wavelet frontends (all three routes) | 0.79 M | 6.6% | route-bound; downstream usually builds its own |
+| patchers (one rate) | 0.10 M | 0.8% | route-bound |
+| spec decoder | 0.24 M | 2.0% | pretraining only |
+| raw decoder | 0.24 M | 2.0% | pretraining only |
+| **total** | **12.05 M** | 100% | |
+
+The shared transformer is 88% of the model and the three frontends together are 7%, which is why there is no per-route encoder file here: three of them would duplicate that transformer three times to deliver frontends that are a small fraction of it. Cut the one you want out of the complete checkpoint instead.
+
+### What transfers, and what does not
+
+`downstream.py` splits a checkpoint into two sets. The channel embedding, its
+gate and the shared transformer **always** load. The wavelet frontend and the
+patcher load **only when your montage is the route they were trained for** --
+same channel count, same patch length -- and are skipped otherwise rather
+than reshaped.
+
+The usual sEMG benchmark montages -- armbands, forearm rings, NinaPro's electrodes -- are none of these routes, so a fresh frontend is the normal case. Their electrode names are outside the vocabulary too. Each unknown name gets an embedding row of its own, initialised from the pretrained `<unk>` row and trained as a montage adapter; `new_channel_rows=False` keeps the one shared `<unk>` vector instead, and channels are then told apart by the position encoding alone.
+
+The three routes the shared transformer was pretrained across:
+
+| route | channels | rate |
+|---|---|---|
+| `A24_2000` | 24 | 2000 Hz |
+| `G64_2000` | 64 | 2000 Hz |
+| `W16_2000` | 16 | 2000 Hz |
+
+To fine-tune ON one of them exactly, take that route's frontend and patcher out
+of the complete checkpoint:
+
+```bash
+python scripts/export_eeg_pretrained_encoder.py \
+    --checkpoint pretrain_emg_c1_moe_epoch38.pth --route G64_2000 \
+    --output encoder_G64_2000.pth
+```
+
+The exporter reads the modality from the checkpoint; its name is historical.
+Omit `--route` and you get `emg_c1_encoder.pth` again. Both pretraining decoders are
+dropped either way: fine-tuning against the pretext objective by accident is
+the failure that prevents.
+
+The last checkpoint (epoch 45) has no logged validation row, so it cannot be compared with the released one here. Which transfers better is a downstream question -- measure both.
+
+## The channel vocabulary travels with the weights
+
+An embedding row means whatever channel held that id when the row was learned.
+Every checkpoint here carries the vocabulary and its hash. Resolving
+electrode names against a different vocabulary silently trains on relabelled
+channels, which is why the hash is checked rather than assumed.
+
+## Architecture
+
+```
+embed_dim   384      depth 6      heads 6
+norm        rmsnorm     ffn swiglu     qk_norm True
+wavelet     max_level 3, kernel 16, bank db2, db4, db7, sym4, sym5
+channel     id embedding, injected at the token site
+mask        frequency_guided, ratio 0.7
+```
+
+## Loading
+
+`EMGC1Downstream` builds the frontend and patcher for your montage and loads this
+file into everything else. Nothing here is route-bound, so the montage, the
+window length and the patch length are yours to choose.
+
+```python
+import torch
+from physiowave.emg_c1.downstream import EMGC1Downstream
+
+ENC = "emg_c1_encoder.pth"
+ck = torch.load(ENC, map_location="cpu", weights_only=False)
+mc = ck["model_config"]                  # the width the weights were trained at
+CH = [f"ch{i}" for i in range(1, 9)]   # an 8-electrode armband
+
+model = EMGC1Downstream(
+    in_channels=len(CH), window_samples=2000, sampling_rate=2000.0,
+    patch_samples=250, num_classes=2, channel_names=CH,
+    embed_dim=mc["embed_dim"], depth=mc["depth"], num_heads=mc["num_heads"],
+    channel_embed_dim=mc.get("channel_embed_dim", 64),
+    channel_vocab_size=ck["channel_vocab_size"])
+
+report = model.load_pretrained(ENC)
+print(model.describe_transfer(report))
+# loaded: transformer N, channel encoder N, gate yes, frontend 0, patcher 0
+
+class Meta:                               # or your dataset's ChannelMeta
+    channel_names = CH
+    channel_mask = None
+
+out = model(torch.randn(2, len(CH), 2000), Meta())   # {"logits": [2, 2]}
+```
+
+`frontend 0, patcher 0` is the expected line, not a warning: those are the two
+things your montage supplies.
+
+**The frontend you build is not randomly initialised.** Its filters are seeded
+from real wavelet families -- `db2, db4, db7, sym4, sym5` -- and learned from there during
+fine-tuning. What a fresh frontend costs you is the adaptation those filters
+underwent during pretraining, not the wavelet prior itself.
+
+Names, not indices, are what this encoder and your data agree on. Pass real
+electrode names and check `channel_vocab_sha256` -- `load_pretrained` refuses a
+mismatch rather than silently training on relabelled channels.
+
+Code: <https://github.com/ForeverBlue816/PhysioWave-T>
+
+---
+Prepared by `scripts/prepare_hf_release.py` from `pretrain_emg_c1_moe`
+on 2026-10-29.

config_resolved.yaml

@@ -0,0 +1,63 @@
+data:
+  manifest_train: <path redacted for release>
+  manifest_val: <path redacted for release>
+  modality: emg
+  weights: temperature:0.5
+model:
+  channel_embed_dim: 64
+  channel_encoding: id
+  channel_injection: token
+  channel_token_gate_init: 0.0
+  depth: 6
+  dropout: 0.0
+  embed_dim: 384
+  ffn: swiglu
+  fold_gamma: 0.1
+  fold_synthesis: 3
+  importance_ratio: 0.6
+  mask_ratio: 0.7
+  masking_strategy: frequency_guided
+  max_level: 3
+  mlp_ratio: 4.0
+  norm: rmsnorm
+  num_heads: 6
+  qk_norm: true
+  scale_fold: dynamic
+  use_separate_channel: true
+  wave_init_mode: pad
+  wave_kernel_size: 16
+  wavelet_names:
+  - db2
+  - db4
+  - db7
+  - sym4
+  - sym5
+objective:
+  fold_kl: 0.001
+  mask_before_frontend: true
+  normalize_spec_target: true
+  raw_beta: 0.5
+  raw_weight: 0.5
+  spec_weight: 0.5
+output_dir: <path redacted for release>
+run_name: pretrain_emg_c1_moe
+seed: 42
+train:
+  batch_size_by_route:
+    A24_2000: 128
+    G64_2000: 48
+    W16_2000: 192
+  clip_grad_norm: 1.0
+  epochs: 60
+  grad_accumulation_steps: 1
+  lr: 0.0003
+  max_skipped_steps_per_epoch: 50
+  min_lr_ratio: 0.01
+  precision: bf16
+  skip_step_grad_norm_above: 100.0
+  steps_per_epoch: 1000
+  val_mask_seed: 1234
+  val_max_batches_per_dataset: 50
+  warmup_epochs: 2
+  weight_decay: 0.05
+trainer: emg_c1_moe

emg_c1_encoder.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c51e7054cf3f7a964896d96f85b3b4fc9629becb404d522ca172083b471b4137
+size 42713558

pretrain_emg_c1_moe_best.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6de2fd4e1bbfd70702496c7ecd50c32fcbf2edd2a5bdfea709246f21c0a63b0b
+size 48346302

pretrain_emg_c1_moe_best_macro.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:111cb1d804c9d4f0f4f4b848eb14a77026505a8e844d4f1d797ff0062df16e99
+size 48348262

pretrain_emg_c1_moe_epoch38.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:cf635e671c9d8236f9e4934fb903fb0d68d7775e9cba03883402baa671a70e4d
+size 48347250

pretrain_emg_c1_moe_final.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2065e8bee79f403426058e64b942da565834316e3f9ee6f1349ce2af8127b382
+size 48346618