--- license: apache-2.0 tags: - physiowave - physiowave-t - foundation-model - emg - semg - electromyography - biosignal - self-supervised - masked-autoencoder - pretrained library_name: pytorch --- # PhysioWave-T-sEMG: multi-route pretrained sEMG encoder One of the three **PhysioWave-T** encoders ([EEG](https://huggingface.co/ForeverBlue/EEG), [ECG](https://huggingface.co/ForeverBlue/ECG), sEMG); the [RALF fusion model](https://huggingface.co/ForeverBlue/RALF-SeizeIT2) combines them. Code: . Masked dual-reconstruction pretraining on surface EMG from wristbands, forearm arrays and high-density grids. One shared transformer serves three electrode layouts through route-specific wavelet frontends, and electrode identity enters as a learned embedding of the electrode NAME rather than of its index. Trained for 45 epochs of 60 configured; validation was logged for the first 39, which is what `metrics_epoch.jsonl` holds (the last logged validation at step 39000) on a mixture of public sEMG corpora, 12.1 M parameters. The files here are built around `pretrain_emg_c1_moe_epoch38.pth`, epoch 39. ## Objective ``` 0.5*L_spec + 0.5*L_raw + 0.001*L_foldKL ``` Two heads at equal weight. One predicts folded wavelet patches, the other predicts the preprocessed sEMG waveform of the same masked patches. Masking is applied BEFORE the wavelet frontend (True), at ratio 0.7, and the spec target is per-patch normalised (True). ## Training curves Pretraining ran for 45 epochs on 16 GPUs over the six corpora, with validation logged for the first 39. The two reconstruction terms behave differently: the folded-wavelet loss is lowest early (epoch 7, which is what `best.pth` selects on the total), while the raw-waveform loss keeps falling to the end. The released main file is the epoch-38 checkpoint, the one with the lowest raw-waveform loss, and the one every downstream sEMG result of the paper was obtained with. ![Training and validation per epoch](training_curves.png) *Train and validation per epoch: the total loss, the two reconstruction terms (folded wavelet patch, raw waveform) and the masked spectral correlation.* ## Validation, at the released checkpoint (epoch 39) | metric | value | what 1.0 would mean | |---|---|---| | total loss | 0.2580 | — | | spec MSE | 0.3207 | — | | spec NMSE | 0.321 | predicting zero | | spec Pearson r | 0.822 | perfect | | raw SmoothL1 | 0.1953 | — | | raw NMSE | 0.381 | predicting zero | | raw Pearson r | 0.782 | perfect | | route-macro total | 0.2315 | — | NMSE is dimensionless: MSE(prediction, target) / MSE(0, target), so 1.0 is what predicting zero scores on that head's own target. The two losses are NOT comparable to each other -- one is an MSE on normalised wavelet coefficients, the other a SmoothL1 on the window-normalised waveform. The columns that compare the heads are the correlations and the NMSEs. The route-macro row weights the three routes equally. The plain rows weight by sample count, and validation is dominated by the largest corpora, so a model specialising toward the smaller ones moves the two rows in opposite directions. ## Files ### Which file do you want | your situation | download | |---|---| | fine-tune on any montage | `emg_c1_encoder.pth` | | your montage IS one of the three routes below | `pretrain_emg_c1_moe_epoch38.pth`, then cut the route out (one command, below) | | continue pretraining, or take the model apart yourself | `pretrain_emg_c1_moe_epoch38.pth` | **`pretrain_emg_c1_moe_epoch38.pth` is the complete model** -- all three wavelet frontends, the patchers, both reconstruction decoders. Nothing is held back; only the optimiser state was stripped, which no downstream use needs. Take it if you want to keep or strip pieces yourself. `emg_c1_encoder.pth` is the same weights minus everything route-bound and pretraining-only. It exists because it cannot be loaded wrong. | file | what it is | |---|---| | `emg_c1_encoder.pth` | encoder alone: channel embedding, gate, transformer | | `pretrain_emg_c1_moe_epoch38.pth` | complete model, the reported checkpoint (epoch 39) | | `pretrain_emg_c1_moe_final.pth` | complete model, last epoch (epoch 45) | | `pretrain_emg_c1_moe_best.pth` | complete model, lowest validation total loss (epoch 7) | | `pretrain_emg_c1_moe_best_macro.pth` | complete model, lowest route-macro total (epoch 25) | | `metrics_epoch.jsonl` | the full training curve | | `config_resolved.yaml` | every resolved hyperparameter | ### Why several checkpoints Each is the minimum of a different validation metric, and they are published together so a downstream comparison can ask which criterion transfers instead of assuming the one that names `best.pth`. Reconstruction loss cannot answer that question; a downstream task can. `best.pth` is selected on the TOTAL loss -- the loss that was trained (epoch 6 counted from 0). It is not the best spec loss, and not the best raw loss. **Why `pretrain_emg_c1_moe_epoch38.pth` is the main file.** It has the lowest validation raw loss; best.pth wins the total loss at epoch 7, before the raw head had converged (raw r 0.740 against 0.782). The paper's sEMG downstream and fusion results use this file. `pretrain_emg_c1_moe_best_spec.pth` would have been byte-identical to `pretrain_emg_c1_moe_best.pth` -- the same epoch won both bars -- so it is not published separately. `pretrain_emg_c1_moe_best_raw.pth` would have been byte-identical to `pretrain_emg_c1_moe_epoch38.pth` -- the same epoch won both bars -- so it is not published separately. ### What is in the complete model | component | parameters | share | | |---|---|---|---| | shared transformer | 10.63 M | 88.2% | encoder, always transfers | | channel-name embedding | 0.02 M | 0.1% | always transfers | | channel projection | 0.02 M | 0.2% | always transfers | | wavelet frontends (all three routes) | 0.79 M | 6.6% | route-bound; downstream usually builds its own | | patchers (one rate) | 0.10 M | 0.8% | route-bound | | spec decoder | 0.24 M | 2.0% | pretraining only | | raw decoder | 0.24 M | 2.0% | pretraining only | | **total** | **12.05 M** | 100% | | The shared transformer is 88% of the model and the three frontends together are 7%, which is why there is no per-route encoder file here: three of them would duplicate that transformer three times to deliver frontends that are a small fraction of it. Cut the one you want out of the complete checkpoint instead. ### What transfers, and what does not `downstream.py` splits a checkpoint into two sets. The channel embedding, its gate and the shared transformer **always** load. The wavelet frontend and the patcher load **only when your montage is the route they were trained for** -- same channel count, same patch length -- and are skipped otherwise rather than reshaped. The usual sEMG benchmark montages -- armbands, forearm rings, NinaPro's electrodes -- are none of these routes, so a fresh frontend is the normal case. Their electrode names are outside the vocabulary too. Each unknown name gets an embedding row of its own, initialised from the pretrained `` row and trained as a montage adapter; `new_channel_rows=False` keeps the one shared `` vector instead, and channels are then told apart by the position encoding alone. The three routes the shared transformer was pretrained across: | route | channels | rate | |---|---|---| | `A24_2000` | 24 | 2000 Hz | | `G64_2000` | 64 | 2000 Hz | | `W16_2000` | 16 | 2000 Hz | To fine-tune ON one of them exactly, take that route's frontend and patcher out of the complete checkpoint: ```bash python scripts/export_eeg_pretrained_encoder.py \ --checkpoint pretrain_emg_c1_moe_epoch38.pth --route G64_2000 \ --output encoder_G64_2000.pth ``` The exporter reads the modality from the checkpoint; its name is historical. Omit `--route` and you get `emg_c1_encoder.pth` again. Both pretraining decoders are dropped either way: fine-tuning against the pretext objective by accident is the failure that prevents. The last checkpoint (epoch 45) has no logged validation row, so it cannot be compared with the released one here. Which transfers better is a downstream question -- measure both. ## The channel vocabulary travels with the weights An embedding row means whatever channel held that id when the row was learned. Every checkpoint here carries the vocabulary and its hash. Resolving electrode names against a different vocabulary silently trains on relabelled channels, which is why the hash is checked rather than assumed. ## Architecture ``` embed_dim 384 depth 6 heads 6 norm rmsnorm ffn swiglu qk_norm True wavelet max_level 3, kernel 16, bank db2, db4, db7, sym4, sym5 channel id embedding, injected at the token site mask frequency_guided, ratio 0.7 ``` ## Loading `EMGC1Downstream` builds the frontend and patcher for your montage and loads this file into everything else. Nothing here is route-bound, so the montage, the window length and the patch length are yours to choose. ```python import torch from physiowave.emg_c1.downstream import EMGC1Downstream ENC = "emg_c1_encoder.pth" ck = torch.load(ENC, map_location="cpu", weights_only=False) mc = ck["model_config"] # the width the weights were trained at CH = [f"ch{i}" for i in range(1, 9)] # an 8-electrode armband model = EMGC1Downstream( in_channels=len(CH), window_samples=2000, sampling_rate=2000.0, patch_samples=250, num_classes=2, channel_names=CH, embed_dim=mc["embed_dim"], depth=mc["depth"], num_heads=mc["num_heads"], channel_embed_dim=mc.get("channel_embed_dim", 64), channel_vocab_size=ck["channel_vocab_size"]) report = model.load_pretrained(ENC) print(model.describe_transfer(report)) # loaded: transformer N, channel encoder N, gate yes, frontend 0, patcher 0 class Meta: # or your dataset's ChannelMeta channel_names = CH channel_mask = None out = model(torch.randn(2, len(CH), 2000), Meta()) # {"logits": [2, 2]} ``` `frontend 0, patcher 0` is the expected line, not a warning: those are the two things your montage supplies. **The frontend you build is not randomly initialised.** Its filters are seeded from real wavelet families -- `db2, db4, db7, sym4, sym5` -- and learned from there during fine-tuning. What a fresh frontend costs you is the adaptation those filters underwent during pretraining, not the wavelet prior itself. Names, not indices, are what this encoder and your data agree on. Pass real electrode names and check `channel_vocab_sha256` -- `load_pretrained` refuses a mismatch rather than silently training on relabelled channels. Code: --- Prepared by `scripts/prepare_hf_release.py` from `pretrain_emg_c1_moe` on 2026-10-29.