Upload folder using huggingface_hub

README.md

@@ -16,9 +16,9 @@ transformer serves four electrode layouts through route-specific wavelet
 frontends, and channel identity enters as a learned embedding of the channel
 NAME rather than of its index.
 
-Trained for 2 epochs
-(8 optimizer updates) on a mixture of public EEG
-corpora, 3.4 M parameters.
+Trained for 60 epochs
+(46080 optimizer updates) on a mixture of public EEG
+corpora, 30.3 M parameters.
 
 ## Objective
 
@@ -36,14 +36,14 @@ applied BEFORE the wavelet frontend (True), at ratio
 
 | metric | value | what 1.0 would mean |
 |---|---|---|
-| total loss | 0.7962 | — |
-| spec MSE | 1.0093 | — |
-| spec NMSE | 1.009 | predicting zero |
-| spec Pearson r | -0.000 | perfect |
-| raw SmoothL1 | 0.5832 | — |
-| raw NMSE | 1.009 | predicting zero |
-| raw Pearson r | -0.000 | perfect |
-| route-macro total | 0.7962 | — |
+| total loss | 0.2488 | — |
+| spec MSE | 0.3046 | — |
+| spec NMSE | 0.305 | predicting zero |
+| spec Pearson r | 0.831 | perfect |
+| raw SmoothL1 | 0.1930 | — |
+| raw NMSE | 0.278 | predicting zero |
+| raw Pearson r | 0.847 | perfect |
+| route-macro total | 0.2403 | — |
 
 NMSE is dimensionless: MSE(prediction, target) / MSE(0, target), so 1.0 is
 what predicting zero scores on that head's own target. The two losses are NOT
@@ -77,25 +77,40 @@ pretraining-only. It exists because it cannot be loaded wrong.
 | file | what it is |
 |---|---|
 | `eeg_c1_encoder.pth` | encoder alone: channel embedding, gate, transformer |
-| `pretrain_eeg_c1_moe_best.pth` | complete model, lowest validation total loss (epoch 1) |
-| `pretrain_eeg_c1_moe_final.pth` | complete model, last epoch, cosine fully annealed |
+| `pretrain_eeg_c1_moe_best.pth` | complete model, lowest validation **total** loss (epoch 57) |
+| `pretrain_eeg_c1_moe_final.pth` | complete model, last epoch (epoch 60) |
+| `pretrain_eeg_c1_moe_best_spec.pth` | complete model, lowest spec MSE (epoch 59) |
+| `pretrain_eeg_c1_moe_best_raw.pth` | complete model, lowest raw SmoothL1 (epoch 57) |
 | `metrics_epoch.jsonl` | the full training curve |
 | `config_resolved.yaml` | every resolved hyperparameter |
 
+### Why several checkpoints
+
+Each is the minimum of a different validation metric, and they are published
+together so a downstream comparison can ask which criterion transfers instead
+of assuming the one that names `best.pth`. The four bars are within validation
+noise of each other here, which is exactly why the question is worth asking and
+why reconstruction loss cannot answer it.
+
+`best.pth` is selected on the TOTAL loss -- the loss that was trained. It is
+not the best spec loss, and not the best raw loss.
+
+`pretrain_eeg_c1_moe_best_macro.pth` would have been byte-identical to `pretrain_eeg_c1_moe_best.pth` -- the same epoch won both bars -- so it is not published separately.
+
 ### What is in the complete model
 
 | component | parameters | share | |
 |---|---|---|---|
-| shared transformer | 0.01 M | 0.4% | encoder, always transfers |
-| channel-name embedding | 0.00 M | 0.1% | always transfers |
-| channel projection | 0.00 M | 0.0% | always transfers |
-| wavelet frontends (all 4 routes) | 3.37 M | 98.3% | route-bound; downstream usually builds its own |
-| patchers (both rates) | 0.01 M | 0.4% | route-bound |
-| spec decoder | 0.01 M | 0.4% | pretraining only |
-| raw decoder | 0.01 M | 0.4% | pretraining only |
-| **total** | **3.43 M** | 100% | |
+| shared transformer | 25.22 M | 83.2% | encoder, always transfers |
+| channel-name embedding | 0.02 M | 0.1% | always transfers |
+| channel projection | 0.03 M | 0.1% | always transfers |
+| wavelet frontends (all 4 routes) | 3.37 M | 11.1% | route-bound; downstream usually builds its own |
+| patchers (both rates) | 0.20 M | 0.7% | route-bound |
+| spec decoder | 0.72 M | 2.4% | pretraining only |
+| raw decoder | 0.72 M | 2.4% | pretraining only |
+| **total** | **30.30 M** | 100% | |
 
-At this width the four frontends (98%) outweigh the shared transformer (0%). There is still one encoder file rather than four, because what a per-route file adds is a frontend that only loads onto that exact montage; cut the one you want out of the complete checkpoint.
+The shared transformer is 83% of the model and the four frontends together are 11%, which is why there is no per-route encoder file here: four of them would duplicate that transformer four times to deliver frontends that are a small fraction of it. Cut the one you want out of the complete checkpoint instead.
 
 ### What transfers, and what does not
 
@@ -144,7 +159,7 @@ which is why the hash is checked rather than assumed.
 ## Architecture
 
 ```
-embed_dim   32      depth 1      heads 4
+embed_dim   512      depth 8      heads 8
 norm        rmsnorm     ffn swiglu     qk_norm True
 wavelet     max_level 3, kernel 16
 channel     id embedding, injected at the token site
@@ -200,5 +215,5 @@ mismatch rather than silently training on relabelled electrodes.
 Code: <https://github.com/ForeverBlue816/PhysioWave-T>
 
 ---
-Prepared by `scripts/prepare_hf_release.py` from `run`
+Prepared by `scripts/prepare_hf_release.py` from `pretrain_eeg_c1_moe`
 on 2026-09-09.

config_resolved.yaml

@@ -4,13 +4,13 @@ data:
   modality: eeg
   weights: temperature:0.5
 model:
-  channel_embed_dim: 8
+  channel_embed_dim: 64
   channel_encoding: id
   channel_injection: token
   channel_token_gate_init: 0.0
-  depth: 1
+  depth: 8
   dropout: 0.0
-  embed_dim: 32
+  embed_dim: 512
   ffn: swiglu
   fold_gamma: 0.1
   fold_synthesis: 3
@@ -20,7 +20,7 @@ model:
   max_level: 3
   mlp_ratio: 4.0
   norm: rmsnorm
-  num_heads: 4
+  num_heads: 8
   qk_norm: true
   scale_fold: dynamic
   use_separate_channel: true
@@ -44,19 +44,19 @@ run_name: pretrain_eeg_c1_moe
 seed: 42
 train:
   batch_size_by_route:
-    E128_512: 1
-    E19_256: 1
-    E32_512: 1
-    E64_256: 1
+    E128_512: 24
+    E19_256: 128
+    E32_512: 96
+    E64_256: 48
   clip_grad_norm: 1.0
-  epochs: 2
+  epochs: 60
   grad_accumulation_steps: 1
   lr: 0.0003
   min_lr_ratio: 0.01
-  precision: fp32
-  steps_per_epoch: 4
+  precision: bf16
+  steps_per_epoch: 768
   val_mask_seed: 1234
   val_max_batches_per_dataset: 100
-  warmup_epochs: 0
+  warmup_epochs: 2
   weight_decay: 0.05
 trainer: eeg_c1_moe

eeg_c1_encoder.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e4f6cbbf6195640ad7989a1c62a69b0ecb898840e6ef559402959906128bac8b
-size 75665
+oid sha256:220e4652345e7ad6b9c9fc293970e82818d3d04512f3389ad7b29c45484024c9
+size 101155630

pretrain_eeg_c1_moe_best.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8feea87150d668aa70fbf6c3e755cf669feb5396649ed03d8c4baedd1aa7ae48
-size 13874923
+oid sha256:82fc7fa7874aa7125cb300a6a015b0679897a68602638b9362698282e083a204
+size 121389646

pretrain_eeg_c1_moe_best_raw.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:30e7fb3e5ae4be0663168069a229d40ed7d3297c6e0e19d6cfd23a09bf55cab8
+size 121391422

pretrain_eeg_c1_moe_best_spec.pth

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36c1bc7fcf2f781b333a45153958c0ab75eee829189922c4cd473191385d6df1
+size 121391850

pretrain_eeg_c1_moe_final.pth

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0ff4af884954ebc781ebc1b806808f76ae7a3111b16b082fa621b7d0cd3d47a7
-size 13875275
+oid sha256:2dcd0c18db845ce2b6d6f10a9eddc85c9fdfadac19e8b7f3f2f7a1b529e8c285
+size 121390138