adammnrr commited on
Commit
e87c8a0
·
verified ·
1 Parent(s): c51f43c

Model card: add the independent CUDA replication (random run order)

Browse files
Files changed (1) hide show
  1. README.md +12 -9
README.md CHANGED
@@ -86,19 +86,22 @@ Use that loss to match the reference training.
86
 
87
  Fine-tuning on the 61-word task with the authors' `run_cls.py` protocol (200
88
  epochs, AdamW, sigmoid cross-entropy, 6 seeds × 12 subjects, paper-time data
89
- revision `056a87b7`, mean ± SE over subjects). The authors' fine-tuning loop
90
- was run with both the original model and the braindecode port, starting from
91
- the same weights:
92
 
93
- | | paper (Table 2) | authors' code | braindecode port |
94
- |---|---|---|---|
95
- | Du-IN (mae) | 62.70 ± 4.69 | 60.52 ± 4.58 | 60.16 ± 4.62 |
96
- | Du-IN (vqvae) | 58.24 ± 4.83 | 55.10 ± 4.51 | 55.00 ± 4.51 |
97
- | scratch | 56.29 ± 5.20 | 54.06 ± 5.12 | 53.73 ± 5.13 |
 
 
 
 
 
98
 
99
  The port matches the authors' code to within 0.4 points. Training the port on
100
  plain logits (`cross_entropy(logits)`) instead of the sigmoid loss gives 55.6
101
- for mae.
102
 
103
  **Split caveat.** The authors' split concatenates a subject's runs in
104
  `os.listdir` order and puts the first 80 % of each word's trials in the
 
86
 
87
  Fine-tuning on the 61-word task with the authors' `run_cls.py` protocol (200
88
  epochs, AdamW, sigmoid cross-entropy, 6 seeds × 12 subjects, paper-time data
89
+ revision `056a87b7`, mean ± SE over subjects). There are two independent runs:
 
 
90
 
91
+ - **A**: the authors' `run_cls.train()` loop run with both the original model
92
+ and the port, starting from the same weights (Gaudi2 / CPU);
93
+ - **B**: a standalone braindecode re-implementation of the same loop with the
94
+ port, on CUDA, with the run order drawn at random per seed (see below).
95
+
96
+ | | paper (Table 2) | A: authors' code | A: port | B: port |
97
+ |---|---|---|---|---|
98
+ | Du-IN (mae) | 62.70 ± 4.69 | 60.52 ± 4.58 | 60.16 ± 4.62 | 59.37 ± 4.69 |
99
+ | Du-IN (vqvae) | 58.24 ± 4.83 | 55.10 ± 4.51 | 55.00 ± 4.51 | 54.02 ± 4.57 |
100
+ | scratch | 56.29 ± 5.20 | 54.06 ± 5.12 | 53.73 ± 5.13 | 53.77 ± 5.10 |
101
 
102
  The port matches the authors' code to within 0.4 points. Training the port on
103
  plain logits (`cross_entropy(logits)`) instead of the sigmoid loss gives 55.6
104
+ for mae in A and 54.5 in B.
105
 
106
  **Split caveat.** The authors' split concatenates a subject's runs in
107
  `os.listdir` order and puts the first 80 % of each word's trials in the