Compactbot commited on
Commit
b3bc31f
·
verified ·
1 Parent(s): 65f1844

Fix config params to the true unique count (100,936). 106,568 is the file sum that double-counts the tied head; the README explains this.

Browse files
Files changed (1) hide show
  1. config.json +2 -1
config.json CHANGED
@@ -9,5 +9,6 @@
9
  "ffn": "gelu",
10
  "ffn_mult": 2.667,
11
  "tie_word_embeddings": true,
12
- "params": 106568
 
13
  }
 
9
  "ffn": "gelu",
10
  "ffn_mult": 2.667,
11
  "tie_word_embeddings": true,
12
+ "params": 100936,
13
+ "params_note": "unique param count (tied head counted once); the safetensors file stores the head twice, so its raw numel sum is 106568"
14
  }