Fix config params to the true unique count (100,936). 106,568 is the file sum that double-counts the tied head; the README explains this.
Browse files- config.json +2 -1
config.json
CHANGED
|
@@ -9,5 +9,6 @@
|
|
| 9 |
"ffn": "gelu",
|
| 10 |
"ffn_mult": 2.667,
|
| 11 |
"tie_word_embeddings": true,
|
| 12 |
-
"params":
|
|
|
|
| 13 |
}
|
|
|
|
| 9 |
"ffn": "gelu",
|
| 10 |
"ffn_mult": 2.667,
|
| 11 |
"tie_word_embeddings": true,
|
| 12 |
+
"params": 100936,
|
| 13 |
+
"params_note": "unique param count (tied head counted once); the safetensors file stores the head twice, so its raw numel sum is 106568"
|
| 14 |
}
|