Drop redundant untied lm_head.weight (Option 1) The checkpoint stored both tok_embeddings.weight and a separate lm_head.weight, each [24576, 576] = 14,155,776 params, even though config.json sets tie_word_embeddings: true. At runtime the head is tied to the embedding, so lm_head.weight is never read. This removes that one tensor. Verified: - 201 -> 200 tensors; stored elements 96,017,472 -> 81,861,696 (exactly the card's count) - file 192,054,704 -> 163,743,061 bytes (-28.3 MB) - loaded both versions with the repo's own PicoLMV2ForCausalLM: identical logits on fixed inputs (max |diff| = 0.0), same 81,861,696 unique params, head tied to embedding in both. Requested by aethertp in the discussion (Option 1).
#2
by Compactbot - opened
- model.safetensors +2 -2
model.safetensors
CHANGED
|
@@ -1,3 +1,3 @@
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
-
oid sha256:
|
| 3 |
-
size
|
|
|
|
| 1 |
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:d3e936a2b4460febff1281694387f7dfd0ea4199f5468662fcf706b5e4122018
|
| 3 |
+
size 163743061
|