Aether

aethertp

AI & ML interests

None yet

Recent Activity

new activity 2 days ago

aethertp/PicoLM-V2.1-81M-Instruct:Drop redundant untied lm_head.weight (Option 1) The checkpoint stored both tok_embeddings.weight and a separate lm_head.weight, each [24576, 576] = 14,155,776 params, even though config.json sets tie_word_embeddings: true. At runtime the head is tied to the embedding, so lm_head.weight is never read. This removes that one tensor. Verified: - 201 -> 200 tensors; stored elements 96,017,472 -> 81,861,696 (exactly the card's count) - file 192,054,704 -> 163,743,061 bytes (-28.3 MB) - loaded both versions with the repo's own PicoLMV2ForCausalLM: identical logits on fixed inputs (max |diff| = 0.0), same 81,861,696 unique params, head tied to embedding in both. Requested by aethertp in the discussion (Option 1).

new activity 2 days ago

aethertp/PicoLM-V2.1-81M-Instruct:Checkpoint stores an untied lm_head: 96.0M params on disk vs 81.86M on the card

updated a model 3 days ago

aethertp/PicoLM-V2-81M-Instruct

View all activity

Organizations

None yet

New activity in aethertp/PicoLM-V2.1-81M-Instruct 2 days ago

Drop redundant untied lm_head.weight (Option 1) The checkpoint stored both tok_embeddings.weight and a separate lm_head.weight, each [24576, 576] = 14,155,776 params, even though config.json sets tie_word_embeddings: true. At runtime the head is tied to the embedding, so lm_head.weight is never read. This removes that one tensor. Verified: - 201 -> 200 tensors; stored elements 96,017,472 -> 81,861,696 (exactly the card's count) - file 192,054,704 -> 163,743,061 bytes (-28.3 MB) - loaded both versions with the repo's own PicoLMV2ForCausalLM: identical logits on fixed inputs (max |diff| = 0.0), same 81,861,696 unique params, head tied to embedding in both. Requested by aethertp in the discussion (Option 1).

#2 opened 2 days ago by

Checkpoint stores an untied lm_head: 96.0M params on disk vs 81.86M on the card

#1 opened 3 days ago by

New activity in aethertp/PicoLM-80M-Instruct 3 days ago

Overcoming the ARC-Easy floor and vocabulary budget on an 80M footprint

#1 opened 7 days ago by

AndrewThompson1233

New activity in aethertp/PicoLM-80M-Instruct 4 days ago

Overcoming the ARC-Easy floor and vocabulary budget on an 80M footprint

#1 opened 7 days ago by

AndrewThompson1233

New activity in aethertp/PicoLM-80M-Instruct 5 days ago

Overcoming the ARC-Easy floor and vocabulary budget on an 80M footprint

#1 opened 7 days ago by

AndrewThompson1233