Untied copy (lm_head = embed_tokens, tie_word_embeddings=false)

a3af314 verified about 1 month ago

711 Bytes

license: apache-2.0
datasets:
  - willcb/V3-wordle
base_model:
  - PrimeIntellect/Qwen3-0.6B

Untied copy of PrimeIntellect/Qwen3-0.6B-Reverse-Text-SFT. Same weights; lm_head.weight is stored as a copy of the input embedding and tie_word_embeddings is false, so trainers that don't support tied LM heads (e.g. prime-rl) can load it. Made with tools/untie_word_embeddings.py from prime-rl.

Qwen3-0.6B-Reverse-Text-SFT

A SFT fine-tune of PrimeIntellect/Qwen3-0.6B. Details here.