Spaces:
Sleeping
Sleeping
Commit History
-
Add OLMo 2, Llama 3.2, Gemma 2, SmolLM2 to model registry
Registers 11 new models across four families, all targeting the final checkpoint only (revisions=[]). OLMo 2 (4 sizes, allenai/, not gated): reuses extract_llama_qkv — architecture confirmed identical to LLaMA layout (separate q/k/v proj under model.layers.{i}.self_attn.*). Llama 3.2 1B and 3B (meta-llama/, gated): reuses extract_llama_qkv. Gemma 2 2B and 9B (google/, gated): adds extract_gemma2_qkv which reads head_dim explicitly from config.json (256 for both sizes) rather than deriving it from hidden_size // n_heads — these differ for Gemma 2. Reuses extract_llama_o and extract_llama_biases (same key prefix, bias-free). SmolLM2 135M/360M/1.7B (HuggingFaceTB/, not gated, instruct variants): reuses extract_llama_qkv — registered as LlamaForCausalLM in transformers. Also adds _read_head_dim() helper and fixes d_head computation in head_pipeline, pair_pipeline, and fuse_timesteps to use hf_config.head_dim when present (Gemma 2) rather than always computing d_model // n_heads. Adds…3c24019