mithraindir's picture
Upload folder using huggingface_hub
c25befa verified
Raw History Blame Contribute Delete
173 Bytes
{
"vocab": 800,
"seq": 128,
"d": 48,
"ff": 72,
"heads": 3,
"kv_heads": 1,
"head_dim": 16,
"layers": 2,
"loops": 2,
"v_mix": 0.5,
"rope_base": 10000.0
}