firstlight-124m / config.json
adarsh-007's picture
Firstlight-124M: base + chat (SFT) weights, model card, SFT loss plot
292ae05 verified
Raw History Blame Contribute Delete
319 Bytes
{
"n_layer": 12,
"n_head": 12,
"vocab_size": 50304,
"n_embd": 768,
"block_size": 1024,
"tokenizer": "tiktoken gpt2",
"special_tokens": {
"<|endoftext|>": 50256,
"<|user_start|>": 50257,
"<|user_end|>": 50258,
"<|assistant_start|>": 50259,
"<|assistant_end|>": 50260
}
}