File size: 144 Bytes
36ee8ed
 
 
 
 
 
 
 
 
1
2
3
4
5
6
7
8
9
{
  "vocab_size": 4096,
  "context_length": 256,
  "d_model": 256,
  "n_layer": 6,
  "n_head": 8,
  "dropout": 0.1,
  "attention_impl": "sdpa"
}