tiny-gpt / model_config.json

Upload model files

36ee8ed verified 9 days ago

144 Bytes

{
  "vocab_size": 4096,
  "context_length": 256,
  "d_model": 256,
  "n_layer": 6,
  "n_head": 8,
  "dropout": 0.1,
  "attention_impl": "sdpa"
}