zekdevs
/

llm-rs

Text Generation

Safetensors

English

Model card Files Files and versions

xet

llm-rs / config.json

zekdevs

Update config.json

3180aab verified about 1 year ago

Raw History Blame

184 Bytes

{
  "vocab_size": 256,
  "max_seq_len": 384,
  "embed_dim": 640,
  "num_heads": 10,
  "num_layers": 8,
  "feed_forward_dim": 2560,
  "layer_norm_eps": 0.00001,
  "activation": "relu"
}

	{
	"vocab_size": 256,
	"max_seq_len": 384,
	"embed_dim": 640,
	"num_heads": 10,
	"num_layers": 8,
	"feed_forward_dim": 2560,
	"layer_norm_eps": 0.00001,
	"activation": "relu"
	}