-
Model V7
Loss: 2.866, Perplexity: 17.567
max_seq_len = 740
batch_size = 256
n_vocab = 50257
n_layers = 24
n_heads = 16
embed_dim = 1024
ffn_dim = embed_dim*4
-
Upload 2 files
Model V6
Cross Entropy Loss of 2.9
-
Upload 2 files
Model V5
Trained on 1M Rows for 10 Epochs
Loss Value 3.277
-
Upload 2 files
New model with max_seq_len=680 and loss of 3.548
-
New Model
Cross Entropy Loss of 3.5
-
Model Updated
-
Model V3
-
Upload 2 files
-
Upload 2 files
V2
-
Upload 3 files
Model Uploaded -
initial commit
