-
working tokenizer
-
Update config.json
-
Upload 6 files
-
Upload 6 files
revet to working
-
simpler tokenizer
-
silu instead of relu
-
relu activations
-
smallest tokenizer and correct token indices
-
fast llama tokenizer
-
tokenizer minimized
-
llama tokenizer
-
much bigger model
-
bfloat by default
-
trying bigger model
-
4 hidden layers
-
intermediate 64
-
byt5 tokenizer
-
intermediate 1
-
64 per head
-
hidden=32*8
-
hidden=64, 2 channels per head
-
32 heads
-
2 hidden layers
-
all dimensions are at least 8
-
hidden_size=8
-
dummy-phi3 first try
