File size: 183 Bytes
8ebc625 | 1 2 3 4 5 6 7 8 9 | # ClearMind-Small
Trained from scratch with ClearMind. Architecture: GPT (RoPE + RMSNorm + SwiGLU + GQA + QK-Norm).
- Parameters: 15.7M
- Hidden size: 384
- Layers: 6
- Vocab: 6400
|
8ebc625 | 1 2 3 4 5 6 7 8 9 | # ClearMind-Small
Trained from scratch with ClearMind. Architecture: GPT (RoPE + RMSNorm + SwiGLU + GQA + QK-Norm).
- Parameters: 15.7M
- Hidden size: 384
- Layers: 6
- Vocab: 6400
|