Perlous
/

ClearMind-Base

Model card Files Files and versions

ClearMind-Base / README.md

Perlous's picture

Initial release: ClearMind-Base 68.8M dense (pretrain+sft+dpo)

aa0d07a verified 5 months ago

|

Raw History Blame Contribute Delete

182 Bytes

ClearMind-Base

Trained from scratch with ClearMind. Architecture: GPT (RoPE + RMSNorm + SwiGLU + GQA + QK-Norm).

Parameters: 68.8M
Hidden size: 768
Layers: 8
Vocab: 6400