smoke test: ClearMind-Small (13.23M dense, pretrain+sft+dpo, minimind tokenizer/data)

8ebc625 verified 6 months ago

183 Bytes

# ClearMind-Small

Trained from scratch with ClearMind. Architecture: GPT (RoPE + RMSNorm + SwiGLU + GQA + QK-Norm).

- Parameters: 15.7M
- Hidden size: 384
- Layers: 6
- Vocab: 6400

	# ClearMind-Small

	Trained from scratch with ClearMind. Architecture: GPT (RoPE + RMSNorm + SwiGLU + GQA + QK-Norm).

	- Parameters: 15.7M
	- Hidden size: 384
	- Layers: 6
	- Vocab: 6400