solintellegence
/

Sol-Lite-Base

Text Generation

small-language-model

recurrent-depth

grouped-query-attention

Model card Files Files and versions

Sol-Lite-Base / training_state.json

j0no12's picture

Release Sol Lite Base

23ad248 verified 26 days ago

Raw History Blame Contribute Delete

467 Bytes

	{
	"parameters": 14995843,
	"training_tokens": 8153333760,
	"optimizer_steps": 248820,
	"sequence_length": 512,
	"tokenizer_sha256": "ed6b75f7fad6994e75932c98cd1bc8dfdcaa88d4be8bd905bab970963f5c10b0",
	"model_safetensors_sha256": "0ef03e7dd15c8dab74daad517f76445470a60505bd2fc81e5daeb49269a49d2a",
	"source_mix": ["FineWeb-Edu", "DCLM", "UltraFineWeb L1 English", "UltraFineWeb L2 English", "FineWeb-HQ", "FineMath", "UltraFineWeb L3 English", "FinePhrase"]
	}