Front-loading study
Four web datasets pretrained with and without MixtureVitae's instruction and reasoning subset, then extended to 16K and given Tulu3 SFT and DPO.
2B • Updated • 7Note MixtureVitae web part, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 17Note MixtureVitae web part, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 8Note MixtureVitae web part, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k
2B • Updated • 8Note MixtureVitae web part, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 25Note MixtureVitae web part, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-mixturevitae-noinstruct-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 14Note MixtureVitae web part, without front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated
Text Generation • 2B • Updated • 24Note MixtureVitae web part, with front-loading · base
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 9Note MixtureVitae web part, with front-loading · Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 8Note MixtureVitae web part, with front-loading · Tulu3 DPO
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k
Text Generation • 2B • Updated • 17Note MixtureVitae web part, with front-loading · extended to 16K
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 13Note MixtureVitae web part, with front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-MixtureVitae-300BT-v1-decontaminated-16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note MixtureVitae web part, with front-loading · 16K, Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-WithChatTemplate
Text Generation • 2B • Updated • 9Note Comma0.1, without front-loading · base
ali-elganzory/1.7b-Comma0.1-300BT-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 9Note Comma0.1, without front-loading · Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 7Note Comma0.1, without front-loading · Tulu3 DPO
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k
Text Generation • 2B • Updated • 35Note Comma0.1, without front-loading · extended to 16K
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 28Note Comma0.1, without front-loading · 16K, Tulu3 SFT
ali-elganzory/1.7b-Comma0.1-300BT-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 31Note Comma0.1, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096
2B • Updated • 24Note Comma0.1, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 99Note Comma0.1, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 19Note Comma0.1, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 46Note Comma0.1, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 56Note Comma0.1, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-comma-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 35Note Comma0.1, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096
Text Generation • 2B • Updated • 8Note Nemotron-CC-HQ, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 17Note Nemotron-CC-HQ, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 16Note Nemotron-CC-HQ, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k
2B • Updated • 9Note Nemotron-CC-HQ, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 24Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 24Note Nemotron-CC-HQ, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096
2B • Updated • 14Note Nemotron-CC-HQ, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 12Note Nemotron-CC-HQ, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note Nemotron-CC-HQ, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 10Note Nemotron-CC-HQ, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 32Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-nemotron-hq-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 14Note Nemotron-CC-HQ, with front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096
Text Generation • 2B • Updated • 9 • 1Note FineWeb-Edu, without front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 13Note FineWeb-Edu, without front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note FineWeb-Edu, without front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-4096-longsft_16k
Text Generation • 2B • Updated • 19 • 1Note FineWeb-Edu, without front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 30Note FineWeb-Edu, without front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-300B-4096-longsft_16k-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 20Note FineWeb-Edu, without front-loading · 16K, Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096
2B • Updated • 7Note FineWeb-Edu, with front-loading · base
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-SFT-Tulu3-decontaminated
Text Generation • 2B • Updated • 11Note FineWeb-Edu, with front-loading · Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-DPO-Tulu3-decontaminated
Text Generation • 2B • Updated • 10Note FineWeb-Edu, with front-loading · Tulu3 DPO
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k
2B • Updated • 8Note FineWeb-Edu, with front-loading · extended to 16K
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-SFT-Tulu3-decontam
Text Generation • 2B • Updated • 16Note FineWeb-Edu, with front-loading · 16K, Tulu3 SFT
ali-elganzory/open-sci-ref-v0.02-1.7b-fineweb-edu-1.4t-mv_reasoning-300B-4096-longsft_16k-DPO-Tulu3-decontam
Text Generation • 2B • Updated • 11Note FineWeb-Edu, with front-loading · 16K, Tulu3 DPO
ontocord/1.7b-Comma0.1-300BT
2B • Updated • 9Note Comma0.1, without front-loading · base (language suite)
permissive-post-train/evaluation-records
Updated • 330Note The evaluation records of every checkpoint here