M31 v7 pretrain 1500

Files changed (5) hide show

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500/README.md ADDED Viewed

	@@ -0,0 +1 @@


1	+ # M31-Genesis-220M-v7\n\nReasoning-first Python model trained from scratch.\n

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500/checkpoint.json ADDED Viewed

+{
+  "experiment": "M31-Genesis-220M-v7",
+  "stage": "pretrain",
+  "step": 1500,
+  "metrics": {
+    "loss": 5.807806015014648,
+    "avg50": 4.972106469869614
+  },
+  "repo_id": "eshanized/M31Tesla",
+  "path": "experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500"
+}

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500/config.json ADDED Viewed

+{
+  "model_type": "m31",
+  "architectures": [
+    "M31"
+  ],
+  "vocab_size": 32768,
+  "hidden_size": 896,
+  "num_hidden_layers": 20,
+  "num_attention_heads": 14,
+  "num_key_value_heads": 7,
+  "intermediate_size": 2816,
+  "max_position_embeddings": 2048,
+  "training_sequence_length": 512,
+  "rope_theta": 10000.0,
+  "rms_norm_eps": 1e-06,
+  "tie_word_embeddings": true,
+  "parameter_count": 228953984,
+  "experiment": "M31-Genesis-220M-v7"
+}

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500/model.safetensors ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:858d76b63dddddbfbdf8300fd9bec5016e8e2e267e390a1eb396ea99bfa4fead
+size 915831512

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001500/tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff