M31 v7 pretrain 1832

Files changed (5) hide show

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832/README.md ADDED Viewed

	@@ -0,0 +1 @@


1	+ # M31-Genesis-220M-v7\n\nReasoning-first Python model trained from scratch.\n

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832/checkpoint.json ADDED Viewed

+{
+  "experiment": "M31-Genesis-220M-v7",
+  "stage": "pretrain",
+  "step": 1832,
+  "metrics": {
+    "stage_complete": true
+  },
+  "repo_id": "eshanized/M31Tesla",
+  "path": "experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832"
+}

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832/config.json ADDED Viewed

+{
+  "model_type": "m31",
+  "architectures": [
+    "M31"
+  ],
+  "vocab_size": 32768,
+  "hidden_size": 896,
+  "num_hidden_layers": 20,
+  "num_attention_heads": 14,
+  "num_key_value_heads": 7,
+  "intermediate_size": 2816,
+  "max_position_embeddings": 2048,
+  "training_sequence_length": 512,
+  "rope_theta": 10000.0,
+  "rms_norm_eps": 1e-06,
+  "tie_word_embeddings": true,
+  "parameter_count": 228953984,
+  "experiment": "M31-Genesis-220M-v7"
+}

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832/model.safetensors ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:9a3cd3e2c7be9915f9b88f74cc8e8d72abd445fa50197cd0c6536a6666a8f737
+size 915831512

experiments/M31-Genesis-220M-v7/checkpoints/pretrain/step-00001832/tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff