M31 pretrain checkpoint 500

experiments/M31-Python-Agent-220M-v2/checkpoints/pretrain/step-00000500/README.md

@@ -0,0 +1,25 @@
+# M31Tesla — M31-Python-Agent-220M-v2
+
+Independent M31 research checkpoint.
+
+Architecture:
+- Parameters: 228,953,984
+- Vocabulary: 32,768
+- Hidden: 896
+- Layers: 20
+- Attention: 14 Q heads / 7 KV heads
+- FFN: 2816 SwiGLU
+- Maximum context: 2048
+- Training sequence: 512
+- RMSNorm + RoPE + GQA
+- Tied input/output embeddings
+
+Curriculum:
+Python repository pretraining -> coding SFT -> agent trajectories -> patch/verification curriculum.
+
+Data sources include HuggingFaceCode/stack-v3-train, NVIDIA OpenCodeInstruct,
+Nebius SWE-rebench OpenHands trajectories, and SWE-smith Python. HumanEval is
+evaluation-only.
+
+This repository path is an experiment artifact, not a claim that all source
+datasets share the same license. Respect each source dataset's terms.

experiments/M31-Python-Agent-220M-v2/checkpoints/pretrain/step-00000500/checkpoint.json

@@ -0,0 +1,10 @@
+{
+  "stage": "pretrain",
+  "step": 500,
+  "metrics": {
+    "loss": 3.427081048488617
+  },
+  "config_hash": "bb78dea6375e803cd388",
+  "repo_id": "eshanized/M31Tesla",
+  "hf_prefix": "experiments/M31-Python-Agent-220M-v2"
+}

experiments/M31-Python-Agent-220M-v2/checkpoints/pretrain/step-00000500/config.json

@@ -0,0 +1,18 @@
+{
+  "model_type": "m31",
+  "architectures": [
+    "M31Model"
+  ],
+  "vocab_size": 32768,
+  "hidden_size": 896,
+  "num_hidden_layers": 20,
+  "num_attention_heads": 14,
+  "num_key_value_heads": 7,
+  "intermediate_size": 2816,
+  "max_position_embeddings": 2048,
+  "rms_norm_eps": 1e-06,
+  "rope_theta": 10000.0,
+  "tie_word_embeddings": true,
+  "parameter_count": 228953984,
+  "experiment": "M31-Python-Agent-220M-v2"
+}

experiments/M31-Python-Agent-220M-v2/checkpoints/pretrain/step-00000500/model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6970ac8d57f08b6c205a788ce5b1d90776efc8948d8dc08ffe8dfeb7965b73a2
+size 1033273608