M31-Entropy pretrain 500

Files changed (5) hide show

experiments/M31-Entropy/checkpoints/pretrain/step-00000500/README.md ADDED Viewed

	@@ -0,0 +1 @@


1	+ # M31-Entropy-220M\n\nReasoning-first Python model trained from scratch.\n

experiments/M31-Entropy/checkpoints/pretrain/step-00000500/checkpoint.json ADDED Viewed

+{
+  "experiment": "M31-Entropy-220M",
+  "stage": "pretrain",
+  "step": 500,
+  "metrics": {
+    "loss": 5.152145743370056,
+    "avg50": 5.1091614651679995
+  },
+  "repo_id": "eshanized/M31Entropy",
+  "path": "experiments/M31-Entropy/checkpoints/pretrain/step-00000500"
+}

experiments/M31-Entropy/checkpoints/pretrain/step-00000500/config.json ADDED Viewed

+{
+  "model_type": "m31entropy",
+  "architectures": [
+    "M31ENTROPY"
+  ],
+  "vocab_size": 128000,
+  "hidden_size": 896,
+  "num_hidden_layers": 20,
+  "num_attention_heads": 14,
+  "num_key_value_heads": 7,
+  "intermediate_size": 2816,
+  "max_position_embeddings": 2048,
+  "training_sequence_length": 1024,
+  "rope_theta": 10000.0,
+  "rms_norm_eps": 1e-06,
+  "tie_word_embeddings": true,
+  "parameter_count": 314281856,
+  "experiment": "M31-Entropy-220M"
+}

experiments/M31-Entropy/checkpoints/pretrain/step-00000500/model.safetensors ADDED Viewed

+version https://git-lfs.github.com/spec/v1
+oid sha256:3b56b52c34e2897cd9c01be5585f3bc822c617600b2cc3db372c897da2c8e16b
+size 1257143008

experiments/M31-Entropy/checkpoints/pretrain/step-00000500/tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff