Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -11,6 +11,6 @@ tags:
|
|
| 11 |
|
| 12 |
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
|
| 13 |
- **Sequence Mixer:** Fractal Memory Pyramid with QK-Norm RoPE
|
| 14 |
-
- **Training Step:**
|
| 15 |
-
- **Current Loss:**
|
| 16 |
- **Optimizer:** Hybrid Muon (2D Dense Linears) + AdamW (MoE Nodes, Embeddings)
|
|
|
|
| 11 |
|
| 12 |
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
|
| 13 |
- **Sequence Mixer:** Fractal Memory Pyramid with QK-Norm RoPE
|
| 14 |
+
- **Training Step:** 4000 / 4000
|
| 15 |
+
- **Current Loss:** 3.5641
|
| 16 |
- **Optimizer:** Hybrid Muon (2D Dense Linears) + AdamW (MoE Nodes, Embeddings)
|