Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -11,6 +11,6 @@ tags:
|
|
| 11 |
|
| 12 |
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
|
| 13 |
- **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
|
| 14 |
-
- **Training Step:**
|
| 15 |
-
- **Loss:**
|
| 16 |
- **Active Parameters per token:** ~34M (out of 56.5M total)
|
|
|
|
| 11 |
|
| 12 |
- **Architecture:** Cellular Nodes with Shared Dense SwiGLU Core
|
| 13 |
- **Distillation Source:** Cosmopedia-v2 (Mixtral-8x22B) & SmolTalk (Llama-3.1-70B)
|
| 14 |
+
- **Training Step:** 2000 / 4000
|
| 15 |
+
- **Loss:** 3.5633
|
| 16 |
- **Active Parameters per token:** ~34M (out of 56.5M total)
|