Update to files

# TalosDDM

254,272 parameters
64 hidden size
2 layers
4 attention heads
2 KV heads
1,024 vocabulary
512 max sequence length
FP32
PyTorch 2.11.0+cu128
CUDA 12.8.1
NVIDIA Tesla T4

### Training Loss

Step 0 — 6.9177
Step 25 — 5.3568
Step 50 — 3.9134
Step 75 — 2.6434
Step 100 — 1.6195
Step 125 — 1.0475
Step 150 — 0.5186
Step 175 — 0.2652

### MoE Configuration

128 hidden size
3 layers
8 attention heads
4 KV heads
16 head dimension
2,048 vocabulary
16 experts
3 active experts
1 shared expert

README.md

@@ -1,3 +1,46 @@
 ---
 license: mit
+datasets:
+- Amirjalaly/ooast_prompts
+language:
+- en
+tags:
+- DDM
+- lightwieght
 ---
+# TalosDDM
+
+254,272 parameters
+64 hidden size
+2 layers
+4 attention heads
+2 KV heads
+1,024 vocabulary
+512 max sequence length
+FP32
+PyTorch 2.11.0+cu128
+CUDA 12.8.1
+NVIDIA Tesla T4
+
+### Training Loss
+
+Step 0 — 6.9177
+Step 25 — 5.3568
+Step 50 — 3.9134
+Step 75 — 2.6434
+Step 100 — 1.6195
+Step 125 — 1.0475
+Step 150 — 0.5186
+Step 175 — 0.2652
+
+### MoE Configuration
+
+128 hidden size
+3 layers
+8 attention heads
+4 KV heads
+16 head dimension
+2,048 vocabulary
+16 experts
+3 active experts
+1 shared expert