Update README.md
Browse files
README.md
CHANGED
|
@@ -34,4 +34,5 @@ Rather than training one model, we trained multiple models, each on a different
|
|
| 34 |
- Initializer Range: `0.2`
|
| 35 |
- RMS Norm Eps: `1e-06`
|
| 36 |
- Pretraining Tp: `1`
|
| 37 |
-
- Use Cache: `false`
|
|
|
|
|
|
| 34 |
- Initializer Range: `0.2`
|
| 35 |
- RMS Norm Eps: `1e-06`
|
| 36 |
- Pretraining Tp: `1`
|
| 37 |
+
- Use Cache: `false`
|
| 38 |
+
- **Total Parameters**: `1,460,416`
|