Update README.md
Browse files
README.md
CHANGED
|
@@ -19,9 +19,9 @@ tags:
|
|
| 19 |
|
| 20 |
# BCLM-1-Small
|
| 21 |
|
| 22 |
-
We are releasing **BCLM-1**, an open family of flagship language models that deliver strong, predictable performance
|
| 23 |
|
| 24 |
-
BCLM-1 is built on a hybrid SSM/attention architecture (Bulldog) for efficient training and inference
|
| 25 |
|
| 26 |
## Models
|
| 27 |
|
|
@@ -68,7 +68,12 @@ Transformers support coming soon.
|
|
| 68 |
|
| 69 |
## Training
|
| 70 |
|
| 71 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 72 |
|
| 73 |
## Tokenizer and Supported Languages
|
| 74 |
|
|
|
|
| 19 |
|
| 20 |
# BCLM-1-Small
|
| 21 |
|
| 22 |
+
We are releasing **BCLM-1**, an open family of flagship language models that deliver strong, predictable performance across a range of tasks.
|
| 23 |
|
| 24 |
+
BCLM-1 is built on a hybrid SSM/attention architecture (Bulldog) for efficient training and inference, excelling at writing, summarization, and recall out of the box.
|
| 25 |
|
| 26 |
## Models
|
| 27 |
|
|
|
|
| 68 |
|
| 69 |
## Training
|
| 70 |
|
| 71 |
+
Trained on NVIDIA H200 GPUs.
|
| 72 |
+
|
| 73 |
+
| Stage | Steps | Loss | Perplexity (ppl) |
|
| 74 |
+
|---|---|---|---|
|
| 75 |
+
| Pretraining | 24,000 | 2.11 | 8.25 |
|
| 76 |
+
| SFT | 6,000 | 1.85 | 6.36 |
|
| 77 |
|
| 78 |
## Tokenizer and Supported Languages
|
| 79 |
|