darin-tanner commited on
Commit
5a43bee
·
verified ·
1 Parent(s): 8a15845

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +8 -3
README.md CHANGED
@@ -19,9 +19,9 @@ tags:
19
 
20
  # BCLM-1-Small
21
 
22
- We are releasing **BCLM-1**, an open family of flagship language models that deliver strong, predictable performance from a compact parameter budget.
23
 
24
- BCLM-1 is built on a hybrid SSM/attention architecture (Bulldog) for efficient training and inference. It handles writing, summarization, and recall tasks out of the box, holding onto information across its full context window where similarly sized models fall apart.
25
 
26
  ## Models
27
 
@@ -68,7 +68,12 @@ Transformers support coming soon.
68
 
69
  ## Training
70
 
71
- BCLM-1-Small was pretrained for 24,000 steps, reaching a training loss of 2.10 (perplexity 8). The subsequent SFT stage ran for 6,000 steps, achieving a validation loss of 1.84.
 
 
 
 
 
72
 
73
  ## Tokenizer and Supported Languages
74
 
 
19
 
20
  # BCLM-1-Small
21
 
22
+ We are releasing **BCLM-1**, an open family of flagship language models that deliver strong, predictable performance across a range of tasks.
23
 
24
+ BCLM-1 is built on a hybrid SSM/attention architecture (Bulldog) for efficient training and inference, excelling at writing, summarization, and recall out of the box.
25
 
26
  ## Models
27
 
 
68
 
69
  ## Training
70
 
71
+ Trained on NVIDIA H200 GPUs.
72
+
73
+ | Stage | Steps | Loss | Perplexity (ppl) |
74
+ |---|---|---|---|
75
+ | Pretraining | 24,000 | 2.11 | 8.25 |
76
+ | SFT | 6,000 | 1.85 | 6.36 |
77
 
78
  ## Tokenizer and Supported Languages
79