---
license: apache-2.0
library_name: transformers
tags:
- coverage
- evaluation
---
# MacroCoverageLM-1B
## 1. Introduction
MacroCoverageLM-1B is an open checkpoint family trained for broad-coverage
reasoning. Every checkpoint in this release comes from the same pretraining
run and differs only in how many optimisation steps it received.
The release reports the run the process selected, with each benchmark suite
weighted equally.
## 2. Evaluation Results
### Comprehensive Benchmark Results
| | Benchmark | Base-1B | Chat-1B | MacroCoverageLM-1B-Small | MacroCoverageLM-1B |
|---|---|---|---|---|---|
| **Core Reasoning Tasks** | Math Reasoning | 0.548 | 0.561 | 0.572 | 0.450 |
| | Logical Reasoning | 0.548 | 0.561 | 0.572 | 0.450 |
| | Common Sense | 0.548 | 0.561 | 0.572 | 0.700 |
| **Language Understanding** | Reading Comprehension | 0.548 | 0.561 | 0.572 | 0.700 |
| | Question Answering | 0.548 | 0.561 | 0.572 | 0.450 |
| | Text Classification | 0.548 | 0.561 | 0.572 | 0.700 |
| | Sentiment Analysis | 0.548 | 0.561 | 0.572 | 0.700 |
| **Generation Tasks** | Code Generation | 0.548 | 0.561 | 0.572 | 0.450 |
| | Creative Writing | 0.548 | 0.561 | 0.572 | 0.700 |
| | Dialogue Generation | 0.548 | 0.561 | 0.572 | 0.700 |
| | Summarization | 0.548 | 0.561 | 0.572 | 0.700 |
| **Specialized Capabilities** | Translation | 0.548 | 0.561 | 0.572 | 0.700 |
| | Knowledge Retrieval | 0.548 | 0.561 | 0.572 | 0.700 |
| | Instruction Following | 0.548 | 0.561 | 0.572 | 0.450 |
| | Safety Evaluation | 0.548 | 0.561 | 0.572 | 0.450 |
### Overall Performance Summary
Scores are the equal-weight mean over the fifteen suites. Each cell is the
accuracy over the full held-out split.