--- license: apache-2.0 library_name: transformers tags: - coverage - evaluation --- # MacroCoverageLM-1B
MacroCoverageLM-1B

Apache-2.0
## 1. Introduction MacroCoverageLM-1B is an open checkpoint family trained for broad-coverage reasoning. Every checkpoint in this release comes from the same pretraining run and differs only in how many optimisation steps it received. The release reports the run the process selected, with each benchmark suite weighted equally.

## 2. Evaluation Results ### Comprehensive Benchmark Results
| | Benchmark | Base-1B | Chat-1B | MacroCoverageLM-1B-Small | MacroCoverageLM-1B | |---|---|---|---|---|---| | **Core Reasoning Tasks** | Math Reasoning | 0.548 | 0.561 | 0.572 | 0.450 | | | Logical Reasoning | 0.548 | 0.561 | 0.572 | 0.450 | | | Common Sense | 0.548 | 0.561 | 0.572 | 0.700 | | **Language Understanding** | Reading Comprehension | 0.548 | 0.561 | 0.572 | 0.700 | | | Question Answering | 0.548 | 0.561 | 0.572 | 0.450 | | | Text Classification | 0.548 | 0.561 | 0.572 | 0.700 | | | Sentiment Analysis | 0.548 | 0.561 | 0.572 | 0.700 | | **Generation Tasks** | Code Generation | 0.548 | 0.561 | 0.572 | 0.450 | | | Creative Writing | 0.548 | 0.561 | 0.572 | 0.700 | | | Dialogue Generation | 0.548 | 0.561 | 0.572 | 0.700 | | | Summarization | 0.548 | 0.561 | 0.572 | 0.700 | | **Specialized Capabilities** | Translation | 0.548 | 0.561 | 0.572 | 0.700 | | | Knowledge Retrieval | 0.548 | 0.561 | 0.572 | 0.700 | | | Instruction Following | 0.548 | 0.561 | 0.572 | 0.450 | | | Safety Evaluation | 0.548 | 0.561 | 0.572 | 0.450 |
### Overall Performance Summary Scores are the equal-weight mean over the fifteen suites. Each cell is the accuracy over the full held-out split.