123sg's picture
Release checkpoint_3962 (highest eval_accuracy 0.790) with benchmark scores
d8758a7 verified
|
Raw History Blame Contribute Delete
1.41 kB
---
license: apache-2.0
library_name: transformers
pipeline_tag: text-generation
---
# ResearchModel-Prairie
![release overview](figures/fig1.png)
## Release scope
This repository records the selected checkpoint from the `prairie-release-candidate` training run. The package includes its configuration, weights, tokenizer metadata, training state, and release figures.
## Evaluation matrix
| Evaluation | Score |
|---|---:|
| Math Reasoning | 0.699 |
| Logical Reasoning | 0.850 |
| Common Sense | 0.813 |
| Reading Comprehension | 0.800 |
| Question Answering | 0.689 |
| Text Classification | 0.850 |
| Sentiment Analysis | 0.833 |
| Code Generation | 0.783 |
| Creative Writing | 0.764 |
| Dialogue Generation | 0.748 |
| Summarization | 0.826 |
| Translation | 0.837 |
| Knowledge Retrieval | 0.753 |
| Instruction Following | 0.822 |
| Safety Evaluation | 0.808 |
Scores are reported on a zero-to-one scale and rounded to three decimal places.
## Usage
Load the configuration and weights with a compatible Transformers runtime. Review `trainer_state.json` for the selected training step and `tokenizer_config.json` for tokenizer settings.
![benchmark profile](figures/fig2.png)
## Release notes
The checkpoint was selected by the workspace evaluation runner using its aggregate `eval_accuracy` result. No other checkpoint files belong in this repository.
![training summary](figures/fig3.png)