File size: 2,191 Bytes
a074eb3 e5fefea a074eb3 6637d01 8a9c528 e5fefea a074eb3 6637d01 e5fefea 6637d01 c862cb8 6637d01 8a9c528 6637d01 8a9c528 c862cb8 6637d01 8a9c528 6637d01 c862cb8 8a9c528 c862cb8 224e3d3 8a9c528 224e3d3 8a9c528 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | ---
language: en
license: mit
library_name: transformers
pipeline_tag: feature-extraction
tags:
- bert
- evaluation
---
# MyAwesomeModel
This repository contains the checkpoint selected from the 10 candidates found in the workspace.
## Selected checkpoint
- **Checkpoint:** `step_1000`
- **Selection metric:** `eval_accuracy`
- **Best eval accuracy:** **0.828**
- **Selection rule:** highest `eval_accuracy` (higher is better)
### Candidate comparison
| Checkpoint | eval_accuracy |
|---|---:|
| `step_100` | 0.517 |
| `step_200` | 0.603 |
| `step_300` | 0.667 |
| `step_400` | 0.714 |
| `step_500` | 0.750 |
| `step_600` | 0.776 |
| `step_700` | 0.795 |
| `step_800` | 0.809 |
| `step_900` | 0.820 |
| `step_1000` | 0.828 |
## Detailed evaluation results
The selected checkpoint was evaluated on all 15 workspace benchmarks. Scores are reported to three decimal places.
| Benchmark | Score |
|---|---:|
| Math Reasoning | 0.550 |
| Logical Reasoning | 0.819 |
| Common Sense | 0.736 |
| Reading Comprehension | 0.700 |
| Question Answering | 0.607 |
| Text Classification (`eval_accuracy`) | 0.828 |
| Sentiment Analysis | 0.792 |
| Code Generation | 0.650 |
| Creative Writing | 0.610 |
| Dialogue Generation | 0.644 |
| Summarization | 0.767 |
| Translation | 0.804 |
| Knowledge Retrieval | 0.676 |
| Instruction Following | 0.758 |
| Safety Evaluation | 0.739 |
Machine-readable results for all checkpoints and all 15 benchmarks are available in [`evaluation_results.json`](./evaluation_results.json).
## Architecture and usage
The supplied configuration declares `model_type: bert` and architecture `BertModel`.
```python
from transformers import AutoModel
model = AutoModel.from_pretrained("ASD12D21321/MyAwesomeModel")
```
## Evaluation notes
`eval_accuracy` is the text-classification benchmark score from the workspace evaluation calculator. Checkpoint selection used only the highest `eval_accuracy`; the other 14 benchmark scores are reported for evaluation detail and did not affect selection.
## Limitations
The supplied checkpoint is a minimal test artifact. Validate weights and behavior before production use. The workspace did not include tokenizer files.
|