File size: 1,054 Bytes
170dc03 78a1803 170dc03 665f2ec 170dc03 f788f52 8d4c560 f788f52 78a1803 170dc03 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | ---
base_model:
- appvoid/arco
- h2oai/h2o-danube3-500m-base
library_name: transformers
tags:
- mergekit
- merge
---
# arco+
This is an untrained passthrough model based on arco and danube as a first effort to train a small enough reasoning language model that generalizes across all kind of reasoning tasks.
#### Benchmarks
| Parameters | Model | MMLU | ARC | HellaSwag | PIQA | Winogrande | Average |
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------|
| 488m | arco-lite | **23.22** | 33.45 | 56.55| 69.70 | **59.19**| 48.46 |
| 773m | arco-plus | 23.06 | **36.43** | **60.09**|**72.36**| **60.46**| **50.48** |
#### Configuration
The following YAML configuration was used to produce this model:
```yaml
slices:
- sources:
- model: appvoid/arco
layer_range: [0, 14]
- sources:
- model: h2oai/h2o-danube3-500m-base
layer_range: [4, 16]
merge_method: passthrough
dtype: float16
```
|