README.md

---
license: other
language:
- en
- es
- fr
- de
- it
- pt
- ru
- ar
- hi
- ko
- zh
library_name: transformers
base_model:
- arcee-ai/Trinity-Mini-Base-Pre-Anneal
license_link: LICENSE
license_name: openmdw-1.1
---
<div align="center">
  <picture>
    <img
      src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/i-v1KyAMOW_mgVGeic9WJ.png"
      alt="Arcee Trinity Mini"
      style="max-width: 100%; height: auto;"
    >
  </picture>
</div>

# Trinity Mini Base

Trinity Mini is an Arcee AI 26B MoE model with 3B active parameters. It is the medium-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.

This base model *pre* fine tuning, and so is not suitable for chatting, and should be trained for your specific domain before use.

Trinity Mini is trained on 10T tokens gathered and curated through a key partnership with [Datology](https://www.datologyai.com/), building upon the excellent dataset we used on [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) with additional math and code.

Training was performed on a cluster of 512 H200 GPUs powered by [Prime Intellect](https://www.primeintellect.ai/) using HSDP parallelism.

More details, including key architecture decisions, can be found on our blog [here](https://www.arcee.ai/blog/the-trinity-manifesto)

Try it out the reasoning model now at [chat.arcee.ai](http://chat.arcee.ai/) or download here: [arcee-ai/Trinity-Mini](https://huggingface.co/arcee-ai/Trinity-Mini)

***

## Model Details

* **Model Architecture:** AfmoeForCausalLM
* **Parameters:** 26B, 3B active
* **Experts:** 128 total, 8 active, 1 shared
* **Context length:** 128k
* **Training Tokens:** 10T
* **License:** [OpenMDW-1.1](https://huggingface.co/arcee-ai/Trinity-Mini-Base#license)

## Benchmarks

### General Benchmarks

| Benchmark | Score |
|-----------|-------|
| ARC-Challenge | 90.0% |
| CommonsenseQA | 79.6% |
| OpenBookQA | 89.0% |
| Winogrande | 75.9% |
| MMLU (5-shot) | 74.7% |
| AGI Eval English | 61.8% |
| BBH CoT (3-shot) | 54.2% |
| MMLU Pro | 45.6% |

### Math & Code Benchmarks

| Benchmark | Score |
|-----------|-------|
| GSM8K | 56.6% |
| Minerva MATH 500 | 51.8% |
| HumanEval+ | 57.3% |
| MBPP+ | 55.3% |

<div align="center">
  <picture>
      <img src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/sSVjGNHfrJKmQ6w8I18ek.png" style="background-color:ghostwhite;padding:5px;" width="17%" alt="Powered by Datology">
  </picture>
</div>

## Try out our reasoning tune

Trinity Mini is available today on openrouter:

https://openrouter.ai/arcee-ai/trinity-mini

```
curl -X POST "https://openrouter.ai/v1/chat/completions" \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "arcee-ai/trinity-mini",
    "messages": [
      {
        "role": "user",
        "content": "What are some fun things to do in New York?"
      }
    ]
  }'
```

## License

Trinity-Mini-Base is released under the OpenMDW-1.1 license.