Ares-Realm-Studios/Trinity-Mini-Base

1---2license: apache-2.03language:4- en5- es6- fr7- de8- it9- pt10- ru11- ar12- hi13- ko14- zh15library_name: transformers16base_model:17- arcee-ai/Trinity-Mini-Base-Pre-Anneal18---19<div align="center">20  <picture>21    <img22      src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/i-v1KyAMOW_mgVGeic9WJ.png"23      alt="Arcee Trinity Mini"24      style="max-width: 100%; height: auto;"25    >26  </picture>27</div>28 29# Trinity Mini Base30 31Trinity Mini is an Arcee AI 26B MoE model with 3B active parameters. It is the medium-sized model in our new Trinity family, a series of open-weight models for enterprise and tinkerers alike.32 33This base model *pre* fine tuning, and so is not suitable for chatting, and should be trained for your specific domain before use.34 35Trinity Mini is trained on 10T tokens gathered and curated through a key partnership with [Datology](https://www.datologyai.com/), building upon the excellent dataset we used on [AFM-4.5B](https://huggingface.co/arcee-ai/AFM-4.5B) with additional math and code.36 37Training was performed on a cluster of 512 H200 GPUs powered by [Prime Intellect](https://www.primeintellect.ai/) using HSDP parallelism.38 39More details, including key architecture decisions, can be found on our blog [here](https://www.arcee.ai/blog/the-trinity-manifesto)40 41Try it out the reasoning model now at [chat.arcee.ai](http://chat.arcee.ai/) or download here: [arcee-ai/Trinity-Mini](https://huggingface.co/arcee-ai/Trinity-Mini)42 43***44 45## Model Details46 47* **Model Architecture:** AfmoeForCausalLM48* **Parameters:** 26B, 3B active49* **Experts:** 128 total, 8 active, 1 shared50* **Context length:** 128k51* **Training Tokens:** 10T52* **License:** [Apache 2.0](https://huggingface.co/arcee-ai/Trinity-Mini-Base#license)53 54## Benchmarks55 56### General Benchmarks57 58| Benchmark | Score |59|-----------|-------|60| ARC-Challenge | 90.0% |61| CommonsenseQA | 79.6% |62| OpenBookQA | 89.0% |63| Winogrande | 75.9% |64| MMLU (5-shot) | 74.7% |65| AGI Eval English | 61.8% |66| BBH CoT (3-shot) | 54.2% |67| MMLU Pro | 45.6% |68 69### Math & Code Benchmarks70 71| Benchmark | Score |72|-----------|-------|73| GSM8K | 56.6% |74| Minerva MATH 500 | 51.8% |75| HumanEval+ | 57.3% |76| MBPP+ | 55.3% |77 78<div align="center">79  <picture>80      <img src="https://cdn-uploads.huggingface.co/production/uploads/6435718aaaef013d1aec3b8b/sSVjGNHfrJKmQ6w8I18ek.png" style="background-color:ghostwhite;padding:5px;" width="17%" alt="Powered by Datology">81  </picture>82</div>83 84## Try out our reasoning tune85 86Trinity Mini is available today on openrouter:87 88https://openrouter.ai/arcee-ai/trinity-mini89 90```91curl -X POST "https://openrouter.ai/v1/chat/completions" \92  -H "Authorization: Bearer $OPENROUTER_API_KEY" \93  -H "Content-Type: application/json" \94  -d '{95    "model": "arcee-ai/trinity-mini",96    "messages": [97      {98        "role": "user",99        "content": "What are some fun things to do in New York?"100      }101    ]102  }'103```104 105## License106 107Trinity-Mini-Base is released under the Apache-2.0 license.