CoolFace
Modelpublic

appvoid/arco-2

sourceHugging Faceapache-2.0updated 1y agoView on Hugging Face
7likes47downloads
Model Card

<style> img{ user-select: none; transition: all 0.2s ease; border-radius: .5rem; } img:hover{ transform: rotate(2deg); filter: invert(100%); } @import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap'); </style>

<div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;">

cubby

This is a passthrough of arco with an experimental model. It improved on arc challenge, only missing 1.2 points to get to the level of modern 3b baseline performance.

If you prefer answering multilingual, general knowledge, trivially simple questions chose qwen or llama. If you prefer solving trivially simple english tasks while being half the size, chose arco.

prompt

there is no prompt intentionally set.

benchmarks

zero-shot results from state-of-the-art small language models

ParametersModelMMLUARC-CHellaSwagPIQAWinograndeAverage
0.5bqwen 244.1328.9249.0569.3156.9949.68
0.3bsmollm25.5237.7156.4171.9359.2750.17
0.5bdanube 324.8136.1860.4673.7861.0151.25
0.5bqwen 2.547.2931.8352.1770.2957.0651.72
0.5barco26.1737.2962.8874.3762.2752.60
0.5barco 225.5138.8263.0274.7061.2552.66
supporters

<a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a>

trivia

arco also means "arc optimized" hence the focus on this cognitive-based benchmark. </div>