Modelpublic
Marcus2112/pythia-160m-minipile_nano_density-proportioned
0likes6downloads
Model Card
| Benchmark | Measure | 160M MiniPile | 160M Density Nano | Percentage Difference in Means | |
|---|---|---|---|---|---|
| ARC-Challenge | acc | ↑ | 0.2125 ± 0.0120 | 0.1928 ± 0.0115 | -9.2706 |
| MMLU | acc | ↑ | 0.2699 ± 0.0037 | 0.2295 ± 0.0035 | -14.9685 |
| HellaSwag | acc | ↑ | 0.2560 ± 0.0044 | 0.2596 ± 0.0044 | 1.4063 |
| WinoGrande | acc | ↑ | 0.4720 ± 0.0140 | 0.5036 ± 0.0141 | 6.6949 |
| Lambada (OpenAI) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (OpenAI) | perplexity | ↓ | 3033175.2693 ± 288926.5827 | 1871303.2176 ± 150515.6406 | -38.3055 |
| Lambada (Std) | acc | ↑ | 0.0000 ± 0.0000 | 0.0000 ± 0.0000 | - |
| Lambada (Std) | perplexity | ↓ | 27067951.3461 ± 2710040.1910 | 10513877.8584 ± 92624.3394 | -61.1575 |
| BLiMP | acc | ↑ | 0.5194 ± 0.0018 | 0.5364 ± 0.0016 | 3.2730 |