jdopensource/JoyAI-LLM-Flash-Base

<div align="center"> <picture> <img src="figures/joyai-logo.png" width="30%" alt="JoyAI-LLM Flash-Base"> </picture> </div> <hr>

<div align="center" style="line-height: 1;"> <a href="https://huggingface.co/jdopensource" target="blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-JD-ffc107?color=ffc107&logoColor=white"/></a> <a href="https://huggingface.co/jdopensource/JoyAI-LLM-Flash-Base/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-ModifiedMIT-f5de53?&color=f5de53"/></a> </div>

1. Model Introduction

JoyAI-LLM Flash-Base is a state-of-the-art mixture-of-experts (MoE) language model with 3 billion activated parameters and 48 billion total parameters. Trained with the Muon optimizer, JoyAI Flash-base achieves exceptional performance across frontier knowledge, reasoning, and coding tasks while being meticulously optimized for agentic capabilities. JoyAI-LLM Flash series aim to accelarate high-throughput, latency-sensitive applications where cost per query must remain minimal.

Key Features

• —Training-Inference Collaboration: apply Muon optimizer with dense MTP, develop novel optimization techniques to resolve instabilities while scaling up, delivering 1.3× to 1.7× the throughput of the non-MTP version.
• —Agentic Intelligence: Specifically designed for tool use, reasoning, and autonomous problem-solving.

2. Model Summary

3. Evaluation Results

<table> <thead> <tr> <th align="center">Benchmark</th> <th align="center">^{JoyAI-LLM Flash-base}</th> <th align="center">^{Qwen3-30B-A3B-base}</th> </tr> </thead> <tbody>

<tr> <td align="center" style="vertical-align: middle">MMLU</td> <td align="center" style="vertical-align: middle"><strong>84.70</strong></td> <td align="center" style="vertical-align: middle">82.12</td> </tr> <tr> <td align="center" style="vertical-align: middle">MMLU-Pro</td> <td align="center" style="vertical-align: middle"><strong>73.14</strong></td> <td align="center" style="vertical-align: middle">61.76</td> </tr> <tr> <td align="center" style="vertical-align: middle">CMMLU</td> <td align="center" style="vertical-align: middle">83.09</td> <td align="center" style="vertical-align: middle"><strong>83.60</strong></td> </tr> <tr> </tr>

<tr> <td align="center" style="vertical-align: middle">HumanEval</td> <td align="center" style="vertical-align: middle">85.37</td> <td align="center" style="vertical-align: middle"><strong>87.80</strong></td> </tr> <tr> <td align="center" style="vertical-align: middle">LiveCodeBench</td> <td align="center" style="vertical-align: middle"><strong>39.91</strong></td> <td align="center" style="vertical-align: middle">37.34</td> </tr> <tr></tr>

<tr> <td align="center" style="vertical-align: middle">GSM8K</td> <td align="center" style="vertical-align: middle">88.78</td> <td align="center" style="vertical-align: middle"><strong>90.37</strong></td> </tr> <tr> </tr> <tr> <td align="center" style="vertical-align: middle">MATH</td> <td align="center" style="vertical-align: middle"><strong>78.16</strong></td> <td align="center" style="vertical-align: middle">59.60</td> </tr> <tr> <td align="center" style="vertical-align: middle">MATH 500</td> <td align="center" style="vertical-align: middle"><strong>77.00</strong></td> <td align="center" style="vertical-align: middle">58.00</td> </tr>

</tbody> </table>

4. License

Both the code repository and the model weights are released under the Modified MIT License.