Syed-Azim/JoyAI-LLM-Flash
022
1---2language:3- zh4- en5pipeline_tag: text-generation6---7<div align="center">8 <picture>9 <img src="figures/joyai-logo.png" width="30%" alt="JoyAI-LLM Flash">10 </picture>11</div>12<hr>13 14<div align="center" style="line-height: 1;">15 <a href="https://huggingface.co/jdopensource" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-JD-ffc107?color=ffc107&logoColor=white"/></a>16 <a href="https://huggingface.co/jdopensource/JoyAI-LLM-Flash/blob/main/LICENSE"><img alt="License" src="https://img.shields.io/badge/License-Modified_MIT-f5de53?&color=f5de53"/></a>17</div>18 19 20 21 22## 1. Model Introduction23 24JoyAI-LLM Flash is a state-of-the-art medium-sized instruct language model with 3 billion activated parameters and 48 billion total parameters. JoyAI-LLM Flash was pretrained on 20 trillion text tokens using Muon optimizer, followed by large-scale supervised fine-tuning (SFT), direct preference optimization (DPO), and reinforcement learning (RL) across diverse environments. JoyAI-LLM Flash achieves strong performance across frontier knowledge, reasoning, coding tasks and agentic capabilities.25 26### Key Features27 28- Fiber Bundle RL: Introduces fiber bundle theory into reinforcement learning, proposing a novel optimization framework, FiberPO. This method is specifically designed to handle the challenges of large-scale and heterogeneous agent training, improving stability and robustness under complex data distributions.29- Training-Inference Collaboration: apply Muon optimizer with dense MTP, develop novel optimization techniques to resolve instabilities while scaling up, delivering 1.3× to 1.7× the throughput of the non-MTP version.30- Agentic Intelligence: designed for tool use, reasoning, and autonomous problem-solving.31 32## 2. Model Summary33 34| | |35| :-----------------------------------------: | :----------------------: |36| **Architecture** | Mixture-of-Experts (MoE) |37| **Total Parameters** | 48B |38| **Activated Parameters** | 3B |39| **Number of Layers** (Dense layer included) | 40 |40| **Number of Dense Layers** | 1 |41| **Attention Hidden Dimension** | 2048 |42| **MoE Hidden Dimension** (per Expert) | 768 |43| **Number of Attention Heads** | 32 |44| **Number of Experts** | 256 |45| **Selected Experts per Token** | 8 |46| **Number of Shared Experts** | 1 |47| **Vocabulary Size** | 129K |48| **Context Length** | 128K |49| **Attention Mechanism** | MLA |50| **Activation Function** | SwiGLU |51| </div> | |52 53 54## 3. Evaluation Results55 56<table>57<thead>58<tr>59<th align="center">Benchmark</th>60<th align="center"><sup>JoyAI-LLM Flash</sup></th>61<th align="center"><sup>Qwen3-30B-A3B-Instuct-2507</sup></th>62<th align="center"><sup>GLM-4.7-Flash<br>(Non-thinking)</sup></th>63</tr>64</thead>65<tbody>66 67 68<tr>69<td align="center" colspan=8><strong>Knowledge & Alignment</strong></td>70</tr>71<tr>72<td align="center" style="vertical-align: middle">MMLU</td>73<td align="center" style="vertical-align: middle"><strong>89.50</strong></td>74<td align="center" style="vertical-align: middle">86.87</td>75<td align="center" style="vertical-align: middle">80.53</td>76</tr>77<tr>78<td align="center" style="vertical-align: middle">MMLU-Pro</td>79<td align="center" style="vertical-align: middle"><strong>81.02</strong></td>80<td align="center" style="vertical-align: middle">73.88</td>81<td align="center" style="vertical-align: middle">63.62</td>82</tr>83<tr>84<td align="center" style="vertical-align: middle">CMMLU</td>85<td align="center" style="vertical-align: middle"><strong>87.03</strong></td>86<td align="center" style="vertical-align: middle">85.88</td>87<td align="center" style="vertical-align: middle">75.85</td>88</tr>89<tr>90<td align="center" style="vertical-align: middle">GPQA-Diamond</td>91<td align="center" style="vertical-align: middle"><strong>74.43</strong></td>92<td align="center" style="vertical-align: middle">68.69</td>93<td align="center" style="vertical-align: middle">39.90</td>94</tr>95<tr>96<td align="center" style="vertical-align: middle">SuperGPQA</td>97<td align="center" style="vertical-align: middle"><strong>55.00</strong></td>98<td align="center" style="vertical-align: middle">52.00</td>99<td align="center" style="vertical-align: middle">32.00</td>100</tr>101<tr>102<td align="center" style="vertical-align: middle">LiveBench</td>103<td align="center" style="vertical-align: middle"><strong>72.90</strong></td>104<td align="center" style="vertical-align: middle">59.70</td>105<td align="center" style="vertical-align: middle">43.10</td>106</tr>107<tr>108<td align="center" style="vertical-align: middle">IFEval</td>109<td align="center" style="vertical-align: middle"><strong>86.69</strong></td>110<td align="center" style="vertical-align: middle">83.18</td>111<td align="center" style="vertical-align: middle">82.44</td>112</tr>113<tr>114<td align="center" style="vertical-align: middle">AlignBench</td>115<td align="center" style="vertical-align: middle"><strong>8.24</strong></td>116<td align="center" style="vertical-align: middle">8.07</td>117<td align="center" style="vertical-align: middle">6.85</td>118</tr>119<tr>120<td align="center" style="vertical-align: middle">HellaSwag</td>121<td align="center" style="vertical-align: middle"><strong>91.79</strong></td>122<td align="center" style="vertical-align: middle">89.90</td>123<td align="center" style="vertical-align: middle">60.84</td>124</tr>125 126<tr>127<td align="center" colspan=8><strong>Coding</strong></td>128</tr>129<tr>130<td align="center" style="vertical-align: middle">HumanEval</td>131<td align="center" style="vertical-align: middle"><strong>96.34</strong></td>132<td align="center" style="vertical-align: middle">95.12</td>133<td align="center" style="vertical-align: middle">74.39</td>134</tr>135<tr>136<td align="center" style="vertical-align: middle">LiveCodeBench</td>137<td align="center" style="vertical-align: middle"><strong>65.60</strong></td>138<td align="center" style="vertical-align: middle">39.71</td>139<td align="center" style="vertical-align: middle">27.43</td>140</tr>141<tr>142<td align="center" style="vertical-align: middle">SciCode</td>143<td align="center" style="vertical-align: middle"><strong>3.08/22.92</strong></td>144<td align="center" style="vertical-align: middle"><strong>3.08/22.92</strong></td>145<td align="center" style="vertical-align: middle">3.08/15.11</td>146</tr>147<tr>148<td align="center" colspan=8><strong>Mathematics</strong></td>149</tr>150<tr>151<td align="center" style="vertical-align: middle">GSM8K</td>152<td align="center" style="vertical-align: middle"><strong>95.83</strong></td>153<td align="center" style="vertical-align: middle">79.83</td>154<td align="center" style="vertical-align: middle">81.88</td>155</tr>156<tr>157<td align="center" style="vertical-align: middle">AIME2025</td>158<td align="center" style="vertical-align: middle"><strong>65.83</strong></td>159<td align="center" style="vertical-align: middle">62.08</td>160<td align="center" style="vertical-align: middle">24.17</td>161</tr>162<tr>163<td align="center" style="vertical-align: middle">MATH 500</td>164<td align="center" style="vertical-align: middle"><strong>97.10</strong></td>165<td align="center" style="vertical-align: middle">89.80</td>166<td align="center" style="vertical-align: middle">90.90</td>167</tr>168 169<tr>170<td align="center" colspan=8><strong>Agentic</strong></td>171</tr>172<tr>173<td align="center" style="vertical-align: middle">SWE-bench Verified</td>174<td align="center" style="vertical-align: middle"><strong>60.60</strong></td>175<td align="center" style="vertical-align: middle">24.44</td>176<td align="center" style="vertical-align: middle">51.60</td>177</tr>178<tr>179<td align="center" style="vertical-align: middle">Tau2-Retail</td>180<td align="center" style="vertical-align: middle"><strong>67.55</strong></td>181<td align="center" style="vertical-align: middle">53.51</td>182<td align="center" style="vertical-align: middle">62.28</td>183</tr>184<tr>185<td align="center" style="vertical-align: middle">Tau2-Airline</td>186<td align="center" style="vertical-align: middle"><strong>54.00</strong></td>187<td align="center" style="vertical-align: middle">32.00</td>188<td align="center" style="vertical-align: middle">52.00</td>189</tr>190<tr>191<td align="center" style="vertical-align: middle">Tau2-Telecom</td>192<td align="center" style="vertical-align: middle">79.83</td>193<td align="center" style="vertical-align: middle">4.39</td>194<td align="center" style="vertical-align: middle"><strong>88.60</strong></td>195</tr>196 197<tr>198<td align="center" colspan=8><strong>Long Context</strong></td>199</tr>200<tr>201<td align="center" style="vertical-align: middle">RULER</td>202<td align="center" style="vertical-align: middle"><strong>95.60</strong></td>203<td align="center" style="vertical-align: middle">89.66</td>204<td align="center" style="vertical-align: middle">56.12</td>205</tr>206</tbody>207</table>208 209 210## 4. Deployment211 212> [!Note]213> You can access JoyAI-LLM Flash API on https://docs.jdcloud.com/cn/jdaip/chat and we provide OpenAI/Anthropic-compatible API for you. 214> Currently, JoyAI-LLM Flash is recommended to run on the following inference engines:215 216* vLLM217* SGLang218 219The minimum version requirement for `transformers` is `4.57.1`.220 221Deployment examples can be found in the [Model Deployment Guide](docs/deploy_guidance.md).222 223 224 225## 5. Model Usage226 227The usage demos below demonstrate how to call our official API.228 229For third-party APIs deployed with vLLM or SGLang, please note that:230 231> [!Note] Recommended sampling parameters: `temperature=0.6`, `top_p=1.0`232 233### Chat Completion234 235This is a simple chat completion script which shows how to call JoyAI-Flash API.236 237```python238from openai import OpenAI239 240client = OpenAI(base_url="http://IP:PORT/v1", api_key="EMPTY")241 242 243def simple_chat(client: OpenAI):244 messages = [245 {246 "role": "user",247 "content": [248 {249 "type": "text",250 "text": "which one is bigger, 9.11 or 9.9? think carefully.",251 }252 ],253 },254 ]255 model_name = client.models.list().data[0].id256 response = client.chat.completions.create(257 model=model_name, messages=messages, stream=False, max_tokens=4096258 )259 print(f"response: {response.choices[0].message.content}")260 261 262if __name__ == "__main__":263 simple_chat(client)264```265 266 267### Tool call Completion268 269This is a simple toll call completion script which shows how to call JoyAI-Flash API.270 271```python272import json273 274from openai import OpenAI275 276client = OpenAI(base_url="http://IP:PORT/v1", api_key="EMPTY")277 278 279def my_calculator(expression: str) -> str:280 return str(eval(expression))281 282 283def rewrite(expression: str) -> str:284 return str(expression)285 286 287def simple_tool_call(client: OpenAI):288 messages = [289 {290 "role": "user",291 "content": [292 {293 "type": "text",294 "text": "use my functions to compute the results for the equations: 6+1",295 },296 ],297 },298 ]299 tools = [300 {301 "type": "function",302 "function": {303 "name": "my_calculator",304 "description": "A calculator that can evaluate a mathematical equation and compute its results.",305 "parameters": {306 "type": "object",307 "properties": {308 "expression": {309 "type": "string",310 "description": "The mathematical expression to evaluate.",311 },312 },313 "required": ["expression"],314 },315 },316 },317 {318 "type": "function",319 "function": {320 "name": "rewrite",321 "description": "Rewrite a given text for improved clarity",322 "parameters": {323 "type": "object",324 "properties": {325 "text": {326 "type": "string",327 "description": "The input text to rewrite",328 }329 },330 },331 },332 },333 ]334 model_name = client.models.list().data[0].id335 response = client.chat.completions.create(336 model=model_name,337 messages=messages,338 temperature=1.0,339 max_tokens=1024,340 tools=tools,341 tool_choice="auto",342 )343 tool_calls = response.choices[0].message.tool_calls344 345 results = []346 for tool_call in tool_calls:347 function_name = tool_call.function.name348 function_args = tool_call.function.arguments349 if function_name == "my_calculator":350 result = my_calculator(**json.loads(function_args))351 results.append(result)352 messages.append({"role": "assistant", "tool_calls": tool_calls})353 for tool_call, result in zip(tool_calls, results):354 messages.append(355 {356 "role": "tool",357 "tool_call_id": tool_call.id,358 "name": tool_call.function.name,359 "content": result,360 }361 )362 response = client.chat.completions.create(363 model=model_name,364 messages=messages,365 temperature=1.0,366 max_tokens=1024,367 )368 print(response.choices[0].message.content)369 370 371if __name__ == "__main__":372 simple_tool_call(client)373 374```375 376---377 378## 6. License379 380Both the code repository and the model weights are released under the [Modified MIT License](LICENSE).