Modelpublic

inclusionAI/Ling-flash-base-2.0

sourceHugging Facemitupdated 1y agoView on Hugging Face

31likes175downloads

Model Card

🤗 <a href="https://huggingface.co/inclusionAI">Hugging Face</a>&nbsp&nbsp | &nbsp&nbsp🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope</a>

Introduction

Today, _Ling-flash-2.0 is officially open-sourced! 🚀 Following the release of the language model [Ling-mini-2.0](https://huggingface.co/inclusionAI/Ling-mini-2.0) and the thinking model [Ring-mini-2.0](https://huggingface.co/inclusionAI/Ring-mini-2.0), we are now open-sourcing the third MoE LLM under the Ling 2.0 architecture: Ling-flash-2.0, a language model with 100B total parameters and 6.1B activated parameters (4.8B non-embedding). Trained on 20T+ tokens of high-quality data, together with supervised fine-tuning and multi-stage reinforcement learning, Ling-flash-2.0 achieves SOTA performance among dense models under 40B parameters, despite activating only ~6B parameters. Compared to MoE models with larger activation/total parameters, it also demonstrates strong competitiveness. Notably, it delivers outstanding performance in complex reasoning, code generation, and frontend development_.

Powerful Complex Reasoning Abilities

We conducted a comprehensive evaluation of Ling-flash-2.0’s reasoning capabilities, reporting strong results on representative benchmarks:

—_Multi-disciplinary knowledge reasoning_: GPQA-Diamond, MMLU-Pro
—_Advanced mathematical reasoning_: AIME 2025, Omni-MATH, OptMATH (advanced mathematical optimization tasks)
—_Challenging code generation_: LiveCodeBench v6, CodeForces-Elo
—_Logical reasoning_: KOR-Bench, ARC-Prize
—_Key regulated industries (Finance, Healthcare)_: FinanceReasoning, HealthBench

Compared with _dense models under 40B (e.g., Qwen3-32B-Non-Thinking, Seed-OSS-36B-Instruct (think budget=0)) and larger-activation/total-parameter MoE models (e.g., Hunyuan-A13B-Instruct, GPT-OSS-120B/low), Ling-flash-2.0 demonstrates stronger complex reasoning power. Moreover, it shows high competitiveness on creative tasks (Creative Writing v3). <img src="https://mdn.alipayobjects.com/huameifi95qp/afts/img/zxAvQ7QtrAwAAAAAQqAAAAgADkZ7AQFr/fmt.webp"/>

Efficient Architecture, High-Speed Inference

Guided by Ling Scaling Laws, Ling 2.0 adopts a _1/32 activation-ratio MoE architecture, optimized across multiple design choices: expert granularity, shared-expert ratio, attention balance, aux-loss-free + sigmoid routing strategy, MTP layers, QK-Norm, Partial-RoPE, and more. These refinements enable small-activation MoE models to achieve 7× efficiency gains over equivalent dense architectures. In other words, with just 6.1B activated parameters (4.8B non-embedding), Ling-flash-2.0_ can match the performance of ~40B dense models. Thanks to its small activation size, it also delivers major inference speed advantages:

—On _H20 hardware, Ling-flash-2.0 achieves 200+ tokens/s, offering 3× speedups_ compared to 36B dense models in everyday use.
—With _YaRN extrapolation, it supports 128K context length, and as output length grows, its relative speedup can reach 7× or more_.

Model Downloads

You can download the following table to see the various stage of Ling-flash-2.0 models. If you are located in mainland China, we also provide the model on ModelScope.cn to speed up the download process.

Model	Context Length	Download
Ling-flash-base-2.0	32K -> 128K (YaRN)	🤗 HuggingFace <br>🤖 ModelScope
Ling-flash-2.0	32K -> 128K (YaRN)	🤗 HuggingFace <br>🤖 ModelScope

</center>

Note: If you are interested in previous version, please visit the past model collections in Huggingface or ModelScope.

Quickstart

🤗 Hugging Face Transformers

Here is a code snippet to show you how to use the chat model with transformers:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "inclusionAI/Ling-flash-base-2.0"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype = torch.bfloat16,
    device_map='auto',
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_name)

text = "What is the capital of France?"
model_inputs = tokenizer([text], return_tensors="pt", return_token_type_ids=False).to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=512
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]

🤖 ModelScope

If you're in mainland China, we strongly recommend you to use our model from 🤖 <a href="https://modelscope.cn/organization/inclusionAI">ModelScope</a>.

Deployment

vLLM

vLLM supports offline batched inference or launching an OpenAI-Compatible API Service for online inference.

Environment Preparation

Since the Pull Request (PR) has not been submitted to the vLLM community at this stage, please prepare the environment by following the steps below:

bash

git clone -b v0.10.0 https://github.com/vllm-project/vllm.git
cd vllm
wget https://raw.githubusercontent.com/inclusionAI/Ling-V2/refs/heads/main/inference/vllm/bailing_moe_v2.patch
git apply bailing_moe_v2.patch
pip install -e .

Offline Inference:

python

from vllm import LLM, SamplingParams

sampling_params = SamplingParams(temperature=0.7, top_p=0.8, repetition_penalty=1.05, max_tokens=16384)

llm = LLM(model="inclusionAI/Ling-flash-base-2.0", dtype='bfloat16')
text = "What is the capital of France?"

outputs = llm.generate([text], sampling_params)

Online Inference:

bash

vllm serve inclusionAI/Ling-flash-base-2.0 \
              --tensor-parallel-size 2 \
              --pipeline-parallel-size 1 \
              --use-v2-block-manager \
              --gpu-memory-utilization 0.90

To handle long context in vLLM using YaRN, we need to follow these two steps:

1.Add a rope_scaling field to the model's config.json file, for example:

json

{
  ...,
  "rope_scaling": {
    "factor": 4.0,
    "original_max_position_embeddings": 32768,
    "type": "yarn"
  }
}

1.Use an additional parameter --max-model-len to specify the desired maximum context length when starting the vLLM service.

For detailed guidance, please refer to the vLLM `instructions`.

SGLang

Environment Preparation

We will later submit our model to SGLang official release, now we can prepare the environment following steps:

shell

pip3 install sglang==0.5.2rc0 sgl-kernel==0.3.7.post1

You can use docker image as well:

shell

docker pull lmsysorg/sglang:v0.5.2rc0-cu126

Then you should apply patch to sglang installation:

shell

# patch command is needed, run `yum install -y patch` if needed
patch -d `python -c 'import sglang;import os; print(os.path.dirname(sglang.__file__))'` -p3 < inference/sglang/bailing_moe_v2.patch

Run Inference

BF16 and FP8 models are supported by SGLang now, it depends on the dtype of the model in ${MODEL_PATH}. They both share the same command in the following:

—Start server:

shell

python -m sglang.launch_server \
    --model-path $MODLE_PATH \
    --host 0.0.0.0 --port $PORT \
    --trust-remote-code \
    --attention-backend fa3

MTP is supported for base model, and not yet for chat model. You can add parameter --speculative-algorithm NEXTN to start command.

—Client:

shell

curl -s http://localhost:${PORT}/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "auto", "messages": [{"role": "user", "content": "What is the capital of France?"}]}'

More usage can be found here

Finetuning

We recommend you to use Llama-Factory to finetune Ling.

License

This code repository is licensed under the MIT License.