learning-unit/L1-16B-A3B

<p align="center"> <img src="banner.png" alt="L1" style="width: 80%;"> </p>

Learning Unit 1

L1 (Learning Unit 1) is the first language model from Lunit and Lunit Consortium, purpose-built for the medical domain. Derived from Gravity-16B-A3B-Base, L1 is designed for clinical reasoning and decision support.

✨ Key Highlights

• —🩺 Medical-Domain Specialized: Developed specifically for clinical reasoning and medical decision support
• —⚡ Efficient MoE: Only 3B parameters active per token out of 16.24B total — fast inference with high capacity
• —💭 Thinking Model: Performs step-by-step reasoning in <think> tags before generating the final answer

Note: L1 reasons internally using <think>...</think> blocks before producing a response. This chain-of-thought process improves answer quality but consumes additional tokens. Set max_tokens accordingly (recommended: 2048+).

📋 Model Specifications

• —Type: Causal Language Model
• —Base Model: Gravity-16B-A3B-Base from Trillion Labs and Lunit Consortium
• —Architecture: GravityMoE (Sparse Mixture-of-Experts with MLA)
• —Total Parameters: 16.24B
• —Active Parameters: 3B
• —Number of Layers: 28
• —Attention Heads: 16
• —KV Heads: 16
• —Hidden Size: 2048
• —MoE Intermediate Size: 1408
• —Routed Experts: 64 (top-8 selection)
• —Shared Experts: 1
• —Context Length: 32,768 tokens
• —Vocabulary Size: 151,552
• —Tokenizer: GLM-4.5
• —Precision: bf16

🚀 Quickstart

SGLang (Recommended)

Install:

pip install "sglang[all] @ git+https://github.com/trillion-labs/sglang-gravity.git#subdirectory=python"

Launch server:

python -m sglang.launch_server \
  --model-path learning-unit/L1-16B-A3B \
  --port 9006 --host 0.0.0.0 \
  --tp 1 --dtype bfloat16 --trust-remote-code \
  --attention-backend triton \
  --moe-runner-backend triton

Query:

curl -X POST http://localhost:9006/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "learning-unit/L1-16B-A3B",
    "messages": [
      {"role": "user", "content": "What are the diagnostic criteria for sepsis?"}
    ],
    "max_tokens": 2048
  }'

Transformers

Install:

pip install "transformers>=5.0" torch

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_name = "learning-unit/L1-16B-A3B"

model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)

messages = [
    {"role": "user", "content": "What are the diagnostic criteria for sepsis?"}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)

generated_ids = model.generate(
    **model_inputs,
    max_new_tokens=2048,
    temperature=0.7,
    do_sample=True,
)
generated_ids = [
    output_ids[len(input_ids):] for input_ids, output_ids in zip(model_inputs.input_ids, generated_ids)
]

response = tokenizer.batch_decode(generated_ids, skip_special_tokens=True)[0]
print(response)

💬 Examples

L1 is specialized for the medical domain and covers a wide range of clinical scenarios. Below are representative examples from real-world clinical use cases.

Medical Q&A

A 45-year-old woman with lupus nephritis on mycophenolate and prednisone develops fever, dry cough, and bilateral ground-glass opacities on chest CT. Her CD4 count is 180. What is your differential diagnosis and recommended workup?

Patient Education

I have diabetes and use insulin daily. What is the proper way to store insulin at home?

Clinical Documentation

Please draft an overnight progress note. Patient labs: RBC 4.5, WBC 8. Vitals: HR 82, BP 118/76, RR 15, Temp 37.1. Nurse reports stable overnight. Plan: continue antibiotics, recheck labs in the morning.

Emergency Triage

다음 응급실 환자에 대해 KTAS triage를 수행하고, 초기 진단 및 감별진단을 제시해주세요. 78세 여성 환자가 119 구급차로 응급실에 내원했습니다. 22시경 갑자기 좌측 안면이 처지고 말이 어눌해지는 증상이 발생했습니다. 두통을 호소하며, 고혈압 병력이 있습니다. 활력징후는 혈압 172/88, 심박수 92, 호흡수 14, 체온 36.8, 산소포화도 98%이고 의식은 명료합니다. 사지 위약감은 없습니다.

Adverse Drug Reaction (ADR) Causality Assessment

다음 환자의 약물이상반응(ADR)에 대해 WHO-UMC 기준으로 인과관계를 평가해주세요. 80세 여성 환자가 기관지확장증으로 입원 중 moxifloxacin 400mg IV를 투여받았습니다. 투여 중 전신 피부 가려움이 새로 발생했고, 약물 중단 후 환자 본인도 가려움이 줄어드는 양상을 표현했으며 이후 회복되었습니다. 재투여는 시행하지 않았습니다. 기존 약물 알레르기력은 없고, 가려움을 유발할 만한 다른 병용약물이나 피부질환은 확인되지 않았습니다.

📊 Benchmark

All benchmarks were evaluated using CoEval, Lunit's open-source medical LLM evaluation framework. Evaluations use greedy decoding (temperature=0). To reproduce these results:

git clone https://github.com/lunit-io/CoEval.git
cd CoEval

Refer to the CoEval Quickstart for setup and evaluation instructions.

MCQA Benchmarks

Chat Task

📝 Citation

@misc{lunit2026l1,
  title={L1: The First Clinical Language Model by Lunit},
  author={Lunit},
  year={2026},
  url={https://huggingface.co/learning-unit/L1-16B-A3B}
}

⚠️ Limitations

• —Not a substitute for professional medical judgment. L1 may generate factually incorrect, incomplete, or outdated clinical information. All outputs should be verified by qualified healthcare professionals.
• —Thinking overhead. Chain-of-thought reasoning in <think> tags increases token consumption and latency compared to non-thinking models of similar size.
• —Context length. Maximum context length is 32,768 tokens.
• —No real-time knowledge. The model's knowledge is limited to its training data cutoff and does not reflect the latest medical guidelines or drug approvals.

🤝 Acknowledgements

This work was supported by the Domain-Specific Foundation Model Project (인공지능 특화 파운데이션 모델 프로젝트), funded by the Ministry of Science and ICT (과학기술정보통신부) and managed by the National IT Industry Promotion Agency (NIPA).

L1 is a collaborative effort by the following consortium members:

Industry

• —Lunit
• —Trillion Labs
• —SK Biopharmaceuticals
• —Kakao Healthcare
• —AIGEN Sciences
• —D-Circle
• —Rebellions
• —Standigm

Academia

• —Prof. Choi Yun-jae's Lab from KAIST
• —Prof. Hong Seung-hoon's Lab from KAIST
• —Prof. Jung Yu-seong's Lab from SNU
• —Prof. Kim Hyun-woo's Lab from KAIST
• —Prof. Kim Tae-gyun's Lab from KAIST
• —Prof. Ye Jong-cheol's Lab from KAIST

Hospitals

• —NHIS Ilsan Hospital
• —Ewha Womans University Seoul Hospital
• —Keimyung University Dongsan Medical Center
• —Konyang University Hospital
• —Korea University Research & Business Foundation
• —Kyung Hee University Hospital at Gangdong
• —Kyung Hee University Medical Center
• —Pusan National University Yangsan Hospital
• —Yongin Severance Hospital

<p align="center"> <img src="consortium.png" alt="Consortium Members" style="width: 80%;"> </p>

📄 License

This model is licensed under the Apache 2.0 License.

📬 Contact

• —Taesoo Kim (김태수) — taesoo.kim@lunit.io
• —Donggeun Yoo (유동근) — dgyoo@lunit.io