dataslab/DLM-2.0-14B-FP8

DLM-2.0-14B-FP8

Overview

This is an FP8-quantized version of dnotitia/DNA-2.0-14B, optimized for efficient inference by DLM (Data Science Lab., Ltd.).

FP8 (8-bit floating point) quantization with static per-tensor scaling reduces model size by approximately 35% while maintaining near-original accuracy. Fully compatible with vLLM for high-throughput production serving.

Model Details

Quantization Details

• —Method: Static FP8 quantization via llm-compressor oneshot
• —Precision: FP8E4M3 for weights, FP8E4M3 for input activations
• —Strategy: Per-tensor symmetric scaling with MinMax observer
• —Calibration: 512 samples from HuggingFaceH4/ultrachat_200k (train_sft split), max sequence length 2048
• —Format: compressed-tensors (safetensors)
• —Preserved layers: lm_head kept in full precision (BF16)
• —Targets: All Linear layers (except lm_head)

Usage

vLLM (Recommended)

vllm serve dataslab/DLM-2.0-14B-FP8 \
  --dtype auto \
  --max-model-len 32768 \
  --enable-reasoning \
  --reasoning-parser deepseek_r1

Extended context (up to 131K with YaRN):

vllm serve dataslab/DLM-2.0-14B-FP8 \
  --dtype auto \
  --rope-scaling '{"rope_type":"yarn","factor":4.0,"original_max_position_embeddings":32768}' \
  --max-model-len 131072 \
  --enable-reasoning \
  --reasoning-parser deepseek_r1

Python (vLLM)

from vllm import LLM, SamplingParams

llm = LLM(model="dataslab/DLM-2.0-14B-FP8")
sampling_params = SamplingParams(
    temperature=0.6, top_p=0.95, top_k=20, max_tokens=4096
)

messages = [
    {"role": "user", "content": "한국의 경제 발전 과정에 대해 설명해주세요."}
]
outputs = llm.chat(messages, sampling_params=sampling_params)
print(outputs[0].outputs[0].text)

Transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("dataslab/DLM-2.0-14B-FP8")
model = AutoModelForCausalLM.from_pretrained(
    "dataslab/DLM-2.0-14B-FP8",
    device_map="auto",
)

messages = [
    {"role": "user", "content": "복잡한 윤리적 딜레마에 대해 다각도로 분석해줘."}
]
inputs = tokenizer.apply_chat_template(
    messages, add_generation_prompt=True, return_dict=True, return_tensors="pt"
).to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=4096,
    temperature=0.6,
    top_p=0.95,
    top_k=20,
    do_sample=True,
)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Dynamic Thinking Mode

This model inherits DNA 2.0's dynamic thinking capability:

• —Thinking mode: Add /think to enable detailed step-by-step reasoning (temperature=0.6)
• —Non-thinking mode: Add /no_think for concise, direct responses (temperature=0.7)

Base Model

DNA 2.0 is developed by Dnotitia Inc. and features:

• —Smoothie Qwen3 foundation with balanced multilingual optimization
• —Uncensored reasoning training for objective, unbiased responses
• —Advanced RL post-training for enhanced mathematical reasoning and Korean language capabilities

For more details, see the arXiv paper (2507.05686).

License

Apache 2.0 — Same as the base model.

Quantized and released by [DLM (Data Science Lab., Ltd.)](http://www.dataslab.co.kr) — [HuggingFace](https://huggingface.co/dataslab)