breakdown881/vilaw-llm-dpo

🏛️ VILaw-LLM (DPO Aligned) — Vietnamese Legal Reasoning LLM

VILaw-LLM-DPO is an advanced domain-specific Large Language Model fine-tuned for Vietnamese Legal Advisory, Statutory Citation, and Multi-hop Legal Reasoning.

The model is adapted from `Qwen/Qwen2.5-7B-Instruct` through a two-stage training regime:

1. 1.Supervised Fine-Tuning (SFT): 4-bit QLoRA adaptation via Unsloth on 29.4K curated legal instruction pairs with strict Unicode NFC normalization.
2. 2.Direct Preference Optimization (DPO): Preference alignment using Hugging Face TRL on synthetic preference pairs (Chosen vs. Rejected) to eliminate hallucination, suppress speculative phrasing, and enforce exact statutory citations (Điều, Khoản, Luật).

📊 Benchmark Evaluation (LLM-as-Judge)

Evaluated against ground truth statutory provisions across Civil, Commercial, Corporate, Labor, and Bidding laws:

🚀 How to Use

1. Using Hugging Face Transformers & PEFT

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "Qwen/Qwen2.5-7B-Instruct"
adapter_model_id = "your-username/vilaw-dpo-lora"  # Or path to local folder

# 1. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(base_model_id)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.float16,
    device_map="auto"
)

# 2. Attach DPO LoRA Adapter
model = PeftModel.from_pretrained(base_model, adapter_model_id)
model.eval()

# 3. Format with ChatML Template
SYSTEM_PROMPT = (
    "Bạn là một chuyên gia tư vấn pháp luật Việt Nam am hiểu sâu sắc các quy định pháp luật. "
    "Hãy trả lời câu hỏi dựa trên các văn bản quy phạm pháp luật hiện hành, "
    "viện dẫn chính xác số Điều, Khoản, tên luật và đưa ra lập luận logic, rõ ràng."
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": "Thời hiệu khởi kiện yêu cầu giải quyết tranh chấp hợp đồng dân sự là bao lâu?"}
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([prompt], return_tensors="pt").to(model.device)

# 4. Generate Response
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_new_tokens=512,
        temperature=0.3,
        repetition_penalty=1.15
    )

response = tokenizer.decode(outputs[0][len(inputs.input_ids[0]):], skip_special_tokens=True)
print(response)

2. Using Unsloth (2x Faster Inference)

from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="your-username/vilaw-dpo-lora",
    max_seq_length=2048,
    load_in_4bit=True
)
FastLanguageModel.for_inference(model)

# Standard inference workflow...

🛠️ Training Specifications

Supervised Fine-Tuning (SFT)

• —Base Architecture: Qwen2.5-7B-Instruct
• —Method: 4-bit QLoRA (NF4 quantization)
• —Rank ($r$): 16 | Alpha ($\alpha$): 32 | Dropout: 0
• —Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• —Effective Batch Size: 16 (per-device 2 $\times$ gradient accumulation 8)
• —Optimizer: AdamW 8-bit | Learning Rate: $2 \times 10^{-4}$ (Cosine schedule)
• —Context Length: 2,048 tokens
• —Training Data: 26,483 train / 2,943 val curated statutory QA pairs

Direct Preference Optimization (DPO)

• —Framework: Hugging Face TRL DPOTrainer with Unsloth PatchDPOTrainer
• —KL Penalty ($\beta$): 0.1
• —Learning Rate: $5 \times 10^{-6}$ (Extremely conservative to preserve SFT representation)
• —Reference Model: Frozen base model (ref_model=None parameter-efficient strategy)
• —Loss Type: Sigmoid DPO Loss
• —Preference Dataset: Synthetic preference pairs covering legal citations vs. speculative answers

🛡️ Intended Use & Safety Guardrails

Intended Use

• —Research in legal NLP and Vietnamese statutory reasoning.
• —Drafting preliminary legal summaries for certified lawyers and legal assistants.
• —Automated legal question-answering when paired with a human-in-the-loop review workflow.

Limitations & Out-of-Scope Use

• —Not Legal Counsel: This model is an artificial intelligence research artifact. Its outputs do not constitute binding legal advice, formal representation, or legal opinions.
• —Statutory Updates: Laws and subordinate decrees (Nghị định, Thông tư) undergo frequent revisions. Users must cross-check references against the official national gazette (Cơ sở dữ liệu Quốc gia về Văn bản Pháp luật - vbpl.vn).
• —Malicious Queries: The model is not intended for circumventing statutes, generating fraudulent documentation, or aiding tax evasion.

📜 Citation

If you utilize VILaw-LLM in your research or application, please cite:

@misc{vilaw_llm_2025,
  title={VILaw-LLM: Vietnamese Legal Reasoning Large Language Model via QLoRA SFT and DPO Alignment},
  author={breakdown881},
  year={2025},
  publisher={Hugging Face},
  howpublished={\url{https://huggingface.co/breakdown881/vilaw-llm-dpo}}
}

📄 License

This adapter model is distributed under the Apache 2.0 License, aligning with the base model weights of the Qwen series.