VLAI-AIVN/Llama-3.2-1B-Instruct-vi-medqa-lora

Model Card for VLAI-AIVN/Llama-3.2-1B-Instruct-vi-medqa-lora

LoRA adapter for Vietnamese medical question answering, fine-tuned from Llama 3.2 1B Instruct 4-bit.

Model Details

Model Description

This model is a PEFT/LoRA adapter trained for Vietnamese medical QA style responses with cautious, informative phrasing. It was trained using Unsloth + TRL SFT workflow and pushed to the Hugging Face Hub.

• —Developed by: ThuanNaN / project contributors
• —Funded by [optional]: [More Information Needed]
• —Shared by [optional]: VLAI-AIVN
• —Model type: Causal language model adapter (LoRA) for instruction-following generation
• —Language(s) (NLP): Vietnamese
• —License: [More Information Needed]
• —Finetuned from model [optional]: unsloth/Llama-3.2-1B-Instruct-bnb-4bit

Model Sources [optional]

• —Repository: https://github.com/ThuanNaN/aio-llmops
• —Paper [optional]: [More Information Needed]
• —Demo [optional]: [More Information Needed]

Uses

Direct Use

• —Vietnamese medical Q&A assistance in chat-style applications
• —Educational and informational responses about common health questions

Downstream Use [optional]

• —Can be mounted as an adapter in vLLM/Transformers serving stacks
• —Can be integrated into healthcare information assistants with human oversight

Out-of-Scope Use

• —Clinical diagnosis or treatment decisions without licensed professionals
• —Emergency triage as a sole decision-making system
• —Any high-stakes medical setting without human review

Bias, Risks, and Limitations

• —The model may produce inaccurate or incomplete medical information.
• —Training data quality and coverage may introduce medical domain and demographic bias.
• —The model can hallucinate facts, contraindications, or unsafe guidance.
• —This is a small 1B-base adapter and may underperform on complex medical reasoning.

Recommendations

• —Always include disclaimers that responses are informational only.
• —Require clinician or domain-expert review before real-world use.
• —Add safety filters and escalation logic for severe-symptom queries.
• —Encourage users to consult licensed healthcare providers.

How to Get Started with the Model

Use the code below to get started with the model.

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base_model_id = "unsloth/Llama-3.2-1B-Instruct-bnb-4bit"
adapter_id = "VLAI-AIVN/Llama-3.2-1B-Instruct-vi-medqa-lora"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForCausalLM.from_pretrained(base_model_id, torch_dtype=torch.bfloat16)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()

messages = [
    {
        "role": "system",
        "content": "Ban la tro ly AI ho tro thong tin y khoa bang tieng Viet. Tra loi ro rang, than trong, khong chan doan qua muc, va khuyen nguoi dung gap bac si khi co dau hieu nguy hiem.",
    },
    {"role": "user", "content": "Toi bi ho va sot nhe 2 ngay, toi nen lam gi?"},
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=256,
        do_sample=True,
        temperature=0.2,
        top_p=0.9,
        repetition_penalty=1.1,
    )

generated = output[0][inputs["input_ids"].shape[1]:]
print(tokenizer.decode(generated, skip_special_tokens=True))

Training Details

Training Data

• —Dataset: hungnm/vietnamese-medical-qa
• —Split strategy: train split was further split into train/validation with test_size=0.05 and seed=42
• —Fields used: question, answer

Training Procedure

Training used supervised fine-tuning (SFT) with chat-formatted prompts:

• —system: medical safety-oriented Vietnamese instruction
• —user: question
• —assistant: reference answer

Preprocessing [optional]

• —Converted each sample into chat conversation text via tokenizer chat template
• —Tokenized with truncation and max sequence length of 2048

Training Hyperparameters

• —Training regime: bf16 mixed precision when supported, otherwise fp16 mixed precision
• —Max sequence length: 2048
• —Epochs: 5
• —Learning rate: 2e-4
• —Weight decay: 0.01
• —Warmup steps: 200
• —LR scheduler: cosine
• —Per-device train batch size: 8
• —Per-device eval batch size: 8
• —Gradient accumulation steps: 2
• —Optimizer: pagedadamw8bit
• —Evaluation strategy: steps (every 100)
• —Checkpoint save strategy: steps (every 100), keep last 2
• —Early stopping: patience=2, threshold=0.0
• —LoRA rank: 16
• —LoRA alpha: 16
• —LoRA dropout: 0
• —Seed: 42

Speeds, Sizes, Times [optional]

[More Information Needed]

Evaluation

Testing Data, Factors & Metrics

Testing Data

• —Validation split from hungnm/vietnamese-medical-qa (5% holdout)

Factors

• —General Vietnamese medical Q&A prompts
• —Generative quality and safety tone

Metrics

• —eval_loss during validation
• —Qualitative generation inspection on held-out examples

Results

• —Training tracked evalloss and saved best model at end based on lowest evalloss.
• —Additional manual spot-check generation was performed in notebook inference cells.

Summary

The adapter learns domain style and Vietnamese response format for medical QA, but outputs still require expert verification and safety controls.

Model Examination [optional]

[More Information Needed]

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

• —Hardware Type: [More Information Needed]
• —Hours used: [More Information Needed]
• —Cloud Provider: [More Information Needed]
• —Compute Region: [More Information Needed]
• —Carbon Emitted: [More Information Needed]

Technical Specifications [optional]

Model Architecture and Objective

• —Base architecture: Llama 3.2 1B Instruct (4-bit quantized base checkpoint)
• —Adaptation method: LoRA on attention and MLP projection modules
• —Objective: next-token prediction under supervised instruction-following format

Compute Infrastructure

[More Information Needed]

Hardware

• —CUDA GPU expected for training (bf16 if supported)

Software

• —PyTorch 2.10.0+cu130
• —Unsloth
• —TRL
• —Transformers
• —Datasets
• —PEFT

Citation [optional]

BibTeX:

@misc{aio_llmops_medqa_lora_2026,
  title={Llama-3.2-1B-Instruct-vi-medqa-lora},
  author={ThuanNaN and contributors},
  year={2026},
  howpublished={\url{https://huggingface.co/VLAI-AIVN/Llama-3.2-1B-Instruct-vi-medqa-lora}}
}

APA:

ThuanNaN, & contributors. (2026). Llama-3.2-1B-Instruct-vi-medqa-lora. Hugging Face. https://huggingface.co/VLAI-AIVN/Llama-3.2-1B-Instruct-vi-medqa-lora

Glossary [optional]

• —LoRA: Low-Rank Adaptation for parameter-efficient fine-tuning
• —SFT: Supervised Fine-Tuning
• —PEFT: Parameter-Efficient Fine-Tuning

More Information [optional]

The training workflow is documented in notebooks/med_qa.ipynb within the aio-llmops repository.

Model Card Authors [optional]

ThuanNaN / aio-llmops contributors

Model Card Contact

[More Information Needed]