MaxinT23/Qwen2.5-7B-Breast-CRAG

Qwen2.5-7B-Breast-CRAG

Breast Cancer Specialized LLM (Full Fine-tuned / LoRA-ready)

Qwen2.5-7B-Breast-CRAG is a 7B-parameter large language model fully fine-tuned for breast cancer clinical consultation, developed as part of the Breast-CRAG system. It is optimized for human-like doctor–patient dialogue and professional breast cancer domain knowledge.

This model can be loaded directly or as a LoRA adapter via LLaMA Factory.

🔗 Links

• —HF Repo: https://huggingface.co/MaxinT23/Qwen2.5-7B-Breast-CRAG
• —Paper: https://link.springer.com/chapter/10.1007/978-3-031-95841-0_19
• —Code: https://github.com/Maxin-C/Breast-CRAG
• —Framework: LLaMA Factory

✨ Model Description

• —Base Model: Qwen2.5-7B-Instruct
• —Task: Breast cancer medical dialogue generation
• —Parameters: 7B
• —Language: Chinese
• —Training: Full fine-tuned + LoRA (PEFT) optimized
• —Purpose: Clinical consultation assistance (research only)
• —System: Core generator of Breast-CRAG (RAG-enhanced)

📊 Training Data

• —Curated breast cancer dialogues: 268K
• —Train split: 91K (30K MedDialog-BC + 61K Huatuo-BC)
• —Data source: MedDialog-CN, Huatuo-26M (filtered/cleaned)
• —Pipeline: Keyword filter → GPT-4o quality filter → dialogue summarization

⚙️ Training Hyperparameters

• —LoRA alpha: 32
• —LoRA rank: 16
• —Dropout: 0.1
• —Learning rate: 5e-5
• —Scheduler: cosine
• —Batch size: 2
• —Gradient accumulation: 8
• —Epochs: 8
• —GPU: RTX 3090 (24GB)
• —Training time: ~34.2 hours

🧪 Key Results

Dialogue (Humanization)

• —Outperforms similar-size open-source LLMs
• —Matches/exceeds GPT-4o on 70% of dialogue metrics

Exam (Specialization)

• —USMLE-BC: 81% accuracy (on par with GPT-4o)
• —Exam-BC Simple: 63% | Hard: 52%

🚀 Usage (LLaMA Factory LoRA Load)

1. Install

pip install "llamafactory[torch]" transformers peft

2. Load via LLaMA Factory

from llamafactory import load_model
from transformers import AutoTokenizer

model_name = "MaxinT23/Qwen2.5-7B-Breast-CRAG"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = load_model(model_name, adapter_name="lora", device="auto")

prompt = "请以乳腺癌专科医生身份，回答以下问题：乳腺癌术后多久可以恢复正常生活？"
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.95, top_p=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

⚠️ Limitations

1. 1.Text-only (no multimodal support)
2. 2.For research purposes only, not for direct clinical use
3. 3.Weak performance on multiple-choice exams
4. 4.Best used with Breast-CRAG Retriever + 1M knowledge chunks

📚 Citation

Chen, Z., Wang, Q., Liu, J., Sun, Y., Zheng, H., Li, H., Duan, H., Lu, X. (2025). Breast-CRAG: A Breast Cancer Large Language Model Leveraging Retrieval-Augmented Generation. In: Artificial Intelligence in Medicine. AIME 2025. Lecture Notes in Computer Science, vol 15735. Springer, Cham. https://doi.org/10.1007/978-3-031-95841-0_19

🙏 Acknowledgments

Supported by the National Key Technologies R&D Program of China (2022YFF1203002) and Sir Run Run Shaw Hospital.