Modelpublic

ulvxa/DeepSeek-R1-Distill-Qwen-7B-Lora-Thesis

sourceHugging Facemitupdated 4mo agoView on Hugging Face

0likes6downloads

Model Card

DeepSeek-R1-7B Wellness Assistant — QLoRA Adapter

A QLoRA fine-tuned LoRA adapter for `deepseek-ai/DeepSeek-R1-Distill-Qwen-7B`, trained as part of a Bachelor's thesis on generative AI-based wellness assistance.

This adapter specialises the base model across three wellness domains:

—Pharmaceutical information — drug composition, indications, side effects, dosage, manufacturer
—Emotional support — empathetic responses, emotional validation, therapeutic dialogue
—Nutritional information — calorie and macronutrient queries based on the USDA nutrient database

Model Details

Field	Value
Developed by	Ulvi Aliyev
Model type	Causal LM — LoRA adapter (PEFT)
Base model	deepseek-ai/DeepSeek-R1-Distill-Qwen-7B
Language	English
License	MIT
Thesis	Evaluation and Implementation of a Generative AI-Based Wellness Assistant

Training Details

Parameter	Value
Method	QLoRA (4-bit NF4 quantization)
LoRA rank (r)	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	qproj, kproj, vproj, oproj, gate\proj, up\proj, down\_proj
Training format	Alpaca instruction-tuning
Epochs	3
Effective batch size	32
Learning rate	2e-4
Max sequence length	16,384 tokens
Hardware	NVIDIA A100-SXM4-40GB (Google Colab)
Training time	~14 hours
Total training samples	195,229 instruction pairs

Training Dataset

Domain	Source	Samples
Pharmaceutical	Indian drug database	59,125
Therapy	GPT-based therapeutic conversations	60,839
Emotion	Labeled emotion classification dataset	60,000
Nutrition	USDA National Nutrient Database	36,958
Total		216,922 (after deduplication: 195,229)

All samples were converted to the Alpaca instruction-tuning format (instruction, input, output) and shuffled before training.

Usage

Install dependencies:

bash

pip install transformers peft bitsandbytes accelerate

Load and run inference:

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

BASE_MODEL = "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B"
ADAPTER    = "ulvxa/deepseek-r1-7b-wellness-assistant"

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)

tokenizer = AutoTokenizer.from_pretrained(ADAPTER)
base      = AutoModelForCausalLM.from_pretrained(
    BASE_MODEL, quantization_config=bnb_config, device_map="auto"
)
model     = PeftModel.from_pretrained(base, ADAPTER)
model.eval()


def ask(instruction: str, input_text: str = "") -> str:
    if input_text.strip():
        prompt = (
            "Below is an instruction that describes a task, paired with an input "
            "that provides further context. Write a response that appropriately "
            f"completes the request.\n\n### Instruction:\n{instruction}\n\n"
            f"### Input:\n{input_text}\n\n### Response:\n"
        )
    else:
        prompt = (
            "Below is an instruction that describes a task. Write a response that "
            f"appropriately completes the request.\n\n### Instruction:\n{instruction}"
            "\n\n### Response:\n"
        )
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(
            **inputs,
            max_new_tokens=256,
            do_sample=False,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.eos_token_id,
        )
    return tokenizer.decode(
        out[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True
    ).strip()


# Pharmaceutical
print(ask("What is Prulastin-M Tablet and what is it used for?"))

# Emotional support
print(ask("Respond empathetically to this message.", "i feel so alone lately"))

# Nutrition
print(ask("How many calories are in 100g of cooked brown rice?"))

Tip — concise therapy responses

Therapy outputs can be verbose. Limit to 128 tokens for tighter replies:

python

out = model.generate(**inputs, max_new_tokens=128, ...)

Limitations

—Not a substitute for professional advice. Medical and pharmaceutical information comes from training data and must not replace a licensed physician or pharmacist.
—Therapy verbosity. Emotional support responses tend to be long; cap max_new_tokens if brevity matters.
—Multilingual noise. Occasional Spanish or Chinese fragments may appear in emotional support responses due to multilingual noise in the therapy training split.
—Nutritional approximation. Primary calorie figures are accurate; full macronutrient profiles may drift slightly from USDA ground truth for edge cases.
—English only. The model was trained on English instruction pairs and performs best in English.
—Hallucination risk. Like all instruction-tuned LLMs, the model can generate plausible-sounding but incorrect information. Always verify medical and pharmaceutical output against authoritative sources.

Citation

bibtex

@misc{aliyev2026wellness,
  author = {Ulvi Aliyev},
  title  = {Evaluation and Implementation of a Generative AI-Based Wellness Assistant},
  year   = {2026},
  note   = {Bachelor's Thesis}
}