kimani9/qwen25vl-poultry-disease-lora

Qwen2.5-VL-7B-Instruct — Poultry Disease Diagnosis LoRA Adapter

A LoRA fine-tune of Qwen/Qwen2.5-VL-7B-Instruct that answers natural-language questions about common chicken diseases from a photo — diagnosis, symptoms, treatment, prevention, contagiousness, severity, and more — through multi-turn conversation.

⚠️ This is an academic ML/agri-tech project, not veterinary guidance. It is not a substitute for a licensed veterinarian and should not be the sole basis for treatment decisions or notifiable-disease reporting.

Model Details

Model Description

This adapter was fine-tuned to turn Qwen2.5-VL-7B-Instruct into a conversational poultry-health assistant. Given an image of a chicken, it can hold a multi-turn conversation answering questions across 26 intents (diagnosis, symptoms, advanced symptoms, treatment, prevention, cause, pathogen, contagiousness, severity, mortality, vaccination, farmer advice, economic impact, recovery, biosecurity, transmission, egg safety, meat safety, management, housing, nutrition, cleaning, medication, isolation, risk factors, emergency actions).

• —Developed by: Kim (Alfin Thama Kimani), independent ML/agri-tech project
• —Funded by: Self-funded (free-tier Kaggle compute)
• —Shared by: Kim
• —Model type: Vision-language model (image-text-to-text), LoRA adapter for causal LM
• —Language(s) (NLP): English
• —License: Apache 2.0 (adapter weights); usage also subject to the base model's license — see Qwen/Qwen2.5-VL-7B-Instruct
• —Finetuned from model: Qwen/Qwen2.5-VL-7B-Instruct

Model Sources

• —Repository: this repo (adapter weights + processor/tokenizer)
• —Base model: https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct
• —Training dataset source: `allandclive/chicken-disease-1` (Kaggle)

Uses

Direct Use

Ask questions about a chicken photo (diagnosis, symptoms, treatment, prevention, etc.) via the chat-style inference code below. Intended as a research/educational demo of VLM fine-tuning for agricultural use cases, and as a starting point for smallholder-farmer-facing tools pending veterinary review.

Downstream Use

Could be embedded into a WhatsApp bot, mobile app, or web app (e.g. a Gradio Space) as a first-pass triage assistant for farmers — always paired with a clear disclaimer and a path to a real veterinarian for confirmation/treatment.

Out-of-Scope Use

• —Not for diagnosing diseases outside the 4 trained classes (Healthy, Coccidiosis, Salmonellosis, Newcastle Disease) — it may still produce a confident but incorrect answer for anything else.
• —Not for autonomous treatment decisions, medication dosing, or notifiable-disease reporting without human veterinary review.
• —Not evaluated on human/animal subjects outside poultry, or on species other than chickens.

Bias, Risks, and Limitations

• —Templated training data. Conversation answers were generated from a structured knowledge base plus reusable phrasing templates (not independently authored per example), so the model may reproduce fixed phrasing patterns rather than always demonstrating deep visual reasoning about novel images.
• —Small class set. Only 4 disease classes are covered; real-world flocks can have many other conditions.
• —No demographic/regional bias evaluation. Image capture conditions (lighting, breed, background, camera quality typical of the source dataset) may not generalize to all farm settings.
• —Not clinically validated. The underlying disease knowledge base was authored for ML training-data generation, not reviewed by a licensed poultry veterinarian.

Recommendations

Users should treat outputs as a first-pass triage aid only, verify against real veterinary guidance, and involve a licensed veterinarian or livestock extension officer for any suspected notifiable disease (e.g. Newcastle Disease) or urgent flock health issue.

How to Get Started with the Model

This repo contains adapter weights only. Load the base model in 4-bit, then attach this adapter with PEFT:

import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor, BitsAndBytesConfig
from peft import PeftModel
from qwen_vl_utils import process_vision_info
from PIL import Image

BASE_MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"
ADAPTER_REPO_ID = "kimani9/qwen25vl-poultry-disease-lora"  

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16,
)

base_model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    BASE_MODEL_ID,
    quantization_config=bnb_config,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)

model = PeftModel.from_pretrained(base_model, ADAPTER_REPO_ID)
model.eval()

processor = AutoProcessor.from_pretrained(ADAPTER_REPO_ID, trust_remote_code=True)

SYSTEM_PROMPT = (
    "You are an AI poultry health assistant that helps smallholder farmers "
    "identify common chicken diseases from photos and gives practical, "
    "safe guidance. You are not a replacement for a licensed veterinarian "
    "for treatment decisions or notifiable disease reporting."
)

def ask(image_path: str, question: str, max_new_tokens: int = 200) -> str:
    image = Image.open(image_path).convert("RGB")
    messages = [
        {"role": "system", "content": [{"type": "text", "text": SYSTEM_PROMPT}]},
        {"role": "user", "content": [
            {"type": "image", "image": image},
            {"type": "text", "text": question},
        ]},
    ]
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, _ = process_vision_info(messages)
    inputs = processor(text=[text], images=image_inputs, padding=True, return_tensors="pt")
    inputs = {k: v.to(model.device) for k, v in inputs.items()}

    with torch.no_grad():
        generated_ids = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)

    trimmed = generated_ids[:, inputs["input_ids"].shape[1]:]
    return processor.batch_decode(trimmed, skip_special_tokens=True)[0].strip()

print(ask("chicken_photo.jpg", "What disease does this chicken have?"))

Requirements:

transformers==4.49.0
accelerate>=0.34.0
peft>=0.13.0
bitsandbytes>=0.43.0
qwen-vl-utils
torch
pillow

Training Details

Training Data

Derived from `allandclive/chicken-disease-1` (Kaggle). Raw labels (healthy, cocci, salmo, ncd, and their PCR-confirmed variants) were cleaned (corrupted/duplicate images removed), remapped into 4 disease classes with a diagnosis_type of Visual or PCR, enriched with a structured disease knowledge base (pathogen, symptoms, treatment, prevention, economic impact, etc.), and expanded into multi-turn (2–5 turn) conversations covering 26 question intents. Split 80% train / 10% validation / 10% test, stratified by disease label.

Training Procedure

Preprocessing

Images resized so the longer side does not exceed 896px. Conversations rendered via the Qwen2.5-VL chat template; labels masked so loss is computed only on assistant-turn tokens.

Training Hyperparameters

• —Training regime: fp16 mixed precision (chosen over bf16 for T4/P100 GPU compatibility — these architectures lack native bf16 tensor-core acceleration)
• —Epochs: 2, with early stopping on validation loss (patience = 3 eval rounds)
• —Batching: per-device batch size 1, gradient accumulation steps 16
• —Optimizer: pagedadamw8bit
• —LR schedule: cosine, warmup ratio 0.03, peak LR 2e-4
• —LoRA config: r=16, alpha=32, dropout=0.05, target modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
• —Best checkpoint selection: lowest validation loss (load_best_model_at_end=True)

Speeds, Sizes, Times

• —Hardware: Kaggle, 2x NVIDIA T4 GPUs (16GB each)
• —Base model loading: 4-bit NF4 quantization with double quantization
• —Adapter weights only (LoRA), so checkpoint size is a small fraction of the 7B base model

Evaluation

Testing Data, Factors & Metrics

Testing Data

Held-out 10% test split from the same source dataset (stratified by disease label), not seen during training.

Factors

Disaggregated informally by disease class (Healthy, Coccidiosis, Salmonellosis, Newcastle Disease) and by diagnosis type (Visual vs. PCR-confirmed).

Metrics

Validation loss (cross-entropy on assistant-turn tokens) tracked during training; qualitative ground-truth-vs-prediction review on sampled validation/test examples.

Results

Validation loss decreased rapidly and stabilized at a low value within the first training epoch. Qualitative samples were reviewed manually (see the training notebook's Section 14B) rather than reported as a single benchmark number — quantitative benchmarking against a held-out clinician-reviewed set is a recommended next step before any real-world use.

Summary

The adapter reliably reproduces the disease knowledge base's phrasing for the 4 trained classes on in-distribution images. Generalization to genuinely novel field photos has not yet been rigorously benchmarked.

Environmental Impact

• —Hardware Type: 2x NVIDIA T4 (16GB)
• —Hours used: Several hours (single training run on Kaggle's free-tier weekly GPU quota)
• —Cloud Provider: Kaggle (Google Cloud-hosted)
• —Compute Region: Not specified by provider
• —Carbon Emitted: Not calculated — estimate via the ML Impact calculator

Technical Specifications

Model Architecture and Objective

Qwen2.5-VL-7B-Instruct (vision-language transformer) with LoRA adapters applied to the language-model decoder's attention and MLP projections. Trained with a causal language modeling objective (next-token prediction) restricted to assistant-turn tokens in a multi-turn conversational format.

Compute Infrastructure

Hardware

Kaggle Notebooks, T4 x2 accelerator (2x 16GB GPUs)

Software

• —Python 3.12
• —transformers 4.49.0
• —PEFT 0.20.0
• —TRL (SFTTrainer)
• —bitsandbytes (4-bit NF4 quantization)
• —qwen-vl-utils

Citation

BibTeX:

@misc{qwen25vl-poultry-disease-lora,
  title  = {Qwen2.5-VL-7B-Instruct Poultry Disease Diagnosis LoRA Adapter},
  author = {Kimani, Alfin Thama},
  year   = {2026},
  note   = {Fine-tuned on data derived from the allandclive/chicken-disease-1 Kaggle dataset}
}

APA: Kimani, A. T. (2026). Qwen2.5-VL-7B-Instruct Poultry Disease Diagnosis LoRA Adapter [Model]. Hugging Face.

Glossary

• —LoRA (Low-Rank Adaptation): a parameter-efficient fine-tuning method that trains small low-rank weight updates instead of the full model.
• —QLoRA: LoRA applied on top of a 4-bit quantized base model, reducing memory needed for fine-tuning large models.
• —PCR-confirmed: a diagnosis type in the source dataset confirmed via polymerase chain reaction lab testing, as opposed to purely visual/photographic assessment.

More Information

See the accompanying Kaggle training notebook for the full dataset construction, training, and evaluation pipeline.

Model Card Authors

Kim (Alfin Thama Kimani)

Model Card Contact

Reach out via the Hugging Face repo's Community tab.

Framework versions

• —PEFT 0.20.0