E27085921/HIKARI-Deneb-8B-SkinDx-Cascade

<p align="center"> <img src="HIKARI_logo.png" alt="HIKARI — Healthcare-oriented Intelligent Knowledge Augmented Retrieval and Inference" width="100%"/> </p>

<h1 align="center">HIKARI-Deneb-8B-SkinDx-Cascade</h1>

<p align="center"> <b>Healthcare-oriented Intelligent Knowledge Augmented Retrieval and Inference</b><br/> Named after <b>Deneb</b> — tail star of Cygnus the Swan, tracing the cascade from group to disease </p>

<p align="center"> <img src="https://img.shields.io/badge/Base%20Model-Qwen3--VL--8B--Thinking-blue?style=flat-square"/> <img src="https://img.shields.io/badge/Task-Skin%20Disease%20Diagnosis-red?style=flat-square"/> <img src="https://img.shields.io/badge/Accuracy-79.80%25-green?style=flat-square"/> <img src="https://img.shields.io/badge/Method-2--Stage%20Cascade%20FT-purple?style=flat-square"/> <img src="https://img.shields.io/badge/License-Apache%202.0-orange?style=flat-square"/> </p>

📦 Model Type: Merged Full Model

This is a fully merged model — the LoRA adapter weights have been merged directly into the base model weights. ✅ No adapter loading needed. Load and run directly with transformers, vLLM, or SGLang. 💾 Size: ~17 GB (4 safetensor shards) 🔌 Lightweight adapter version: [E27085921/HIKARI-Deneb-8B-SkinDx-Cascade-LoRA](https://huggingface.co/E27085921/HIKARI-Deneb-8B-SkinDx-Cascade-LoRA) (~1.1 GB)

Overview

HIKARI-Deneb uses 2-Stage Cascade Fine-Tuning for 10-class skin disease diagnosis. Instead of starting from the raw base model, Stage 2 disease training begins from the weights of HIKARI-Subaru-8B-SkinGroup (the 4-class group classifier). The model inherits group-level visual discrimination before learning disease-level distinctions.

Why Cascade?

Starting from the group classifier teaches the model to compare visual features across skin disease groups before learning fine-grained disease distinctions — making it significantly more effective when used with RAG reference images.

🔧 Quick Inference — transformers

from transformers import Qwen3VLForConditionalGeneration, AutoProcessor
import torch
from PIL import Image

model_id = "E27085921/HIKARI-Deneb-8B-SkinDx-Cascade"

processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = Qwen3VLForConditionalGeneration.from_pretrained(
    model_id, torch_dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)

image = Image.open("skin_lesion.jpg").convert("RGB")
group = "inflammatory"  # from Stage 1 (HIKARI-Subaru)

PROMPT = (
    "This skin lesion belongs to the group '{group}'. "
    "Examine the lesion morphology (papules, plaques, macules), "
    "color (red, violet, white, brown), scale/crust, border sharpness, "
    "and distribution pattern. Based on these visual features, "
    "what is the specific skin disease?"
)

messages = [{"role": "user", "content": [
    {"type": "image", "image": image},
    {"type": "text", "text": PROMPT.format(group=group)},
]}]
text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = processor(text=[text], images=[image], return_tensors="pt").to(model.device)

with torch.no_grad():
    out = model.generate(**inputs, max_new_tokens=64, temperature=0.0, do_sample=False)

print(processor.batch_decode(out[:, inputs["input_ids"].shape[1]:], skip_special_tokens=True)[0].strip())

🔌 LoRA Adapter Version

from peft import PeftModel
from transformers import Qwen3VLForConditionalGeneration
import torch

base = Qwen3VLForConditionalGeneration.from_pretrained(
    "Qwen/Qwen3-VL-8B-Thinking", torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(base, "E27085921/HIKARI-Deneb-8B-SkinDx-Cascade-LoRA")

→ [E27085921/HIKARI-Deneb-8B-SkinDx-Cascade-LoRA](https://huggingface.co/E27085921/HIKARI-Deneb-8B-SkinDx-Cascade-LoRA)

📄 Citation

@misc{hikari2026,
  title  = {HIKARI: RAG-in-Training for Skin Disease Diagnosis
            with Cascaded Vision-Language Models},
  author = {Watin Promfiy and Pawitra Boonprasart},
  year   = {2026},
  institution = {King Mongkut's Institute of Technology Ladkrabang,
                 Department of Information Technology, Bangkok, Thailand}
}

<p align="center">Made with ❤️ at <b>King Mongkut's Institute of Technology Ladkrabang (KMITL)</b></p>