Ankitdajhgf565/vit-large-skin-cancer-ham10000

ViT-Large — Skin Cancer Classification (HAM10000)

Fine-tuned google/vit-large-patch16-224 on the HAM10000 dermoscopy dataset to classify 7 types of skin lesions. Trained on a free Google Colab T4 GPU in 37 minutes using HuggingFace Trainer with FP16 mixed precision.

This was a personal project to learn medical image fine-tuning — not a clinical tool.

How to use

from transformers import pipeline

classifier = pipeline(
    "image-classification",
    model="Kuldeepmishra3/vit-large-skin-cancer-ham10000",
)

results = classifier("skin_image.jpg", top_k=3)
for r in results:
    print(f"{r['label']}: {r['score']:.2%}")

Or with manual preprocessing:

from transformers import ViTForImageClassification, ViTImageProcessor
from PIL import Image
import torch

model = ViTForImageClassification.from_pretrained("Kuldeepmishra3/vit-large-skin-cancer-ham10000")
processor = ViTImageProcessor.from_pretrained("Kuldeepmishra3/vit-large-skin-cancer-ham10000")

image = Image.open("skin_image.jpg").convert("RGB")
inputs = processor(images=image, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class = logits.argmax(-1).item()
print(model.config.id2label[predicted_class])

Classes

Results

Evaluated on the HAM10000 validation split (2,492 images):

Per-class F1:

Melanoma was the hardest — it frequently gets confused with Melanocytic Nevi, which is a known challenge in dermoscopy even for trained dermatologists.

Training details

• —Base model: google/vit-large-patch16-224 (303M params)
• —Dataset: HAM10000 via marmal88/skin_cancer on HuggingFace
• —Epochs: 5 (best checkpoint at epoch 4)
• —Batch size: 16
• —Learning rate: 2e-5 with cosine schedule
• —Warmup steps: 300
• —Precision: FP16
• —GPU: Tesla T4 (Google Colab free tier)
• —Training time: ~37 minutes

Augmentations used during training: random horizontal/vertical flip, rotation up to 30°, color jitter. No augmentation at validation time.

Limitations

• —Not a clinical diagnostic tool — do not use for real medical decisions
• —Performance drops on minority classes (akiec, df, vasc) due to class imbalance in HAM10000
• —Only trained on dermoscopy images — won't work well on regular phone camera skin photos
• —Dataset is mostly from European populations which may affect generalization

Dataset

HAM10000 (Human Against Machine with 10000 training images) is a large collection of multi-source dermatoscopic images of common pigmented skin lesions. The dataset has significant class imbalance — Melanocytic Nevi makes up around 67% of all samples.