weileluc/vit-base-oxford-iiit-pets

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

vit-base-oxford-iiit-pets

This model is a fine-tuned version of google/vit-base-patch16-224 on the pcuenq/oxford-pets dataset. It achieves the following results on the evaluation set:

• —Loss: 0.2027
• —Accuracy: 0.9432

Model description

This model is a fine-tuned version of Google's ViT base model (vit-base-patch16-224) adapted for image classification on the Oxford-IIIT Pet dataset. It distinguishes between 37 cat and dog breeds using transfer learning and achieves strong performance with minimal training effort.

The model was trained using the Hugging Face Trainer API and can be used for pet image classification tasks or as a base for further fine-tuning.

Intended uses & limitations

Intended uses:

• —Classifying pet images (dogs and cats) by breed
• —Educational purposes for transfer learning and computer vision
• —Comparisons with zero-shot models such as CLIP

Limitations:

• —The model is only trained on 37 specific breeds from the Oxford-IIIT Pet dataset
• —May perform poorly on images outside this dataset (e.g. unusual angles, bad lighting, non-pets)

Training and evaluation data

The model was fine-tuned on the Oxford-IIIT Pet dataset, which contains 7,349 images of 37 different dog and cat breeds.

The dataset was split into:

• —80% for training
• —10% for validation
• —10% for testing

The evaluation results are reported on the validation set.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 0.0003
• —trainbatchsize: 16
• —evalbatchsize: 8
• —seed: 42
• —optimizer: Use OptimizerNames.ADAMWTORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizerargs=No additional optimizer arguments
• —lrschedulertype: linear
• —num_epochs: 5

Training results

Framework versions

• —Transformers 4.50.3
• —Pytorch 2.6.0+cpu
• —Datasets 3.5.0
• —Tokenizers 0.21.1

🔍 Zero-Shot Classification Comparison

This model (vit-base-oxford-iiit-pets) was compared to a zero-shot model using CLIP (openai/clip-vit-large-patch14) on the Oxford-IIIT Pet dataset.

Zero-Shot CLIP Evaluation Results:

• —Accuracy: 0.8800 %
• —Precision: 0.8768 %
• —Recall: 0.8800 %

The fine-tuned ViT model achieved:

• —Accuracy: 94.32 %

This shows that transfer learning using ViT outperforms CLIP on this dataset.

🧪 Live Demo

👉 Try it live: Gradio App on Hugging Face Spaces