Bektur756/vit-pizza-burger-sushi

ViT Pizza, Burger and Sushi Classifier

Vision Transformer fine-tuned to classify food images into three categories:

• —Pizza
• —Burger
• —Sushi

Base model

google/vit-base-patch16-224

Dataset

A balanced three-class subset of ethz/food101.

• —Train: 1912 images
• —Validation: 338 images
• —Test: 750 images

The Food-101 hamburger class is exposed as burger.

Architecture

• —Input tensor: (Batch Size, 3, 224, 224)
• —Patch size: 16 × 16
• —Number of image patches: 196
• —Hidden size: 768
• —Classification outputs: 3
• —Fine-tuned Transformer blocks: last 2
• —Trainable parameters: 14,179,587
• —Total parameters: 85,800,963

The first CLS token representation is passed to the final three-class classifier.

Augmentation

Training augmentation includes:

• —Random resized crop
• —Horizontal flip
• —Rotation
• —Affine transformations
• —Perspective transformations
• —Brightness, contrast, saturation and hue changes
• —Random erasing

Validation, test and inference use deterministic resize, center crop and normalization.

Test results

• —Accuracy: 0.9987
• —Macro F1: 0.9987

Latency

Latency includes image preprocessing and model inference with batch size 1.

• —Tesla T4 mean: 17.44 ms
• —Tesla T4 p95: 22.04 ms
• —Tesla T4 throughput: 57.33 images/second
• —CPU mean: 884.89 ms
• —CPU p95: 1907.68 ms
• —CPU throughput: 1.13 images/second

Usage

from PIL import Image
import torch

from transformers import (
    AutoImageProcessor,
    AutoModelForImageClassification,
)

repository_id = "Bektur756/vit-pizza-burger-sushi"

processor = AutoImageProcessor.from_pretrained(
    repository_id
)

model = AutoModelForImageClassification.from_pretrained(
    repository_id
)

image = Image.open("food.jpg").convert("RGB")

inputs = processor(
    images=image,
    return_tensors="pt",
)

with torch.inference_mode():
    logits = model(**inputs).logits

probabilities = torch.softmax(
    logits,
    dim=-1,
)[0]

label_id = int(
    probabilities.argmax()
)

print(
    model.config.id2label[label_id],
    float(probabilities[label_id]),
)

## Experiment artifacts

This repository includes:

- Confusion matrices
- Training and validation graphs
- F1 score history
- Classification report
- Latency measurements
- Dataset and augmentation examples