Datasetpublic

Mhmd08/inaturalist

iNaturalist QA Dataset This repository contains the complete iNaturalist dataset prepared for a question-answering / classification task. It includes every example from the original splits where: The taxon column was not null. The image at the provided url was reachable and successfully downloaded. All images are stored locally, so you can train and evaluate without relying on external URLs. 🔗 Original Dataset Source The original dataset can be found at:… See the full description on the dataset page: https://huggingface.co/datasets/Mhmd08/inaturalist.

sourceHugging Facemitupdated 1y agoView on Hugging Face

0likes19downloads

Dataset Card

iNaturalist QA Dataset

This repository contains the complete iNaturalist dataset prepared for a question-answering / classification task. It includes every example from the original splits where:

1.The taxon column was not null.
2.The image at the provided url was reachable and successfully downloaded.

All images are stored locally, so you can train and evaluate without relying on external URLs.

🔗 Original Dataset Source

The original dataset can be found at: [ba188/inaturalist](https://huggingface.co/datasets/ba188/inaturalist)

📂 Repository Layout

.
├── inaturalist_train.csv       # Train Dataset (contains Broken link)
├── inaturalist_test.csv        # Test Dataset (contains Broken link)
├── train_images/               # Images of valid links in train dateset
│   ├── 0.jpg
│   ├── 1.png
│   └── …
└── test_images/                # Images of valid links in test dateset
    ├── 0.jpg
    ├── 1.png
    └── …

🛠 Prerequisites

bash

pip install pandas datasets pillow torch torchvision

Optionally, if you plan to clone and pull images via Git:

bash

# Git LFS is required if you want to pull the actual image blobs
git lfs install

🔄 Data Filtering

We start by loading and cleaning the metadata:

python

import pandas as pd

TRAIN_CSV = "inaturalist_train.csv"
TEST_CSV  = "inaturalist_test.csv"

# 1. Read the CSVs
# 2. Drop any rows where 'taxon' is NaN
# 3. Reset the row indices so they go 0…N-1
train_df = (
    pd.read_csv(TRAIN_CSV)
      .dropna(subset=["taxon"])
      .reset_index(drop=True)
)

test_df = (
    pd.read_csv(TEST_CSV)
      .dropna(subset=["taxon"])
      .reset_index(drop=True)
)

Because we only include examples with a valid taxon and a downloaded image, the DataFrame indices match the image filenames one-to-one.

📦 Loading with 🤗 Datasets

You can also load everything directly from the Hub (images will be pulled via Git LFS):

python

from datasets import load_dataset

ds = load_dataset("Mhmd08/inaturalist")
train_df = ds["train"].to_pandas()
test_df  = ds["test"].to_pandas()

🔍 PyTorch Dataset Wrapper

Below is an example torch.utils.data.Dataset that pairs each image with its integer label:

python

import os
from PIL import Image
from torch.utils.data import Dataset

# Build a mapping from taxon string → integer class
label2id = {taxon: idx for idx, taxon in enumerate(sorted(train_df["taxon"].unique()))}

class InatLocalDataset(Dataset):
    def __init__(self, df, img_dir, transform):
        """
        df       : cleaned pandas DataFrame with 'taxon'
        img_dir  : path to local train_images/ or test_images/
        transform: torchvision transforms to apply
        """
        self.transform = transform
        self.items = []
        for idx, row in df.iterrows():
            fname_base = str(idx)
            # find the first file that starts with the index
            for fn in os.listdir(img_dir):
                if fn.startswith(fname_base + "."):
                    full_path = os.path.join(img_dir, fn)
                    label = label2id[row["taxon"]]
                    self.items.append((full_path, label))
                    break

    def __len__(self):
        return len(self.items)

    def __getitem__(self, i):
        path, label = self.items[i]
        img = Image.open(path).convert("RGB")
        return self.transform(img), label

Usage Example

python

from torchvision import transforms
from torch.utils.data import DataLoader

# Define your augmentations / preprocessing
train_tf = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.RandomHorizontalFlip(),
    transforms.ToTensor(),
])

test_tf = transforms.Compose([
    transforms.Resize((224, 224)),
    transforms.ToTensor(),
])

# Instantiate datasets
train_ds = InatLocalDataset(train_df, "train_images/", train_tf)
test_ds  = InatLocalDataset(test_df,  "test_images/",  test_tf)

# Wrap in DataLoader
train_loader = DataLoader(train_ds, batch_size=32, shuffle=True, num_workers=4)
test_loader  = DataLoader(test_ds,  batch_size=32, shuffle=False, num_workers=4)

📜 License

This dataset is shared under the MIT License. Feel free to reuse and adapt!\\\`