akshara-ns/stairs-vs-nonstairs-automl

Stairs vs. non-stairs — AutoML-selected image classifier

Predicts whether a photograph shows stairs. The backbone architecture and its training configuration were selected by a budgeted AutoGluon hyperparameter search (Bayesian searcher, patience-based early stopping). The winning configuration fine-tunes an ImageNet-pretrained mobilenetv3_large_100 at 224 px.

Purpose and intended use

Intended: a worked example of budgeted neural architecture and hyperparameter search on a very small image dataset, with holdouts kept strictly to original photographs.

Not intended: any navigation, mobility, accessibility, or safety system. See Ethical notes.

Data

• —Source: `ArinRoths/StairvsNonStair_Dataset`, photographed by a 24-679 classmate for HW1 and used exactly as published. I am not the dataset author.
• —Scale: 32 original photographs (16 stairs, 16 non-stairs). 22 of them were expanded offline by the dataset author into 352 synthetic variants — mild_brightness, mild_contrast, mild_gaussian_blur, small_rotation.
• —License: the dataset page declares no license. If the author declares a license, this card should be updated to match.

Partitions

The published partitions were reused unchanged, after verifying in code that they are parent-disjoint and that both holdouts contain original photographs only.

Input and preprocessing

• —Input: an RGB photograph, passed to the predictor as a file path in a column named image.
• —Inside the model: resize the shorter side, center-crop to 224 × 224, and apply ImageNet mean/std normalization. Do not normalize images yourself.
• —To match training data: the dataset author corrected EXIF orientation, converted to RGB, resized preserving aspect ratio, and padded to a square with grey (128,128,128) before the 224 px resize. Photographs prepared differently are out of distribution.

Augmentation pipeline

1. 1.Offline, by the dataset author: four mild photometric and geometric recipes (above).
2. 2.Online, during training: TrivialAugment on every training image (one random operation at a random strength per sample).

Validation and test images receive no augmentation — only resize, crop, and normalization.

Model selection

Training setup (every trial)

• —Fine-tuning: all layers fine-tuned from ImageNet-pretrained timm weights, with layer-wise learning-rate decay (earlier layers receive a smaller learning rate; the decay factor is searched).
• —Schedule: cosine learning-rate decay with a 10% warm-up; up to 10 epochs.
• —Validation: twice per epoch on the 5 validation photographs; patience of 4 checks.
• —Checkpoints: the final model averages the best checkpoints of the winning trial (AutoGluon's "greedy soup"), keeping an averaged checkpoint only if it does not hurt validation log-loss.
• —Precision: 16-bit mixed precision on GPU. Seed: 24679 for training and for the searcher.

Search space: backbone ∈ {ResNet-18, ResNet-34, MobileNetV3-Small, MobileNetV3-Large, EfficientNet-B0}; input size ∈ {160, 224}; batch size ∈ {16, 32, 64}; optimizer ∈ {AdamW, SGD}; learning rate 1e-5 – 3e-3 (log); layer-wise LR decay 0.7 – 1.0; weight decay 1e-5 – 1e-1 (log); label smoothing ∈ {0, 0.1}; Mixup/CutMix ∈ {off, on}.

Selected configuration:

Why the shipped model is a refit. In AutoGluon 1.6.1 the tuner keeps each trial's highest-scoring checkpoints, which for log-loss are the worst ones, and builds its final model from them. On this run that model scored 1.747 on validation, against 0.146 for the trial it came from. The winning configuration was therefore retrained once with a normal fit (same seed, same settings), whose checkpointing minimises log-loss correctly; the shipped model scores 0.193 on validation, and the notebook asserts it matches that run's best checkpoints.

Every trial is listed in search_trials.csv in this repository. By architecture:

                       count     min  median
architecture                                
efficientnet_b0            2  0.7031  1.6127
mobilenetv3_large_100      2  0.1933  0.9892
mobilenetv3_small_100      2  1.2665  2.2805
resnet18                   1  0.7518  0.7518
resnet34                   5  0.4649  0.5135

Results — 5 original test photographs

                   accuracy  balanced accuracy  precision (stairs)  recall (stairs)  F1 (stairs)  log-loss
selected model          0.8              0.833               0.667              1.0          0.8     0.226
majority baseline       0.6              0.500               0.000              0.0          0.0       NaN

4 of 5 correct. Exact 95% confidence interval on accuracy: [0.28, 0.99]. With five test photographs, one error moves accuracy by 20 percentage points. The interval is the honest summary; the point estimate is not.

Per-class metrics (test):

              precision    recall  f1-score   support

  non-stairs      1.000     0.667     0.800         3
      stairs      0.667     1.000     0.800         2

    accuracy                          0.800         5
   macro avg      0.833     0.833     0.800         5
weighted avg      0.867     0.800     0.800         5

Every prediction, including the five validation photographs used for selection:

                      partition                 source_id       true  predicted  P(stairs)  correct
                           test     image_2026_stairs_014     stairs     stairs      0.999     True
                           test     image_2026_stairs_008     stairs     stairs      0.907     True
                           test image_2026_non-stairs_000 non-stairs     stairs      0.603    False
                           test image_2026_non-stairs_002 non-stairs non-stairs      0.098     True
                           test image_2026_non-stairs_001 non-stairs non-stairs      0.005     True
validation (used for selection) image_2026_non-stairs_004 non-stairs non-stairs      0.000     True
validation (used for selection) image_2026_non-stairs_009 non-stairs non-stairs      0.062     True
validation (used for selection)     image_2026_stairs_013     stairs     stairs      0.519     True
validation (used for selection)     image_2026_stairs_000     stairs     stairs      0.788     True
validation (used for selection)     image_2026_stairs_009     stairs     stairs      0.992     True

Known failure modes and limitations

• —32 original photographs in total, 22 for training. The effective sample size is the number of photographs, not the number of rows.
• —Five test photographs. Any accuracy figure here is compatible with a much better or much worse model; see the interval above.
• —Stairs present but not the subject — small, off-centre, or partly hidden — produce the least confident calls (a validation staircase half hidden by a bush scored P = 0.52).
• —Untested conditions: night, rain, snow, motion blur, crowds occluding the stairs, extreme viewing angles, spiral or open-riser stairs, escalators (likely called stairs), and ramps and kerb cuts — unknown behaviour, and the confusion that matters most for accessibility use.
• —Preprocessing mismatch: images without the author's grey square padding are out of distribution.
• —Uncalibrated probabilities: P(stairs) is not a trustworthy confidence estimate at this scale.

Ethical notes

Accessibility tooling needs validation at a scale this dataset cannot provide, designed with the people it serves.

Reproducing

Seed 24679; the budget and search space above.

Hardware / compute budget: Tesla T4. The search ran 12 trials in 52.2 minutes. The searcher is seeded, so the first 6 configurations it proposes are identical on every run. Later proposals depend on earlier trials' scores, and GPU training is not bit-for-bit deterministic, so a re-run can drift after that point and even select a different configuration — at n = 5 that is expected, not alarming.

Usage

import zipfile, pandas as pd
from huggingface_hub import hf_hub_download
from autogluon.multimodal import MultiModalPredictor

zip_file = hf_hub_download("akshara-ns/stairs-vs-nonstairs-automl", "autogluon_image_predictor_dir.zip")
zipfile.ZipFile(zip_file).extractall("stairs_predictor")
predictor = MultiModalPredictor.load("stairs_predictor")

photos = pd.DataFrame({"image": ["/path/to/photo.jpg"]})
print(predictor.predict(photos))          # 1 = stairs, 0 = non-stairs
print(predictor.predict_proba(photos))    # column 1 is P(stairs)

Requires autogluon.multimodal==1.6.1; loading with a different version is not guaranteed to work.

Credit and AI usage

Dataset by a 24-679 classmate: `ArinRoths/StairvsNonStair_Dataset`. Model by Akshara Nadayanur Sathis Kanna, 24-679, Carnegie Mellon University. Built with AutoGluon and timm pretrained weights.

AI-usage disclosure. Generative AI assisted in drafting the notebook code and this card; I reviewed, ran, corrected, and edited all of it. Every number above is produced by executing the notebook.

License: model weights Apache-2.0; the training data carries no declared license (see Data).