dronefreak/mc3-18-hmdb51-kinetics

  

MC3-18 HMDB51 (Kinetics-400 Init)

MC3-18 (Mixed Convolution 3D) fine-tuned on HMDB51 split 1, initialized from Kinetics-400 pretrained weights, trained as part of the video pipeline in human-action-classification. A reference baseline, not a state-of-the-art result -- published HMDB51 split-1 methods using ensembles, test-time augmentation, and multi-crop evaluation reach roughly 70-75%. A sibling model initialized from UCF-101 weights instead of Kinetics-400 is also available; see Related Resources below.

<br>

<!-- ROW 1: Identity & Tech Stack --> <div style="display: flex; justify-content: center; align-items: center; gap: 8px; margin-bottom: 8px; flex-wrap: wrap;"> <img src="https://img.shields.io/badge/Task-VideoClassification-blue?style=flat-square" alt="Task"> <img src="https://img.shields.io/badge/Architecture-MC3--18-0aa1a7?style=flat-square" alt="Architecture"> <img src="https://img.shields.io/badge/PretrainedOn-Kinetics--400-purple?style=flat-square" alt="Pretrained on Kinetics-400"> </div>

<!-- ROW 2: Performance Metrics --> <div style="display: flex; justify-content: center; align-items: center; gap: 8px; margin-bottom: 8px; flex-wrap: wrap;"> <img src="https://img.shields.io/badge/Accuracy-56.34%25-yellow?style=flat-square" alt="Accuracy"> <img src="https://img.shields.io/badge/F1-54.72%25-orange?style=flat-square" alt="F1 Score"> <img src="https://img.shields.io/badge/Params-11.5M-lightgrey?style=flat-square" alt="Params"> </div>

<!-- ROW 3: Metadata --> <div style="display: flex; justify-content: center; align-items: center; gap: 8px; margin-bottom: 24px; flex-wrap: wrap;"> <img src="https://img.shields.io/badge/License-Apache--2.0-lightgrey?style=flat-square" alt="License"> <a href="https://github.com/dronefreak/human-action-classification"><img src="https://img.shields.io/badge/Source-human--action--classification-black?style=flat-square" alt="Source"></a> </div>

Performance

Precision, recall and F1 are computed with macro averaging over HMDB51's 51 (equally-sized, 30 videos each) test classes -- macro recall equals accuracy here because the test split is exactly class-balanced.

Overfitting

A ~19-point train/validation gap is expected here: HMDB51 has only ~70 training videos per class, and MC3-18's 11.5M parameters are enough to memorize a set that size even with augmentation. See the UCF-101-initialized sibling model for a comparison with a smaller gap at similar accuracy.

Evaluation Protocol

Metrics above come from VideoTrainer.validate() in hac.video.training.train, run on HMDB51 split 1's test set (1,530 videos, 51 classes), at the checkpoint's best-performing epoch. Each clip: 8 frames sampled uniformly (no frame skipping), resized preserving aspect ratio to roughly 128x171, center-cropped to 112x112, normalized with Kinetics-400 statistics -- a single center clip per video, no test-time augmentation or multi-crop averaging.

Why 8 frames instead of 16? HMDB51 contains many short videos (some 10-20 frames total). An 8-frame window with no frame-skipping avoids the frame-repetition/tiling that a longer window would force on those clips, at the cost of a shorter temporal receptive field than the UCF-101 models in this project use.

Usage

Install Dependencies

Not yet published on PyPI -- install from source:

git clone https://github.com/dronefreak/human-action-classification
cd human-action-classification
pip install -e .

Load the Model from Hugging Face

import json
import torch
from huggingface_hub import hf_hub_download
from hac.video.models.classifier import Video3DCNN

config_path = hf_hub_download(repo_id="dronefreak/mc3-18-hmdb51-kinetics", filename="config.json")
weights_path = hf_hub_download(
    repo_id="dronefreak/mc3-18-hmdb51-kinetics",
    filename="mc3-18-hmdb51-kinetics.pth",
)

with open(config_path) as f:
    config = json.load(f)

model = Video3DCNN(
    num_classes=config["num_classes"],  # 51
    model_name=config["model_type"],
    pretrained=False,
)

checkpoint = torch.load(weights_path, map_location="cpu", weights_only=False)
model.load_state_dict(checkpoint["model_state_dict"])
model.eval()

Run Inference on a Video

The repo's VideoPredictor wraps frame sampling, transforms, and the forward pass end-to-end (pass num_frames=8 to match this model's training configuration):

from hac.video.inference.predictor import VideoPredictor

predictor = VideoPredictor(model_path=weights_path, num_frames=8, device="cpu")
result = predictor.predict_video("path/to/video.mp4", top_k=5)

print(result["top_class"], result["top_confidence"])

Note: VideoPredictor's built-in class list defaults to UCF-101's 101 classes -- for HMDB51 you'll want to pass/override the 51 class names listed below rather than relying on the predictor's default.

Training Configuration

Rows marked "checkpoint ..." are read directly out of the optimizer/scheduler state and config dict stored inside mc3-18-hmdb51-kinetics.pth. Rows marked "training script default" reflect hac.video.training.train's CLI defaults/flags at the time of training but weren't independently re-derived from the checkpoint for this exact run -- no separate run-config file was saved alongside it.

Kinetics-400 vs. UCF-101 Initialization

This project also ships an MC3-18/HMDB51 model initialized from UCF-101 weights instead of Kinetics-400 -- see mc3-18-hmdb51-ucf-transfer for that model's own card and verified numbers before drawing any conclusions from comparing the two; don't assume the two cards' headline framing is still in sync with each other.

• —Kinetics-400 init (this model): larger, more diverse pretraining corpus; uses 8-frame clips to avoid tiling short HMDB51 videos.
• —UCF-101 init: domain-closer to HMDB51 (similar YouTube/movie sources, overlapping action categories); uses 16-frame clips to match its own UCF-101 pretraining configuration, which causes frame tiling on HMDB51's shorter videos.

HMDB51 Classes

The model predicts 51 action classes: brushhair, cartwheel, catch, chew, clap, climb, climbstairs, dive, drawsword, dribble, drink, eat, fallfloor, fencing, flicflac, golf, handstand, hit, hug, jump, kick, kickball, kiss, laugh, pick, pour, pullup, punch, push, pushup, ridebike, ridehorse, run, shakehands, shootball, shootbow, shootgun, sit, situp, smile, smoke, somersault, stand, swingbaseball, sword, swordexercise, talk, throw, turn, walk, wave.

Known Limitations

• —Overfits on HMDB51's small per-class sample size (~19-point train/val gap) despite MixUp/CutMix/label smoothing.
• —Single model, no ensembling; no test-time augmentation (multi-crop, multi-clip temporal sampling).
• —Optimized for 8-frame clips; accuracy on substantially longer or shorter temporal windows is unverified.
• —Trained and evaluated on HMDB51 split 1 only -- performance on splits 2/3 is unverified.

Repository Contents

mc3-18-hmdb51-kinetics.pth
config.json
README.md

config.json doubles as the Hub's download-count query file: since this repo has no library_name integration the Hub recognizes, it falls back to counting requests against config.json (per Hugging Face's download-stats docs) -- the loading snippet above fetches it as part of normal usage, so downloads register.

Related Resources

• —mc3-18-hmdb51-ucf-transfer -- sibling model, same architecture/dataset, initialized from UCF-101 weights instead of Kinetics-400
• —mc3-18-ucf101 -- the UCF-101 model this project's UCF-101-init sibling was transferred from
• —human-action-classification -- the training/inference framework used to produce this checkpoint

Citation

If you use this model, please consider citing the HMDB51 dataset, the MC3 architecture, and the training framework:

@inproceedings{kuehne2011hmdb,
  title={HMDB: a large video database for human motion recognition},
  author={Kuehne, Hildegard and Jhuang, Hueihan and Garrote, Est{\'\i}baliz and Poggio, Tomaso and Serre, Thomas},
  booktitle={2011 International Conference on Computer Vision},
  pages={2556--2563},
  year={2011},
  organization={IEEE}
}

@inproceedings{tran2018closer,
  title={A Closer Look at Spatiotemporal Convolutions for Action Recognition},
  author={Tran, Du and Wang, Heng and Torresani, Lorenzo and Ray, Jamie and LeCun, Yann and Paluri, Manohar},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2018}
}

@misc{saksena2025mc3hmdbkinetics,
  author = {Saumya Saksena},
  title = {{MC3-18 HMDB51 (Kinetics-400 Init)}},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/dronefreak/mc3-18-hmdb51-kinetics}},
  note = {Trained with the human-action-classification framework, Top-1 Accuracy: 56.34\%}
}

@software{saksena2026hac,
  author       = {Saumya Saksena},
  title        = {{Human Action Classification: Pose-based and Video-based Models}},
  year         = 2026,
  publisher    = {GitHub},
  journal      = {GitHub repository},
  howpublished = {\url{https://github.com/dronefreak/human-action-classification}}
}

License

Apache-2.0