Tonic/video-swin-transformer
Collection shoaib6174/videoswintransformer/1
Collection of Video Swin Transformers feature extractor models.
<!-- task: video-feature-extraction -->
Overview
This collection contains different Video Swin Transformer [1] models. The original model weights are provided from [2]. There were ported to Keras models (tf.keras.Model) and then serialized as TensorFlow SavedModels. The porting steps are available in [3].
About the models
These models can be directly used to extract features from videos. These models are accompanied by Colab Notebooks with fine-tuning steps for action-recognition task and video-classification.
The table below provides a performance summary:
These scores for all the models are taken from [2].
Video Swin Transformer Feature extractors Models
- swin_tiny_patch244_window877_kinetics400_1k
- swin_small_patch244_window877_kinetics400_1k
- swin_base_patch244_window877_kinetics400_1k
- swin_base_patch244_window877_kinetics400_22k
- swin_base_patch244_window877_kinetics600_22k
- swin_base_patch244_window1677_sthv2
Notes
The input shape for these models are [None, 3, 32, 224, 224] representing [batch_size, channels, frames, height, width]. To create models with different input shape use this notebook.
References
[1] Video Swin Transformer Ze et al. [2] Video Swin Transformers GitHub [3] GSOC-22-Video-Swin-Transformers GitHub