mubaraknumann/genera-cloud-image-classification

Genera - Cloud Image Classification Model

Version: 1.0.0 Last Updated: May 18 2025 Contact: [numanmubarak@protonmail.com][Github - https://github.com/mubaraknumann]

You can use the attached streamlit code to test the model using GUI. Steps -

1. 1.Place Files: Put the model.keras and labelmapping.json in the same directory as testmodel_streamlit.py.
2. 2.Install Libraries: pip install streamlit tensorflow numpy Pillow
3. 3.Execute the .py file or run from Terminal: streamlit run testmodelstreamlit.py

Table of Contents

1. 1.Model Description
2. 2.Intended Uses & Limitations
3. 3.How to Use
4. 4.Prerequisites
5. 5.Loading the Model
6. 6.Making Predictions
7. 7.Training Procedure
8. 8.Dataset: UGCI
9. 9.Architecture: RepVGG with NECA Attention
10. 10.Data Preprocessing & Augmentation
11. 11.Training Details
12. 12.Evaluation Results
13. 13.Custom Layers
14. 14.Roadblocks & Solutions During Development
15. 15.Future Work
16. 16.Citation
17. 17.License
18. 18.Acknowledgements

1. Model Description

Genera is a deep learning model designed for the classification of twelve distinct cloud genera from ground-based sky imagery. This model is an implementation of a custom RepVGG-style architecture enhanced with a New Efficient Channel Attention (NECA) mechanism, inspired by recent advancements in computer vision for atmospheric science (specifically drawing ideas from Shi et al., 2024, "Improved RepVGG ground-based cloud image classification with attention convolution").

The model was trained on the UGCI (Ultimate Ground-level Cloud Image) dataset, a custom-collected dataset of ground-based cloud images. It aims to provide a robust and efficient solution for automated cloud type identification, which can be a foundational component for personalized weather intelligence systems, meteorological research, and citizen science applications.

Key Features:

• —Classifies 12 cloud types: Altocumulus, Altostratus, Cirrocumulus, Cirrostratus, Cirrus, Clear Sky, Contrail, Cumulonimbus, Cumulus, Nimbostratus, Stratocumulus, and Stratus.
• —Utilizes a RepVGG architecture for efficient inference via structural re-parameterization.
• —Incorporates NECA-style channel attention for improved feature extraction.

2. Intended Uses & Limitations

Intended Uses:

• —Automated classification of cloud types from ground-based, full-sky or partial-sky images.
• —Supporting meteorological observations and local weather analysis.
• —Educational tool for learning about cloud formations.
• —Potential component for citizen science projects related to atmospheric monitoring.
• —Research baseline for further development in image-based atmospheric science.

Limitations:

• —The model is trained on the UGCI dataset. Its performance may vary on images with significantly different characteristics (e.g., camera types not represented in UGCI, extreme lighting conditions, heavy obstructions).
• —Currently classifies 12 cloud genera; it does not identify specific cloud species, varieties, or supplementary features beyond the primary genus.
• —Does not predict weather phenomena directly (e.g., rain, snow), only the cloud type which may be associated with such phenomena.
• —The accuracy for minority classes in the UGCI dataset (e.g., Altostratus, Stratus before dataset expansion) was initially lower, though significantly improved with more data. Performance on rare cloud types may still be less robust.

3. How to Use

Prerequisites

• —Python 3.8+
• —TensorFlow 2.10+ (or the version you used)
• —NumPy
• —Pillow (for image manipulation)
• —(Optional for full environment: pandas, scikit-learn, seaborn, matplotlib for data handling and visualization as in the training scripts)

You can install necessary packages using pip:

pip install tensorflow numpy Pillow

Loading the Model

The model is saved in the Keras native format (.keras). You will need to provide the definitions of the custom layers (RepVGGBlock and NECALayer) when loading.

IMPORTANT: You must have the RepVGGBlock and NECALayer class definitions available in your Python environment before running this.

--- CUSTOM LAYER DEFINITIONS ---

--- RepVGGBlock Class Definition ---

class RepVGGBlock(layers.Layer): def _init(self, inchannels, outchannels, kernelsize=3, stride=1, groups=1, deploy=False, usese=False, **kwargs): super(RepVGGBlock, self).init(**kwargs) self.configinitialinchannels = inchannels self.configoutchannels = outchannels self.configkernelsize = kernelsize self.configstridesval = stride self.configgroups = groups self.deploymodeinternal = deploy self.configusese = usese # Placeholder, not used in this version of RepVGGBlock self.actualinchannels = None

self.rbrdenseconv = layers.Conv2D( filters=self.configoutchannels, kernelsize=self.configkernelsize, strides=self.configstridesval, padding='same', groups=self.configgroups, usebias=False, name=self.name + 'denseconv' ) self.rbrdensebn = layers.BatchNormalization(name=self.name + 'densebn') self.rbr1x1conv = layers.Conv2D( filters=self.configoutchannels, kernelsize=1, strides=self.configstridesval, padding='valid', groups=self.configgroups, usebias=False, name=self.name + '1x1conv' ) self.rbr1x1bn = layers.BatchNormalization(name=self.name + '1x1bn') self.rbridentitybn = None self.rbrreparam = layers.Conv2D( filters=self.configoutchannels, kernelsize=self.configkernelsize, strides=self.configstridesval, padding='same', groups=self.configgroups, usebias=True, name=self.name + 'reparamconv' )

def build(self, inputshape): self.actualinchannels = inputshape[-1] if self.configinitialinchannels is None: self.configinitialinchannels = self.actualinchannels elif self.configinitialinchannels != self.actualinchannels: raise ValueError(f"Input channel mismatch for {self.name}: Expected {self.configinitialinchannels}, got {self.actualinchannels}")

if self.rbridentitybn is None and \ self.actualinchannels == self.configoutchannels and self.configstridesval == 1: self.rbridentitybn = layers.BatchNormalization(name=self.name + 'identitybn')

super(RepVGGBlock, self).build(input_shape) # Call super build first

# Ensure all sub-layers are built if not self.rbrdenseconv.built: self.rbrdenseconv.build(inputshape) if not self.rbrdensebn.built: self.rbrdensebn.build(self.rbrdenseconv.computeoutputshape(inputshape)) if not self.rbr1x1conv.built: self.rbr1x1conv.build(inputshape) if not self.rbr1x1bn.built: self.rbr1x1bn.build(self.rbr1x1conv.computeoutputshape(inputshape)) if self.rbridentitybn is not None and not self.rbridentitybn.built: self.rbridentitybn.build(inputshape) if not self.rbrreparam.built: self.rbrreparam.build(inputshape)

def call(self, inputs): if self.deploymodeinternal: return self.rbrreparam(inputs) else: # Training mode outdense = self.rbrdensebn(self.rbrdenseconv(inputs)) out1x1 = self.rbr1x1bn(self.rbr1x1conv(inputs)) if self.rbridentitybn is not None: outidentity = self.rbridentitybn(inputs) return outdense + out1x1 + outidentity else: return outdense + out1x1

def fusebntensor(self, convlayer, bnlayer): kernel = convlayer.kernel; dtype = kernel.dtype; outchannels = kernel.shape[-1] gamma = getattr(bnlayer, 'gamma', tf.ones(outchannels, dtype=dtype)) beta = getattr(bnlayer, 'beta', tf.zeros(outchannels, dtype=dtype)) runningmean = getattr(bnlayer, 'movingmean', tf.zeros(outchannels, dtype=dtype)) runningvar = getattr(bnlayer, 'movingvariance', tf.ones(outchannels, dtype=dtype)) epsilon = bnlayer.epsilon; std = tf.sqrt(runningvar + epsilon) fusedkernel = kernel (gamma / std) if conv_layer.use_bias: fused_bias = beta + (gamma (convlayer.bias - runningmean)) / std else: fusedbias = beta - (runningmean * gamma) / std return fusedkernel, fusedbias

def reparameterize(self): if self.deploymodeinternal: return branchestocheck = [self.rbrdenseconv, self.rbrdensebn, self.rbr1x1conv, self.rbr1x1bn] if self.rbridentitybn: branchestocheck.append(self.rbridentitybn) for branchlayer in branchestocheck: if not branchlayer.built: # Or len(branchlayer.weights) == 0 raise Exception(f"ERROR: Branch layer {branch_layer.name} for {self.name} not built. Call model with data first.")

kerneldense, biasdense = self.fusebntensor(self.rbrdenseconv, self.rbrdensebn) kernel1x1unpadded, bias1x1 = self.fusebntensor(self.rbr1x1conv, self.rbr1x1bn) padamount = self.configkernelsize // 2 kernel1x1padded = tf.pad(kernel1x1unpadded, [[padamount,padamount],[padamount,padamount],[0,0],[0,0]]) finalkernel = kerneldense + kernel1x1padded finalbias = biasdense + bias1x1 if self.rbridentitybn is not None: runningmeanid = self.rbridentitybn.movingmean; runningvarid = self.rbridentitybn.movingvariance gammaid = self.rbridentitybn.gamma; betaid = self.rbridentitybn.beta epsilonid = self.rbridentitybn.epsilon; stdid = tf.sqrt(runningvarid + epsilonid) kernelidscaler = gammaid / stdid biasidterm = betaid - (runningmeanid * gammaid) / stdid identitykernelnp = np.zeros((self.configkernelsize, self.configkernelsize, self.actualinchannels, self.configoutchannels), dtype=np.float32) for i in range(self.actualinchannels): identitykernelnp[padamount, padamount, i, i] = kernelidscaler[i].numpy() kernelidfinal = tf.converttotensor(identitykernelnp, dtype=tf.float32) finalkernel += kernelidfinal; finalbias += biasidterm if not self.rbrreparam.built: raise Exception(f"CRITICAL ERROR: {self.rbrreparam.name} of {self.name} not built before setweights.") self.rbrreparam.setweights([finalkernel, finalbias]) self.deploymode_internal = True

def getconfig(self): config = super(RepVGGBlock, self).getconfig() config.update({ "inchannels": self.configinitialinchannels, "outchannels": self.configoutchannels, "kernelsize": self.configkernelsize, "stride": self.configstridesval, "groups": self.configgroups, "deploy": self.deploymodeinternal, "usese": self.configusese }) return config @classmethod def fromconfig(cls, config): return cls(config) --- End of RepVGGBlock ---**

--- NECALayer Class Definition ---

class NECALayer(layers.Layer): def _init(self, channels, gamma=2, b=1, **kwargs): super(NECALayer, self).init(**kwargs) self.channels = channels self.gamma = gamma self.b = b tfchannels = tf.cast(self.channels, tf.float32) kfloat = (tf.math.log(tfchannels) / tf.math.log(2.0) + self.b) / self.gamma kint = tf.cast(tf.round(kfloat), tf.int32) if tf.equal(kint % 2, 0): self.kscalarval = kint + 1 else: self.kscalarval = kint self.kscalarval = tf.maximum(1, self.kscalarval) kernelsizeforconv1d = (int(self.kscalarval.numpy()),) self.gap = layers.GlobalAveragePooling2D(keepdims=True) self.conv1d = layers.Conv1D(filters=1, kernelsize=kernelsizeforconv1d, padding='same', usebias=False, name=self.name + 'eca_conv1d') self.sigmoid = layers.Activation('sigmoid')

def call(self, inputs): if self.channels != inputs.shape[-1]: raise ValueError(f"Input channels {inputs.shape[-1]} != layer channels {self.channels} for {self.name}") x = self.gap(inputs) x = tf.squeeze(x, axis=[1, 2]) x = tf.expanddims(x, axis=-1) x = self.conv1d(x) x = tf.squeeze(x, axis=-1) attention = self.sigmoid(x) attentionreshaped = tf.reshape(attention, [-1, 1, 1, self.channels]) return inputs * attention_reshaped

def getconfig(self): config = super(NECALayer, self).getconfig() config.update({"channels": self.channels, "gamma": self.gamma, "b": self.b}) return config @classmethod def from_config(cls, config): return cls(**config)

--- End of NECALayer ---

--- END OF CUSTOM LAYER DEFINITIONS ---

import tensorflow as tf from tensorflow import keras

MODELFILE = 'path/to/your/repvggnecadeployfinal.keras' # Replace with actual path LABELMAPPINGFILE = 'path/to/your/label_mapping.json' # Replace with actual path

customobjects = {'RepVGGBlock': RepVGGBlock, 'NECALayer': NECALayer} loadedmodel = tf.keras.models.loadmodel(MODELFILE, customobjects=customobjects, compile=False) print("Model loaded successfully!") loaded_model.summary() # Optional: to see the loaded architecture

Load label mapping

import json with open(LABELMAPPINGFILE, 'r') as f: labelmapdata = json.load(f) inttolabel = {int(k): v for k, v in labelmapdata['inttolabel'].items()}

Making Predictions

from PIL import Image import numpy as np

def preprocessimageforprediction(imagepathorpilimage, targetsize=(299, 299)): if isinstance(imagepathorpilimage, str): img = Image.open(imagepathorpilimage) else: # Assuming PIL image img = imagepathorpilimage

img = img.convert('RGB') # Ensure 3 channels img = img.resize(targetsize) imgarray = np.array(img, dtype=np.float32) imgarray = imgarray / 255.0 # Normalize to [0, 1] imgarray = np.expanddims(imgarray, axis=0) # Add batch dimension return imgarray

# Example prediction: imagepath = 'path/to/your/cloudimage.jpg' # Replace with your image path inputtensor = preprocessimageforprediction(imagepath) predictions = loadedmodel.predict(inputtensor) predictedprobabilities = predictions[0]

# Get top prediction predictedclassindex = np.argmax(predictedprobabilities) predictedclassname = inttolabel.get(predictedclassindex, "Unknown Class") confidence = predictedprobabilities[predictedclassindex]

print(f"Predicted Cloud Type: {predictedclassname}") print(f"Confidence: {confidence*100:.2f}%")

# Display all class probabilities (optional) for i, prob in enumerate(predictedprobabilities): classname = inttolabel.get(i, f"Class{i}") print(f"- {classname}: {prob*100:.2f}%")

4. Training Procedure

Dataset: UGCI

The model was trained on UGCI, a custom dataset of ground-based cloud images collected by [https://github.com/mubaraknumann].

Total Images (after expansion): ~32,742 images (Train: 22,918, Val: 4,912, Test: 4,912).

Classes (12): Altocumulus, Altostratus, Cirrocumulus, Cirrostratus, Cirrus, Clear Sky, Contrail, Cumulonimbus, Cumulus, Nimbostratus, Stratocumulus, Stratus.

Image Characteristics: Images were captured using various mobile and stationary cameras, representing diverse lighting conditions and geographical locations, primarily focusing on full-sky or wide-angle views.

Data Splitting: Stratified 70% training, 15% validation, 15% testing.

Architecture: RepVGG with NECA Attention

The model architecture is based on RepVGG, which features structural re-parameterization (multi-branch for training, single fused 3x3 convolution per block for inference). Each RepVGG block is followed by a New Efficient Channel Attention (NECA) module and a ReLU activation.

RepVGG Configuration:

Stages: 4

Blocks per stage: [1, 2, 4, 1]

Channels per stage: [64, 128, 256, 512]

NECA Parameters: gamma=2, b=1 for adaptive kernel size calculation in the 1D convolution.

Data Preprocessing & Augmentation

Input Size: Images were resized to 299x299 pixels.

Normalization: Pixel values were scaled to the `` range.

Training Augmentations (Stronger):

Random Horizontal Flips

Random Rotations (up to ~30 degrees, factor 0.1)

Random Zoom (up to 10%)

Random Translation (up to 5%)

Random Brightness adjustments (factor 0.3)

Random Contrast adjustments (factor 0.3)

Training Details

Framework: TensorFlow/Keras

Compute - Nvidia A100 GPU (Google Colab)

Optimizer: AdamW (learningrate=1e-4, weightdecay=5e-5)

Loss Function: Sparse Categorical Crossentropy

Class Imbalance: Addressed using balanced class weights during training.

Callbacks:

ModelCheckpoint (saving best model based on val_accuracy)

EarlyStopping (monitoring valloss, patience 20, restorebest_weights=True)

ReduceLROnPlateau (monitoring val_loss, patience 10)

Epochs: Trained for 200 epochs (~7 hours) (EarlyStopping intervened). The best model was restored from Epoch 171 of the final run.

Batch Size: 32

5. Evaluation Results

The final RepVGG+NECA deploy model (loaded from file) achieved the following on the UGCI test set:

Overall Test Accuracy: ~90.15%

Overall Test Loss: ~0.3968

Macro Average F1-score: ~0.88

Weighted Average F1-score: ~0.90

Per-Class Performance (F1-score from final run):

Altocumulus: 0.94

Altostratus: 0.89

Cirrocumulus: 0.85

Cirrostratus: 0.82

Cirrus: 0.91

Clear Sky: 1.00

Contrail: 0.68

Cumulonimbus: 0.94

Cumulus: 0.95

Nimbostratus: 0.85

Stratocumulus: 0.88

Stratus: 0.87

Training/Validation Graph

Confusion Matrix

Per Class F1 Scores

6. Custom Layers

This model utilizes two custom Keras layers. Their Python class definitions are required to load and use the model.

RepVGGBlock: Implements the re-parameterizable block.

NECALayer: Implements the New Efficient Channel Attention mechanism.

(You would typically provide the code for these layers in a separate .py file or directly in notebooks/scripts that use the model).

7. Roadblocks & Solutions During Development

The development process involved several key challenges, primarily related to the custom RepVGGBlock:

Initial Training Instability: Early RepVGG training attempts suffered from exploding validation losses, addressed by significantly reducing the learning rate and using AdamW with weight decay.

Reparameterization Errors: Numerous ValueError and AttributeError issues occurred when trying to convert the trained multi-branch RepVGG blocks to their single-branch inference form. This was due to Keras's layer build lifecycle and how it handles sub-layers that are not in the active computation graph during training mode.

Solution: The RepVGGBlock class was iteratively refined. The final robust solution involved:

Defining all potential sub-layers (training branches AND the deploy-mode fused Conv2D layer) in _init_.

Implementing a build(self, inputshape) method in RepVGGBlock that explicitly ensures all these sub-layers (including the deploy-mode rbrreparam Conv2D) are built when the RepVGGBlock itself is built by Keras (e.g., when data first flows through the model).

The reparameterize() method then calculates fused weights and sets them onto the already existing and built rbr_reparam sub-layer.

Save/Load Consistency: Ensuring that the saved deploy-mode model correctly loaded and performed identically to the in-memory reparameterized version also required careful management of the custom layer's config and state. The final approach proved successful.

Data Augmentation Issues: Initial problems with augmented images appearing black were traced to RandomBrightness layer defaults and fixed by specifying value_range=(0.0, 1.0).

A "quick test" script was developed to rapidly iterate on and debug the RepVGGBlock's reparameterization and save/load mechanism without requiring full model training cycles.

8. Future Work

Further hyperparameter tuning and exploration of more advanced data augmentation.

Experimentation with different RepVGG architectural variants (depth/width).

Continued refinement of minority class performance (contrail).

9. Citation

If you use this model or code in your research, please consider citing this repository (and any associated paper, if applicable).

Mohammed Numan Mubarak. (2025). Genera - Cloud Image Classification Model. Retrieved from [huggingface.co/mubaraknumann/genera-cloud-image-classification]

10. License

This project, including the model weights and source code, is licensed under the MIT License. See the LICENSE file for more details.

11. Acknowledgements

This work was inspired by the methodologies presented in "Improved RepVGG ground-based cloud image classification with attention convolution" by Shi et al. (2024).