francesco-zatto/stacked-bilstm-sexism-detector

LSTM Sexism Classifier (EXIST 2023)

This repository contains a custom PyTorch Bidirectional LSTM model trained for multi-class sexism detection. It was developed as part of an academic assignment based on Task 2 of the EXIST 2023 (EXpert Identification of SExist Tweets) dataset.

Model Details

• —Architecture: Bidirectional LSTM with a Dense classification head (Available in 1-layer Baseline or 2-layer Stacked variants).
• —Language: English (en).
• —Task: Multi-class Text Classification.
• —Embeddings: Pre-trained GloVe embeddings (glove-wiki-gigaword-100), with Out-Of-Vocabulary (OOV) terms embedded using context-window neighborhood averaging.

Intended Use

This model categorizes English tweets into one of four categories based on the author's intent:

1. 1.- (Non-sexist)
2. 2.DIRECT (Directly sexist messages)
3. 3.JUDGEMENTAL (Messages condemning sexist behaviors)
4. 4.REPORTED (Messages reporting a sexist situation)

Preprocessing Requirements

Because this is a custom PyTorch model, input text must be cleaned using the specific pipeline defined during training before being passed to the model. The pipeline includes:

• —Emoji translation (via the emoji library)
• —Removal of Twitter noise (mentions, URLs, hashtags, "via" tags)
• —Special character removal and curly quote normalization
• —Contraction handling (e.g., "won't" -> "will not")
• —Lemmatization (via nltk.WordNetLemmatizer)

Training Data

The model was trained on the English subset of the EXIST 2023 Task 2 dataset. Labels were aggregated from six annotators using majority voting. Tweets without a clear majority were discarded. Class imbalance was handled using weighted Cross-Entropy Loss.

Evaluation Metrics

The model was evaluated on the provided test split using Macro-averaged metrics across 5 different random seeds to ensure robust estimation.

• —Macro F1: 0.3899 +- 0.0192
• —Macro Precision: 0.3901 +- 0.0162
• —Macro Recall: 0.4160 +- 0.0213

How to Get Started with the Model

Since this is a pure PyTorch model (.pth weights) rather than a standard Hugging Face Transformer, you must instantiate the architecture in your code before loading the weights.

import torch
from huggingface_hub import hf_hub_download

# 1. Download the weights from the Hub
weights_path = hf_hub_download(
    repo_id="your-username/your-repo-name", 
    filename="best_lstm_weights.pth"
)

# 2. Instantiate your custom model class (BaselineModel or StackedModel)
# NOTE: You must have your custom class defined in your script
model = StackedModel() 

# 3. Load the state dictionary
model.load_state_dict(torch.load(weights_path, map_location=torch.device('cpu')))
model.eval()

print("Model successfully loaded!")