abullard1/germeval2025-vio-moderngbert-cw_and_focal

<br> <br> <div style="text-align: center;"> <img src="https://i.ibb.co/RkR4QLpL/Shared-Task-Logo-Final-11zon.png" style="max-width: 30%; display: block; margin: 0 auto;"> </div>

<br> <br> <br>

<div style="text-align: center;"> <h1>🏆 GermEval 2025: Violence Detection (Class-Weighted + Focal Loss)</h1> <a href="https://github.com/abullard1/abullardUR-GermEval-Shared-Task-2025"> <p><strong>abullardUR@GermEval Shared Task 2025 Submission</strong></p> </div> <hr>

🎯 Model Summary

This model is a fine-tuned version of [LSX-UniWue/ModernGBERT_134M](https://huggingface.co/LSX-UniWue/ModernGBERT_134M), specifically designed for Violence Detection (VIO) in German social media content. It was developed as part of the GermEval 2025 Shared Task on Harmful Content Detection.

🏅 Competition Performance

• —Final Ranking: 3rd out of 8 teams
• —Primary Metric (Macro-F1): 0.81 (+17% over official baseline (0.69 → 0.81))
• —Approach: Combined class-weighted cross-entropy + focal loss for severe imbalance handling

📊 Task Details

• —Task Type: Binary classification
• —Classes: False (no violence), True (violence-related content detected)
• —Domain: German social media (Twitter, 2014-2016)
• —Data Source: Right-wing extremist network posts

⚠️ Limitations and Bias

Known Limitations

• —Domain Specificity: Trained on 2014-2016 German Twitter data from right-wing extremist networks
• —Temporal Bias: Language patterns may not reflect contemporary usage
• —Class Imbalance: 92.8% non-violent vs 7.2% violent content (12.8:1 ratio)
• —Cultural Context: May not generalize to other German-speaking regions or contexts
• —Hard Example Focus: Focal loss may down-weight some important minority examples

Ethical Considerations

• —Model trained on potentially harmful content for research purposes only
• —Should not be used to amplify or generate violent content
• —Requires careful handling due to sensitive training data

🚀 How to Use

Quick Start

from transformers import AutoProcessor, AutoModelForSequenceClassification

# Load model and processor
model_id = "abullard1/germeval2025-vio-moderngbert-cw_and_focal"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(model_id, trust_remote_code=True).eval()

# Run inference
text = "Diese Situation macht mich wütend!"
inputs = processor(text, return_tensors="pt", truncation=True)
probs = model(**inputs).logits.softmax(-1).detach().cpu().numpy()
print(f"Predictions: {probs}")

Class Labels

• —Label 0: No violence detected
• —Label 1: Violence-related content detected

📈 Training Details

Training Data

• —Source: GermEval 2025 Shared Task VIO dataset
• —Size: 7,783 samples
• —Split: 80% training (6,226), 20% validation (1,557)
• —Class Distribution: 92.8% non-violent, 7.2% violent

Training Procedure

• —Base Model: ModernGBERT-134M (8192 token context)
• —Architecture: Mean-pooling classification head
• —Loss Function: Class-weighted cross-entropy + Focal Loss
• —Optimizer: AdamW with linear scheduling
• —Early Stopping: Patience of 5 epochs on validation Macro-F1

Hyperparameters

• —Learning Rate: 3e-5
• —Weight Decay: 0.0811
• —Batch Size: 16/32 (train/eval)
• —Epochs: 3
• —Warmup Steps: 100
• —Focal Loss Gamma: 0.519

📚 Citation

@inproceedings{bullard2025germeval,
  title   = {abullardUR@GermEval Shared Task 2025: Fine-tuning ModernGBERT on Highly Imbalanced German Social Media for Harmful Content Detection},
  author  = {Bullard, Samuel},
  year    = {2025},
  booktitle = {Proceedings of KONVENS 2025 Workshops}
}

🙏 Acknowledgments

• —GermEval 2025 Organizers: University of Stuttgart and University of Mannheim
• —Prof. Dr. Udo Kruschwitz (University of Regensburg) for supervision
• —ModernGBERT Team: LSX-UniWue for the ModernGBERT-134M German language base-model

📄 License

This model inherits the Research-only RAIL-M license from ModernGBERT. See license details.