CoolFace
Modelpublic

abullard1/germeval2025-dbo-moderngbert-cw

sourceHugging Faceopenrailupdated 3mo agoView on Hugging Face
0likes10downloads
Model Card

<br> <br> <div style="text-align: center;"> <img src="https://i.ibb.co/RkR4QLpL/Shared-Task-Logo-Final-11zon.png" style="max-width: 30%; display: block; margin: 0 auto;"> </div>

<br> <br> <br>

<div style="text-align: center;"> <h1>🏆 GermEval 2025: Democratic Basic Order Attack Detection (Class-Weighted)</h1> <a href="https://github.com/abullard1/abullardUR-GermEval-Shared-Task-2025"> <p><strong>abullardUR@GermEval Shared Task 2025 Submission</strong></p> </div> <hr>

🎯 Model Summary

This model is a fine-tuned version of [LSX-UniWue/ModernGBERT_134M](https://huggingface.co/LSX-UniWue/ModernGBERT_134M), specifically designed for Attacks on Democratic Basic Order (DBO) Detection in German social media content. It was developed as part of the GermEval 2025 Shared Task on Harmful Content Detection.

🏅 Competition Performance

  • —Final Ranking: 6th out of 7 teams
  • —Primary Metric (Macro-F1): 0.63 (+34% over official baseline (0.47 → 0.63))
  • —Approach: Class-weighted cross-entropy loss for extreme multi-class imbalance (104.6:1 ratio)

📊 Task Details

  • —Task Type: Multi-class classification (4 classes)
  • —Classes:
  • —Nothing (84.2%): No attack on democratic order
  • —Criticism (10.8%): Legitimate criticism
  • —Agitation (4.2%): Harmful agitation
  • —Subversive (0.8%): Most severe attacks on democratic principles
  • —Domain: German social media (Twitter, 2014-2016)
  • —Data Source: Right-wing extremist network posts

⚠️ Limitations and Bias

Known Limitations

  • —Extreme Class Imbalance: Subversive class represents only 0.8% of data (60 samples)
  • —Domain Specificity: Trained on 2014-2016 German Twitter data from right-wing extremist networks
  • —Temporal Bias: Language patterns may not reflect contemporary usage
  • —Insufficient Examples: Subversive class may lack sufficient training examples

Ethical Considerations

  • —Model trained on potentially harmful content for research purposes only
  • —Should not be used to amplify or generate harmful content
  • —Requires careful handling due to sensitive training data
  • —May exhibit bias toward certain political contexts

🚀 How to Use

Quick Start

python
from transformers import AutoProcessor, AutoModelForSequenceClassification

# Load model and processor
model_id = "abullard1/germeval2025-dbo-moderngbert-cw"
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForSequenceClassification.from_pretrained(model_id, trust_remote_code=True).eval()

# Run inference
text = "Die aktuelle Regierung sollte ihre Politik überdenken."
inputs = processor(text, return_tensors="pt", truncation=True)
probs = model(**inputs).logits.softmax(-1).detach().cpu().numpy()
print(f"Predictions: {probs}")

Class Labels

  • —Label 0: Nothing (no attack on democratic order)
  • —Label 1: Criticism (legitimate criticism)
  • —Label 2: Agitation (harmful agitation against democratic order)
  • —Label 3: Subversive (severe attacks on democratic principles)

📈 Training Details

Training Data

  • —Source: GermEval 2025 Shared Task DBO dataset
  • —Size: 7,454 samples
  • —Split: 80% training (5,963), 20% validation (1,491)
  • —Class Distribution:
  • —Nothing: 6,277 (84.2%)
  • —Criticism: 804 (10.8%)
  • —Agitation: 313 (4.2%)
  • —Subversive: 60 (0.8%)

Training Procedure

  • —Base Model: ModernGBERT-134M (8192 token context)
  • —Architecture: Mean-pooling classification head
  • —Loss Function: Class-weighted cross-entropy (inverse frequency weighting)
  • —Optimizer: AdamW with linear scheduling
  • —Early Stopping: Patience of 5 epochs on validation Macro-F1

Hyperparameters

  • —Learning Rate: 1e-4
  • —Weight Decay: 0.0417
  • —Batch Size: 8/16 (train/eval)
  • —Epochs: 5
  • —Warmup Steps: 500

📚 Citation

bibtex
@inproceedings{bullard2025germeval,
  title   = {abullardUR@GermEval Shared Task 2025: Fine-tuning ModernGBERT on Highly Imbalanced German Social Media for Harmful Content Detection},
  author  = {Bullard, Samuel},
  year    = {2025},
  booktitle = {Proceedings of KONVENS 2025 Workshops}
}

🙏 Acknowledgments

  • —GermEval 2025 Organizers: University of Stuttgart and University of Mannheim
  • —Prof. Dr. Udo Kruschwitz (University of Regensburg) for supervision
  • —ModernGBERT Team: LSX-UniWue for the ModernGBERT-134M German language base-model

📄 License

This model inherits the Research-only RAIL-M license from ModernGBERT. See license details.