gefero/conflict_detection_ROBERTA_based

Model Card for XLM-RoBERTa Spanish Conflict Detection Classifier

<!-- Quick summary of what the model is/does. -->

A fine-tuned XLM-RoBERTa model for detecting social conflict mentions in Spanish news articles. The model is trained on the "Conflicto Social en Noticias" dataset and achieves 91.07% macro-F1 score on test data, making it suitable for automated content classification and conflict-related news filtering.

Model Details

Model Description

<!-- Longer summary of what this model is. -->

This is a binary text classification model based on FacebookAI/xlm-roberta-base fine-tuned to detect whether Spanish news articles discuss social conflict or not. The model was trained using a rigorous multi-seed approach (10 random seeds) to ensure robustness and generalization.

The classification task is binary:

• —CONFLICTO (1): News articles that discuss social conflict
• —NO_CONFLICTO (0): News articles that do not discuss social conflict

The model achieved strong performance across multiple evaluation runs, with consistent metrics indicating reliable predictions on unseen test data.

Model Details

• —Developed by: Germán Rosati (Factor~Data, SICSS-Buenos Aires)
• —Model type: Transformer-based text classification
• —Language(s) (NLP): Spanish (es)
• —License: MIT
• —Finetuned from model: FacebookAI/xlm-roberta-base

Model Sources

• —Repository: https://github.com/gefero/factordatatutoNLPSICSS
• —Dataset: agusnieto77/conflicto-social-noticias-4034

Uses

Direct Use

This model can be used for:

• —Automated news classification: Identify news articles discussing social conflict in Spanish-language sources
• —Content moderation: Flag conflict-related content for review or categorization
• —News aggregation: Filter and organize news by conflict relevance
• —Research and analytics: Systematic analysis of conflict coverage in news media
• —Social media monitoring: Detect posts discussing social conflict

Downstream Use [optional]

This model can be integrated into:

• —News recommendation systems to provide conflict-focused news feeds
• —Content management systems for automated news categorization
• —Data pipelines for media analysis research
• —Misinformation detection systems (as a conflict-detection component)

Out-of-Scope Use

This model is not suitable for:

• —Languages other than Spanish (though XLM-RoBERTa is multilingual, the model was fine-tuned only on Spanish data)
• —Content moderation decisions without human review (should be used as a scoring/filtering tool, not final arbiter)
• —Real-time moderation of live content streams without performance testing in your specific domain
• —Classification of informal text, social media, or user-generated content not resembling news articles (model trained on news)

Bias, Risks, and Limitations

Limitations

1. 1.Language: Model trained exclusively on Spanish news articles. Performance on other languages or dialects is unknown.

1. 1.Domain: Model trained on news articles. Performance on other text types (social media, academic text, etc.) may be degraded.

1. 1.Temporal bias: Dataset represents a specific time period. Linguistic evolution and emerging conflict narratives may not be captured.

1. 1.Class balance: Dataset contains both conflict and non-conflict examples. Performance may vary based on class distribution in your specific use case.

1. 1.Truncation: Text is truncated to 256 tokens (matching model's training setup). Very long articles may lose important context.

1. 1.Context sensitivity: "Conflict" detection is based on textual patterns. Sarcasm, irony, or indirect references may be misclassified.

Risks and Biases

• —Labeling bias: Model inherits any biases present in the original dataset annotation process
• —Geographic bias: News sources and conflict types in training data may not represent all Spanish-speaking regions equally
• —Media bias: Model trained on news articles, which may have their own coverage biases
• —Potential overreach: Model might flag articles mentioning conflict in non-concerning contexts (e.g., historical analysis, conflict resolution discussion)

Recommendations

1. 1.Always validate: Test the model on your specific data before production deployment
2. 2.Human review: Use model predictions as a starting point for human review, not as final decisions
3. 3.Monitor performance: Track model performance over time and across different domains
4. 4.Document decisions: Keep clear records of how the model is being used and any adjustments made
5. 5.Consider context: Combine model predictions with other signals for robust classification decisions

How to Get Started with the Model

Installation

pip install transformers torch

Quickstart - Using the Pipeline API

from transformers import pipeline

# Initialize the model
classifier = pipeline(
    "text-classification",
    model="gefero/conflict_detection_ROBERTA_based"
)

# Classify text
texts = [
    "El gobierno anunció nuevas políticas de seguridad social",
    "Miles de personas protestaron en las calles contra las medidas económicas"
]

results = classifier(texts)
for text, result in zip(texts, results):
    print(f"Text: {text[:50]}...")
    print(f"Label: {result['label']} (score: {result['score']:.4f})\n")

Quickstart - Using the Model Directly

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_name = "gefero/conflict_detection_ROBERTA_based"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

# Prepare input
text = "Manifestantes se enfrentan con la policía"
inputs = tokenizer(
    text,
    truncation=True,
    padding="max_length",
    max_length=256,
    return_tensors="pt"
)

# Get predictions
with torch.no_grad():
    outputs = model(**inputs)
    logits = outputs.logits
    predictions = torch.argmax(logits, dim=-1)
    confidence = torch.softmax(logits, dim=-1).max().item()

labels = {0: "NO_CONFLICTO", 1: "CONFLICTO"}
print(f"Prediction: {labels[predictions.item()]} (confidence: {confidence:.4f})")

Training Details

Training Data

• —Dataset: agusnieto77/conflicto-social-noticias-4034
• —Language: Spanish (es)
• —Domain: News articles
• —Splits used:
• —Training set: 70% (~2,823 examples)
• —Development set: 10% (~403 examples)
• —Test set: 20% (~806 examples)

Training Procedure

Preprocessing

• —Tokenization: XLM-RoBERTa tokenizer
• —Max length: 256 tokens (increased from default 128 to capture longer articles)
• —Truncation: Long articles truncated to max_length
• —Padding: Padded to max_length for batch processing

Training Hyperparameters

• —Base model: FacebookAI/xlm-roberta-base
• —Learning rate: 2e-5
• —Batch size: 16 (training), 64 (evaluation)
• —Epochs: 5
• —Weight decay: 0.01
• —Warmup steps: 0.1 (proportion of total training steps)
• —Evaluation strategy: Evaluate at the end of each epoch
• —Best model selection: Based on macro-F1 on development set
• —Hardware: GPU with FP16 (mixed precision) when available
• —Random seeds: 10 seeds [0, 1, 7, 13, 42, 100, 123, 2024, 31337, 65535] for robust evaluation

Training Details

• —Framework: Hugging Face Transformers
• —Optimizer: AdamW (default)
• —Metric for best model: macro-F1 (average of precision and recall across both classes)
• —Multi-seed training: Model was trained 10 times with different random seeds to ensure robustness and to provide confidence intervals on performance metrics

Evaluation

Testing Data, Factors & Metrics

Testing Data

• —Split: Test set (20% of original data, stratified split)
• —Size: ~806 examples
• —Language: Spanish
• —Domain: News articles from the original dataset

Metrics

• —Macro-F1 (primary metric): Average F1-score across both classes
• —Used for model selection during training
• —Balances precision and recall
• —Better for imbalanced or binary classification tasks

• —Accuracy: Overall correctness of predictions
• —F1-CONFLICTO: F1-score specifically for the CONFLICTO class (conflict-related news)
• —Precision: True positives / (true positives + false positives)
• —Recall: True positives / (true positives + false negatives)

Results

Official Test Results (10 runs with different seeds)

Per-Seed Results

Summary

The model demonstrates excellent performance with:

• —High macro-F1 (0.91): Balanced and strong predictions on both classes
• —High accuracy (0.94): Correct classification in 94% of cases
• —Robust across seeds: Low standard deviation indicates consistent generalization
• —Strong conflict detection (F1-CONFLICTO: 0.86): Reliably identifies conflict-related news

Environmental Impact

Compute Infrastructure

Hardware

• —GPU: NVIDIA GPU (exact model unspecified, but typical for Colab)
• —CPU: Supporting processors on Colab infrastructure
• —RAM: Standard Colab allocation

Training Time

• —Per seed: ~5-10 minutes (5 epochs per training run)
• —Total: ~50-100 minutes for 10 complete runs
• —Cloud Platform: Google Colaboratory (free tier)

Carbon Emissions

Estimated CO2 emissions for multi-seed training approach: Low to minimal (Colab's data centers use renewable energy sources). Individual training runs are short (~10 min each) and performed on highly optimized infrastructure.

For detailed calculations, see ML Impact Calculator.

Technical Specifications

Model Architecture and Objective

• —Architecture: Transformer-based sequence classification
• —Base: XLM-RoBERTa (12 layers, 768 hidden dimensions, 110M parameters)
• —Task-specific layer: Linear classification head for 2 classes

• —Objective: Binary cross-entropy loss (standard for text classification)

• —Multilingual base: XLM-RoBERTa trained on 100+ languages, fine-tuned here for Spanish-specific conflict detection

Input/Output

• —Input: Spanish text (news articles)
• —Max length: 256 tokens
• —Output: Class probabilities for [NO_CONFLICTO, CONFLICTO]

Citation [optional]

If you use this model in research, please cite:

BibTeX:

@software{rosati2024conflictdetection,
  author = {Rosati, Germán},
  title = {XLM-RoBERTa Spanish Conflict Detection Classifier},
  year = {2024},
  publisher = {Hugging Face Hub},
  url = {https://huggingface.co/gefero/conflict_detection_ROBERTA_based}
}

APA:

Rosati, G. (2024). XLM-RoBERTa Spanish Conflict Detection Classifier [Machine learning model]. Hugging Face Hub. Retrieved from https://huggingface.co/gefero/conflictdetectionROBERTA_based

Model Card Authors

• —Germán Rosati (Factor~Data, SICSS-Buenos Aires)

Model Card Contact

• —Email: german.rosati@gmail.com
• —GitHub: https://github.com/gefero