SIRIS-Lab/social-sciences_binary_classifier

📗 SPECTER2 – Social Sciences Classifier (Binary Classification)

This model is a fine-tuned version of allenai/specter2_base for identifying whether a scientific publication belongs to the Social Sciences domain.

It achieves the following results on the evaluation set:

• —Loss: 0.1382
• —Accuracy: 0.9670
• —F1 Micro: 0.9670
• —F1 Macro: 0.9480
• —F1 Weighted: 0.9670

Model description

This model performs binary document classification and predicts whether a publication belongs to the Social Sciences domain.

The model was trained using title and abstract text from multiple openly available datasets with native disciplinary annotations, including:

• —MAG / SciDocs
• —Elsevier Open Access (ASJC subject areas)
• —ERC panel datasets (publications and funded projects)

Each dataset was converted into a common binary label indicating whether a document belongs to the Social Sciences according to mappings from the original classification systems. :contentReference[oaicite:0]{index=0}

Key characteristics

• —Base model: allenai/specter2_base
• —Task: binary document classification
• —Labels:
• —False → Non-Social Sciences
• —True → Social Sciences
• —Activation: softmax
• —Loss: CrossEntropyLoss

Intended uses

This model is suitable for:

• —Identifying Social Sciences publications
• —Research information systems
• —Funding portfolio analysis
• —Metadata enrichment
• —Bibliometric analyses

The model accepts:

• —title
• —abstract
• —title + abstract (recommended)

Training data

Training data combines approximately 20,000 documents sampled from multiple sources:

• —MAG/SciDocs
• —ERC panel datasets (publications and projects)
• —Elsevier Open Access publications

Each source provides its own disciplinary taxonomy. Categories corresponding to Social Sciences were mapped into a common binary classification problem. :contentReference[oaicite:1]{index=1}

Training procedure

Preprocessing

• —Input text: title + abstract
• —Maximum sequence length: 512 tokens
• —Tokenization using the SPECTER2 tokenizer

Training hyperparameters

• —learning_rate: 2e-5
• —trainbatchsize: 32
• —evalbatchsize: 32
• —num_epochs: 4
• —max_length: 512
• —optimizer: AdamW
• —metric for best model: F1 Macro

Evaluation results

Limitations

• —The model predicts whether a publication belongs to the Social Sciences domain only.
• —It does not distinguish between individual Social Sciences disciplines.
• —Labels are derived from mappings between different disciplinary taxonomies and should be interpreted as high-level domain assignments.

Framework versions

• —Transformers 4.57.1
• —PyTorch 2.8.0
• —Datasets 3.6.0
• —Tokenizers 0.22.1