SIRIS-Lab/social-sciences_binary_classifier

1# 📗 SPECTER2 – Social Sciences Classifier (Binary Classification)2 3This model is a fine-tuned version of **allenai/specter2_base** for identifying whether a scientific publication belongs to the **Social Sciences** domain.4 5It achieves the following results on the evaluation set:6 7- Loss: 0.13828- Accuracy: 0.96709- F1 Micro: 0.967010- F1 Macro: 0.948011- F1 Weighted: 0.967012 13## Model description14 15This model performs **binary document classification** and predicts whether a publication belongs to the **Social Sciences** domain.16 17The model was trained using title and abstract text from multiple openly available datasets with native disciplinary annotations, including:18 19- [MAG / SciDocs](https://github.com/allenai/scidocs)20- [Elsevier Open Access (ASJC subject areas)](https://researchcollaborations.elsevier.com/en/datasets/elsevier-oa-cc-by-corpus/)21- [ERC panel datasets (publications and funded projects)](https://huggingface.co/datasets/SIRIS-Lab/erc-classification-dataset)22 23Each dataset was converted into a common binary label indicating whether a document belongs to the Social Sciences according to mappings from the original classification systems. :contentReference[oaicite:0]{index=0}24 25**Key characteristics**26 27- Base model: `allenai/specter2_base`28- Task: binary document classification29- Labels:30  - `False` → Non-Social Sciences31  - `True` → Social Sciences32- Activation: softmax33- Loss: CrossEntropyLoss34 35## Intended uses36 37This model is suitable for:38 39- Identifying Social Sciences publications40- Research information systems41- Funding portfolio analysis42- Metadata enrichment43- Bibliometric analyses44 45The model accepts:46 47- title48- abstract49- title + abstract (recommended)50 51## Training data52 53Training data combines approximately **20,000** documents sampled from multiple sources:54 55- MAG/SciDocs56- ERC panel datasets (publications and projects)57- Elsevier Open Access publications58 59Each source provides its own disciplinary taxonomy. Categories corresponding to **Social Sciences** were mapped into a common binary classification problem. :contentReference[oaicite:1]{index=1}60 61## Training procedure62 63### Preprocessing64 65- Input text: `title + abstract`66- Maximum sequence length: **512 tokens**67- Tokenization using the SPECTER2 tokenizer68 69### Training hyperparameters70 71- learning_rate: 2e-572- train_batch_size: 3273- eval_batch_size: 3274- num_epochs: 475- max_length: 51276- optimizer: AdamW77- metric for best model: F1 Macro78 79## Evaluation results80 81| Metric | Value |82|--------|------:|83| Accuracy | 0.9670 |84| F1 Micro | 0.9670 |85| F1 Macro | 0.9480 |86| F1 Weighted | 0.9670 |87 88## Limitations89 90- The model predicts whether a publication belongs to the **Social Sciences** domain only.91- It does not distinguish between individual Social Sciences disciplines.92- Labels are derived from mappings between different disciplinary taxonomies and should be interpreted as high-level domain assignments.93 94## Framework versions95 96- Transformers 4.57.197- PyTorch 2.8.098- Datasets 3.6.099- Tokenizers 0.22.1