asjc-classification/scibert_multilabel_asjc_classifier

1---2license: mit3datasets:4- RichardErkhov/April_2023_Public_Data_File_from_Crossref5metrics:6- precision7- recall8- f19base_model:10- allenai/scibert_scivocab_uncased11pipeline_tag: text-classification12tags:13- scientometrics14- asjc15- multi-label16task_categories:17- text-classification18widget:19- text: "title={Jodometrie}, container_title={Fresenius' Zeitschrift für analytische Chemie, Zeitschrift für analytische Chemie}, abstract={}"20 21---22# 🧠 Open Multi-Label ASJC Classification23 24## Model Overview25This model fine-tunes **allenai/scibert_scivocab_uncased** across **307 ASJC subject categories**, enabling document-level classification beyond traditional journal-level schemes.26 27- **Task**: Multi-label classification  28- **Labels**: 307 ASJC subjects (compare [google sheet](https://docs.google.com/spreadsheets/d/1kqmGk2x0msodbaKDYt2RixyyB3MqOGrWS2azRGNsodw) for all labels)  29- **Base Model**: [SciBERT](https://arxiv.org/pdf/1903.10676)  30- **Training Data**: Crossref 2023 dataset (titles, abstracts, container titles)  31- **License**: MIT  32- **Framework**: Hugging Face Transformers  33 34---35 36## 📚 Intended Use37- Classify individual research documents into multiple ASJC subjects.38- Analyze disciplinary orientation of **collections** (authors, institutions, databases).39- Works with **title**, **abstract**, and optionally **container title** metadata.40 41---42 43## 🛠 Training Details44- **Preprocessing**:45  - Removed “multidisciplinary” and 26 “miscellaneous” categories → 307 subjects.46  - Multi-hot encoding for multi-label classification.47  - Data augmentation for underrepresented classes.48- **Fine-tuning**:49  - Optimizer: AdamW  50  - Loss: Binary Cross-Entropy  51  - Learning Rate: 2e-5  52  - Epochs: 1  53  - Batch Size: 16  54  - Threshold for label assignment: 0.3  55 56---57 58## 📈 Metrics59| Input Features                    | Labels | Precision | Recall | F1-Score (weighted) |60|-----------------------------------|--------|-----------|--------|---------------------|61| Title + Container Title + Abstract| 307    | 0.912     | 0.885  | 0.892               |62| Title + Abstract                  | 307    | 0.607     | 0.503  | 0.532               |63| Title + Container Title           | 307    | 0.949     | 0.957  | 0.952               |64| Title only                        | 307    | 0.528     | 0.416  | 0.448               |65 66For **26 parent subjects**, weighted F1-score improves to **0.934** using full metadata (Title + Container Title + Abstract) and **0.694** using Title + Abstract. 67 68The evaluation dataset is publicly available: [Test-Dataset](https://huggingface.co/datasets/asjc-classification/test-dataset)69 70---71 72## ✅ Model Strengths73- Handles **interdisciplinary** and **general science journals**.74- Works even without container title (lower accuracy).75- Scalable for large collections.76 77---78 79## ⚠️ Limitations80- Performance relies on metadata completeness (title, abstract, container title).81- Lower accuracy for rare subjects and missing source info.82- Snapshot of ASJC schema as of April 2023 (not updated for emerging fields).83 84---85 86## 🔍 Example Usage87 88```python89from transformers import TextClassificationPipeline, pipeline90import torch91 92# --- Custom multi-label pipeline ---93class ASJCMultiLabelPipeline(TextClassificationPipeline):94    """95    Multi-label classification pipeline for ASJC categories.96    Uses a configurable threshold to return all labels with scores above the threshold.97    """98    def __init__(self, *args, **kwargs):99        # Allow threshold override; default falls back to model config100        self.threshold = kwargs.pop("threshold", None)101        super().__init__(*args, **kwargs)102        if self.threshold is None:103            self.threshold = getattr(self.model.config, "threshold", 0.3)104 105    def postprocess(self, model_outputs, **kwargs):106        # Convert logits to probabilities using sigmoid107        scores = torch.sigmoid(torch.tensor(model_outputs["logits"])).tolist()108 109        results = []110        for i, score in enumerate(scores[0]):111            if score >= self.threshold:112                label = self.model.config.id2label[(i)]113                results.append({"label": label, "score": float(score)})114 115        # Sort by descending score116        results = sorted(results, key=lambda x: x["score"], reverse=True)117        return results118 119# --- Create the pipeline explicitly using the custom class ---120pipe = pipeline(121    task="text-classification",122    model="asjc-classification/scibert_multilabel_asjc_classifier",123    pipeline_class=ASJCMultiLabelPipeline124)125 126# --- Example text input ---127text = (128    "title={Dose optimization of β-lactams antibiotics in pediatrics and adults: A systematic review}, "129    "container_title={Frontiers in Pharmacology}, "130    "abstract={Background: β-lactams remain the cornerstone of the empirical therapy to treat various bacterial infections. This systematic review aimed to analyze the data describing the dosing regimen of β-lactams.Methods: Systematic scientific and grey literature was performed in accordance with Preferred Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. The studies were retrieved and screened on the basis of pre-defined exclusion and inclusion criteria. The cohort studies, randomized controlled trials (RCT) and case reports that reported the dosing schedule of β-lactams are included in this study.Results: A total of 52 studies met the inclusion criteria, of which 40 were cohort studies, 2 were case reports and 10 were RCTs. The majority of the studies (34/52) studied the pharmacokinetic (PK) parameters of a drug. A total of 20 studies proposed dosing schedule in pediatrics while 32 studies proposed dosing regimen among adults. Piperacillin (12/52) and Meropenem (11/52) were the most commonly used β-lactams used in hospitalized patients. As per available evidence, continuous infusion is considered as the most appropriate mode of administration to optimize the safety and efficacy of the treatment and improve the clinical outcomes.Conclusion: Appropriate antibiotic therapy is challenging due to pathophysiological changes among different age groups. The optimization of pharmacokinetic/pharmacodynamic parameters is useful to support alternative dosing regimens such as an increase in dosing interval, continuous infusion, and increased bolus doses.}"131)132 133# --- Get multi-label predictions ---134result = pipe(text)135print(result)136 137# Predicted labels:138# [139#   {'label': 'Pharmacology (medical)', 'score': 0.9922493696212769}, 140#   {'label': 'Pharmacology', 'score': 0.902540922164917}141# ]142 143# Expected labels:144# - Pharmacology (medical)145# - Pharmacology146```147 148---149 150## 📖 Citation151If you use this work, please cite:152 153```bibtex154@article{Gusenbauer.2025,155author = {Gusenbauer, Michael and Endermann, Jochen and Huber, Harald and Strasser, Simon and Granitzer, Andreas-Nizar and Ströhle, Thomas},156year = {2025},157title = {Fine-tuning SciBERT to enable ASJC-based assessments of the disciplinary orientation of research collections},158keywords = {All Science Journal Classification;Disciplinary coverage;Fine-tuning;multi-label classification;SciBERT;Transformer-based language models},159issn = {0138-9130},160journal = {Scientometrics},161doi = {10.1007/s11192-025-05490-0},162}