LukeFP/physh_topic_supervised_classifier

language: ["en"] libraryname: "pytorch" pipelinetag: "text-classification" base_model: "google/embeddinggemma-300m" tags: ["multi-label-classification", "hierarchical-classification", "physics", "physh", "transfer-learning"] metrics: ["f1"] ---

PhySH Topic Supervised Classifier

A hierarchical multi-label classifier that assigns APS Physics Subject Headings (PhySH) to physics papers using their titles and abstracts.

Model description

The model uses frozen `google/embeddinggemma-300m` representations with a two-stage conditioned classifier:

1. 1.Discipline classifier: Predicts 18 broad APS disciplines.
2. 2.Concept classifier: Predicts 186 research-area concepts, conditioned on the discipline probabilities produced by the first stage.

This classifier-chain design incorporates the hierarchy between broad disciplines and more specific research concepts.

Architecture

The input is formatted as:

{title} [SEP] {abstract}

EmbeddingGemma produces a 768-dimensional, L2-normalized representation. The encoder remains frozen during training.

Frozen encoder

google/embeddinggemma-300m → 768 dimensions

Discipline head

768 → 1024 → 512 → 18

Concept head

[768-dimensional embedding; 18 discipline probabilities]
786 → 1024 → 512 → 186

Both classification heads use ReLU activations and dropout with p = 0.3. Their outputs are independent sigmoid scores rather than a softmax distribution, allowing multiple labels to be assigned to each paper.

The two MLP heads contain approximately 2.75 million trainable parameters in total.

Training

• —Objective: Multi-label binary cross-entropy with logits
• —Optimizer: Adam
• —Encoder: Frozen
• —Hardware: Apple Metal Performance Shaders (MPS)
• —Optional objective: Focal loss
• —Training run: 20260130_140842

Evaluation

The model predicts approximately 2.12 labels per paper, compared with approximately 2.15 labels per paper in the evaluation data.

Evaluation-data note: Add the dataset source, number of examples, train/validation/test split sizes, and split methodology before treating these results as independently reproducible.

Inference

Inference applies a configurable threshold to each sigmoid score. If no score passes the threshold, the highest-scoring label is returned as a top-1 fallback.

Thresholds should be selected according to the intended precision–recall tradeoff:

• —Interactive demo threshold: 0.35
• —Batch-labeling threshold: 0.85

Try the model in the interactive demo.

Intended uses

This model is intended for:

• —Organizing and exploring physics literature
• —Suggesting PhySH labels for paper titles and abstracts
• —Supporting search, recommendation, and bibliometric workflows
• —Research on hierarchical multi-label classification

Predictions should be treated as label suggestions rather than authoritative APS classifications.

Limitations

• —Performance is lower on rare concepts, as reflected by the gap between micro-F1 and macro-F1.
• —Predictions depend on the information available in the title and abstract.
• —The model may not generalize to fields, terminology, or document types that differ substantially from its training data.
• —Output quality and label frequency depend on the selected inference threshold.
• —The frozen encoder limits task-specific representation learning.

Citation

If you use this model, please cite the model repository:

@misc{physh_topic_supervised_classifier,
  author    = {Luke F},
  title     = {PhySH Topic Supervised Classifier},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/LukeFP/physh_topic_supervised_classifier}
}