bhpac/mdeberta-v3-hunter-smm4h2026-ade

mDeBERTa-v3 Hunter — #SMM4H-HeaRD 2026 Task 1 (ADE screening)

This is the Stage 1 "Hunter" of the Bhramastra Hunter-Judge pipeline: a multilingual, high-recall binary filter that flags social-media posts that may mention an adverse drug event (ADE). It is microsoft/mdeberta-v3-base fine-tuned with a class-weighted loss for the strongly imbalanced ADE class (~7% prevalence). Posts it flags ("suspects") are meant to be passed to a downstream adjudicator (the Stage 2 "Judge", an LLM, or human review).

• —Paper: Bhramastra at #SMM4H-HeaRD 2026
• —Code: https://github.com/Team-Bhramastra/smm4h-task1-2026
• —Base model: mDeBERTa-v3-base (He et al., 2021)
• —Languages: en, de, fr, ru, ja, zh (trained); fa (zero-shot at inference only)

Standalone performance

These are standalone Hunter numbers (Table 6 ablation of the paper). They are not the performance of the full Hunter-Judge pipeline, which reaches 0.6653 F1.

The model is deliberately tuned for recall; its low precision is expected and is the reason a second stage is needed.

Intended use

• —Intended: a high-recall pre-filter that reduces the volume of posts sent to a downstream adjudicator (LLM or human review).
• —Not intended: use as a standalone ADE classifier (roughly two of three positive predictions are false positives), or for clinical decision-making of any kind.

Usage

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

repo = "bhpac/mdeberta-v3-hunter-smm4h2026-ade"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForSequenceClassification.from_pretrained(repo).eval()

x = tok("I got a terrible rash after taking ibuprofen.", return_tensors="pt", truncation=True)
with torch.no_grad():
    p_ade = torch.softmax(model(**x).logits, -1)[0, 1].item()  # index 1 = ADE

The pipeline applies language-specific probability thresholds calibrated for recall; see the code repository for the values and how they are used.

Training

Data. Fine-tuned on the official #SMM4H-HeaRD 2026 Task 1 training set plus translated CADEC instances, under the shared task's data-use agreement. No training data is redistributed with these weights.

Training hyperparameters

• —Base model: microsoft/mdeberta-v3-base
• —Loss: class-weighted cross-entropy, class weights [1.0, 10.0]
• —Learning rate: 2e-5 (matches the paper)
• —Weight decay: 0.01 (matches the paper)

Provenance of this checkpoint. The exact script and run configuration that produced this checkpoint could not be fully reconstructed from project history. hunter/train.py in the code repository has been corrected to implement the weighted-loss methodology (class weights [1, 10]) with the learning rate (2e-5) and weight decay (0.01) reported in the paper; other settings in that script (epochs, evaluation schedule, checkpoint-selection metric) are not guaranteed to match the original run. This released checkpoint, not any particular training script, is the authoritative artifact for reproducing the paper's reported results: the system's test-set predictions were scored by the #SMM4H-HeaRD 2026 organizers on their held-out test set (paper, Table 1).

Limitations

Low precision when used alone; performance on languages/domains outside the shared-task data (including zero-shot Persian) is not guaranteed; social-media text only; may reflect biases present in the training data.

License

MIT, matching the code repository. Note that the base model and the shared-task data carry their own terms.

Citation

@inproceedings{pachori-2026-bhramastra,
    title = "Bhramastra at {\#}{SMM4H}-{H}ea{RD} 2026: A Multi-Stage Hunter-Judge Pipeline using {DSP}y-Optimized {LLM}s for Multilingual {ADE} Detection",
    author = "Pachori, Bhaarat",
    booktitle = "Proceedings of the 11th Social Media Mining for Health Research and Applications ({SMM4H}-{H}ea{RD} 2026) Workshop and Shared Tasks",
    year = "2026",
    pages = "49--55",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2026.smm4h-1.9/",
    doi = "10.18653/v1/2026.smm4h-1.9",
}