Modelpublic

fillo-rinaldi/t5-base-scitail

sourceHugging Faceupdated 10d agoView on Hugging Face

0likes30downloads

Model Card

T5-base SciTail

Full fine-tuning of google-t5/t5-base on allenai/scitail (tsv_format). The encoder-decoder backbone and the two-class classification head were fine-tuned.

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_id = "fillo-rinaldi/t5-base-scitail"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

Training

—Dataset: allenai/scitail, tsv_format
—Optimizer: AdamW, learning rate 5e-6, weight decay 0.01
—Batch size: 4 × 32 accumulation = effective batch 128
—Precision: fp32; epochs: 50; seed: 42
—Best checkpoint: validation accuracy 93.71%, test accuracy 92.90%

The original full checkpoint, extracted head, and training summaries are in artifacts/.