CoolFace
Modelpublic

ErfanMoosaviMonazzah/backpack-gpt2-nli

sourceHugging Faceapache-2.0updated 3y agoView on Hugging Face
1likes21downloads
Model Card

Model Card for Backpack-GPT2-NLI

This is a fine-tuned version of backpack-gpt2 with a NLI classification head on the esnli dataset. Results:

  • —On Validation Set:
  • —CrossEntropyLoss: 0.3168
  • —Accuracy: 0.9006
  • —F1: 0.9004
  • —On Test Set:
  • —CrossEntropyLoss: 0.3277
  • —Accuracy: 0.8958
  • —F1: 0.8955

Model Description

How to Get Started with the Model

python
from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained('gpt2')
tokenizer.pad_token = tokenizer.eos_token

def tokenize_function(examples):
    concatenated_sentences = [f'{premise.strip(".")}. ^ {hypothesis.strip(".")}.' for premise, hypothesis in zip(examples['premise'], examples['hypothesis'])]

    tokenized_inputs = tokenizer(
        concatenated_sentences,
        padding="max_length",
        truncation=True,
        max_length=512,
        return_tensors="pt",
    )
    return tokenized_inputs

model = AutoModelForSequenceClassification.from_pretrained('ErfanMoosaviMonazzah/backpack-gpt2-nli', trust_remote_code=True)
model.eval()

tokenized_sent = tokenize_function({
    'premise':['A boy is jumping on skateboard in the middle of a red bridge.',
               'Two women who just had lunch hugging and saying goodbye.',
               'Children smiling and waving at camera'],
    'hypothesis':['The boy does a skateboarding trick.',
                  'The friends have just met for the first time in 20 years, and have had a great time catching up.',
                  'The kids are frowning']
})
model.predict(input_ids=tokenized_sent['input_ids'], attention_mask=tokenized_sent['attention_mask'])

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 5e-5
  • —trainbatchsize: 64
  • —evalbatchsize: 64
  • —seed: 2023
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_ratio: 0
  • —num_epochs: 3

Training results

StepTraining LossValidation LossPrecisionRecallF1Accuracy
5120.6149000.4637130.8267920.8246390.8251330.824731
10240.5033000.4317960.8448310.8394140.8399800.839565
15360.4756000.4007710.8487410.8470090.8462870.847795
20480.4559000.3759810.8590640.8573570.8577490.857448
25600.4404000.3655370.8620000.8620780.8619170.862426
30720.4331000.3651800.8647170.8596930.8602370.859785
35840.4251000.3463400.8723120.8706350.8708650.870961
40960.4133000.3437610.8736060.8730460.8731740.873298
46080.4120000.3448900.8826090.8821200.8822550.882341
51200.4026000.3367440.8764630.8756290.8758270.875737
56320.3906000.3232480.8825980.8807790.8811290.880817
61440.3883000.3380290.8772550.8770410.8771260.877261
66560.3908000.3333010.8763570.8763620.8759650.876753
71680.3838000.3282970.8835930.8836750.8836290.883967
76800.3808000.3318540.8823620.8803730.8807640.880512
81920.3684000.3230760.8817300.8813780.8814190.881528
87040.3670000.3139590.8892040.8890470.8890530.889352
92160.3156000.3336370.8855180.8839650.8842660.883967
97280.3031000.3194160.8886670.8880920.8882560.888234
102400.3072000.3178270.8875750.8876470.8874180.888031
107520.3001000.3118100.8909080.8908270.8907470.891181
112640.3034000.3110100.8898710.8879390.8883090.887929
117760.3005000.3092820.8910410.8898190.8900770.889860
122880.3036000.3269180.8912720.8912500.8909420.891689
128000.3003000.3016880.8945160.8946190.8944810.894940
133120.3022000.3021730.8964410.8965270.8964620.896769
138240.2998000.2934890.8950470.8951720.8950840.895448
143360.2946000.2976450.8958650.8960120.8958860.896261
148480.2967000.3007510.8952770.8954010.8953040.895651
153600.2931000.2930490.8968550.8967050.8967570.896871
158720.2936000.2942010.8959330.8955570.8956240.895651
163840.2901000.2893670.8978470.8978890.8978400.898090
168960.2936000.2839900.8988890.8987240.8987890.898903
174080.2858000.3082570.8982500.8981020.8981620.898293
179200.2524000.3271640.8988600.8988070.8988310.899004
184320.2195000.3152860.8988770.8988350.8988310.899004
189440.2179000.3127380.8988570.8989580.8988860.899207
194560.1864000.3206690.8992520.8991660.8991940.899411
199680.1990000.3168400.9004580.9004550.9004260.900630

Model Card Authors

Erfan Moosavi Monazzah