Softechlb/Sent_analysis_CVs

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

Softechlb/SentanalysisCVs

This model is distilled from the zero-shot classification pipeline on the Multilingual Sentiment dataset using this script.

In reality the multilingual-sentiment dataset is annotated of course, but we'll pretend and ignore the annotations for the sake of example.

Teacher model: MoritzLaurer/mDeBERTa-v3-base-mnli-xnli Teacher hypothesis template: "The sentiment of this text is {}." Student model: distilbert-base-multilingual-cased

Inference example

from transformers import pipeline

distilled_student_sentiment_classifier = pipeline(
    model="Softechlb/Sent_analysis_CVs", 
    return_all_scores=True
)

# english
distilled_student_sentiment_classifier ("I love this movie and i would watch it again and again!")
>> [[{'label': 'positive', 'score': 0.9731044769287109},
  {'label': 'neutral', 'score': 0.016910076141357422},
  {'label': 'negative', 'score': 0.009985478594899178}]]

# malay
distilled_student_sentiment_classifier("Saya suka filem ini dan saya akan menontonnya lagi dan lagi!")
[[{'label': 'positive', 'score': 0.9760093688964844},
  {'label': 'neutral', 'score': 0.01804516464471817},
  {'label': 'negative', 'score': 0.005945465061813593}]]

# japanese
distilled_student_sentiment_classifier("私はこの映画が大好きで、何度も見ます！")
>> [[{'label': 'positive', 'score': 0.9342429041862488},
  {'label': 'neutral', 'score': 0.040193185210227966},
  {'label': 'negative', 'score': 0.025563929229974747}]]

### Training log

Training completed. Do not forget to share your model on huggingface.co/models =)

{'trainruntime': 2009.8864, 'trainsamplespersecond': 73.0, 'trainstepspersecond': 4.563, 'trainloss': 0.6473459283913797, 'epoch': 1.0} 100%|███████████████████████████████████████| 9171/9171 [33:29<00:00, 4.56it/s] [INFO|trainer.py:762] 2023-05-06 10:56:18,555 >> The following columns in the evaluation set don't have a corresponding argument in DistilBertForSequenceClassification.forward and have been ignored: text. If text are not expected by DistilBertForSequenceClassification.forward, you can safely ignore this message. [INFO|trainer.py:3129] 2023-05-06 10:56:18,557 >> *** Running Evaluation *** [INFO|trainer.py:3131] 2023-05-06 10:56:18,557 >> Num examples = 146721 [INFO|trainer.py:3134] 2023-05-06 10:56:18,557 >> Batch size = 128 100%|███████████████████████████████████████| 1147/1147 [08:59<00:00, 2.13it/s] 05/06/2023 11:05:18 - INFO - _main - Agreement of student and teacher predictions: 88.29% [INFO|trainer.py:2868] 2023-05-06 11:05:18,251 >> Saving model checkpoint to ./distilbert-base-multilingual-cased-sentiments-student [INFO|configurationutils.py:457] 2023-05-06 11:05:18,251 >> Configuration saved in ./distilbert-base-multilingual-cased-sentiments-student/config.json [INFO|modelingutils.py:1847] 2023-05-06 11:05:18,905 >> Model weights saved in ./distilbert-base-multilingual-cased-sentiments-student/pytorchmodel.bin [INFO|tokenizationutilsbase.py:2171] 2023-05-06 11:05:18,905 >> tokenizer config file saved in ./distilbert-base-multilingual-cased-sentiments-student/tokenizerconfig.json [INFO|tokenizationutilsbase.py:2178] 2023-05-06 11:05:18,905 >> Special tokens file saved in ./distilbert-base-multilingual-cased-sentiments-student/specialtokens_map.json

### Framework versions

- Transformers 4.28.1
- Pytorch 2.0.0+cu118
- Datasets 2.11.0
- Tokenizers 0.13.3