CoolFace
Modelpublic

bthomas/article2keyword2.1b_barthez-orangesum-title_finetuned16_for_mlm

sourceHugging Faceapache-2.0updated 4y agoView on Hugging Face
0likes18downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

article2keyword2.1bbarthez-orangesum-titlefinetuned16formlm

This model is a fine-tuned version of moussaKam/barthez-orangesum-title on an unknown dataset. It achieves the following results on the evaluation set:

  • —Loss: 0.0525

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 2e-05
  • —trainbatchsize: 4
  • —evalbatchsize: 4
  • —seed: 42
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —num_epochs: 16
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation Loss
0.29761.013530.0543
0.05662.027060.0509
0.04873.040590.0458
0.04334.054120.0456
0.045.067650.0460
0.03736.081180.0454
0.03557.094710.0465
0.03288.0108240.0474
0.03179.0121770.0470
0.0310.0135300.0488
0.028511.0148830.0489
0.027212.0162360.0500
0.026213.0175890.0510
0.025814.0189420.0511
0.024515.0202950.0522
0.023916.0216480.0525

Framework versions

  • —Transformers 4.21.1
  • —Pytorch 1.11.0
  • —Datasets 2.3.2
  • —Tokenizers 0.12.1