abhi11nav/codebert-gpt2-commitgen

1---2tags:3- generated_from_trainer4- seq2seq5model-index:6- name: codebert-gpt2-commitgen7  results: []8language:9- en10metrics:11- rouge12---13 14<!-- This model card has been generated automatically according to the information the Trainer had access to. You15should probably proofread and complete it, then remove this comment. -->16 17# codebert-gpt2-commitgen18 19This model is a fine-tuned version [](https://huggingface.co/) on  dataset provided in the paper titled "Towards Automatic Generation of Short Summaries of Commits" by 20Siyuan Jiang and Collin McMillan. 21Heres are the links22 23Paper :https://arxiv.org/abs/1708.0949224Data : https://sjiang1.github.io/commitgen25 26## Model description27 28This is a sequence2sequence model with microsoft/codebert-base as encoder and gpt2 as decoder. Givena gitdiff file, this model can generate a short commit message summarizing the change.29 30 31## Intended uses & limitations32 33The intended use is to automate github commit message. One limitation to consider is that the model can generate a summary of changes, but is only confined to type of change and might not be able to provide details about the change or output specific keywords related to change. 34 35## Training and evaluation data36 37More information needed38 39## Training procedure40 41### Training hyperparameters42 43The following hyperparameters were used during training:44- learning_rate: 5e-0545- train_batch_size: 1646- eval_batch_size: 1647- seed: 4248- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0849- lr_scheduler_type: linear50- lr_scheduler_warmup_steps: 200051- num_epochs: 352 53### Training results54 55- global_step=452156- training_loss=3.5599446506580457- train_runtime: 3300.049258- train_samples_per_second: 21.91959- train_steps_per_second: 1.3760- total_flos: 1.062667587499776e+1661- train_loss: 3.5599446506580462 63### Framework versions64 65- Transformers 4.25.166- Pytorch 1.13.0+cu11667- Tokenizers 0.13.2