zhijunjunlin/github_cybersecurity_READMEs

1---2license: apache-2.03base_model: distilroberta-base4tags:5- generated_from_trainer6model-index:7- name: github_cybersecurity_READMEs8  results: []9---10 11<!-- This model card has been generated automatically according to the information the Trainer had access to. You12should probably proofread and complete it, then remove this comment. -->13 14# github_cybersecurity_READMEs15 16This model is a fine-tuned version of [distilroberta-base](https://huggingface.co/distilroberta-base) on an unknown dataset.17It achieves the following results on the evaluation set:18- Loss: 1.519019 20## Model description21 22More information needed23 24## Intended uses & limitations25 26More information needed27 28## Training and evaluation data29 30More information needed31 32## Training procedure33 34### Training hyperparameters35 36The following hyperparameters were used during training:37- learning_rate: 2e-0538- train_batch_size: 839- eval_batch_size: 840- seed: 4241- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0842- lr_scheduler_type: linear43- num_epochs: 3.044 45### Training results46 47| Training Loss | Epoch | Step | Validation Loss |48|:-------------:|:-----:|:----:|:---------------:|49| 1.9275        | 1.0   | 3043 | 1.7824          |50| 1.7237        | 3.0   | 7662 | 1.5187          |51 52 53### Framework versions54 55- Transformers 4.40.0.dev056- Pytorch 2.2.1+cu12157- Datasets 2.18.058- Tokenizers 0.15.259