skylord/pharma_classification

1---2license: apache-2.03base_model: distilbert-base-uncased4tags:5- generated_from_trainer6metrics:7- accuracy8- f19model-index:10- name: pharma_classification11  results: []12---13 14<!-- This model card has been generated automatically according to the information the Trainer had access to. You15should probably proofread and complete it, then remove this comment. -->16 17# pharma_classification18 19This model is a fine-tuned version of [distilbert-base-uncased](https://huggingface.co/distilbert-base-uncased) on the None dataset.20It achieves the following results on the evaluation set:21- Loss: 0.531522- Accuracy: 0.958123- F1: 0.950624 25## Model description26 27More information needed28 29## Intended uses & limitations30 31More information needed32 33## Training and evaluation data34 35More information needed36 37## Training procedure38 39### Training hyperparameters40 41The following hyperparameters were used during training:42- learning_rate: 5e-0543- train_batch_size: 844- eval_batch_size: 845- seed: 4246- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-0847- lr_scheduler_type: linear48- training_steps: 3000049 50### Training results51 52| Training Loss | Epoch | Step  | Validation Loss | Accuracy | F1     |53|:-------------:|:-----:|:-----:|:---------------:|:--------:|:------:|54| 0.0035        | 5.99  | 5000  | 0.2892          | 0.9539   | 0.9554 |55| 0.0137        | 11.98 | 10000 | 0.2620          | 0.9641   | 0.9600 |56| 0.0           | 17.96 | 15000 | 0.4022          | 0.9611   | 0.9586 |57| 0.0001        | 23.95 | 20000 | 0.3838          | 0.9611   | 0.9552 |58| 0.0           | 29.94 | 25000 | 0.4363          | 0.9575   | 0.9490 |59| 0.0           | 35.93 | 30000 | 0.5315          | 0.9581   | 0.9506 |60 61 62### Framework versions63 64- Transformers 4.39.0.dev065- Pytorch 2.2.0+cu12166- Datasets 2.18.067- Tokenizers 0.15.268