cwinkler/distilbert-base-uncased-finetuned-greenpatent

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

Classification of patent title - "green" or "no green"

This model classifies patents into "green patents" or "no green patents" by their titles.

Examples of "green patents" titles:

• —"A method for recycling waste" - score: 0.714
• —"A method of reducing pollution" - score: 0.786
• —"An apparatus to improve environmental aspects" - score: 0.570
• —"A method to improve waste management" - score: 0.813
• —"A device to use renewable energy sources" - score: 0.98
• —"A technology for efficient electrical power generation"- score: 0.975
• —"A method for the production of fuel of non-fossil origin" - score: 0.975
• —"Biofuels from waste" - score: 0.88
• —"A combustion technology with mitigation potential" - score: 0.947
• —"A device to capture greenhouse gases" - score: 0.871
• —"A method to reduce the greenhouse effect" - score: 0.887
• —"A device to improve the climate" - score: 0.650
• —"A device to stop climate change" - score: 0.55

Examples of "no green patents" titles:

• —"A device to destroy the nature" - score: 0.19
• —"A method to produce smoke" - score: 0.386

Examples of the model's limitation

• —"A method to avoid trash" - score: 0.165
• —"A method to reduce trash" - score: 0.333
• —"A method to burn the Amazonas" - score: 0.501
• —"A method to burn wood" - score: 0.408
• —"Green plastics" - score: 0.126
• —"Greta Thunberg" - score: 0.313 (How dare you, model?); BUT: "A method of using Greta Thunberg to stop climate change" - score: 0.715

Examples were inspired by https://www.epo.org/news-events/in-focus/classification/classification.html

distilbert-base-uncased-finetuned-greenpatent

This model is a fine-tuned version of distilbert-base-uncased on the green patent dataset. The green patent dataset was split into 70 % training data and 30 % test data (using ".traintestsplit(test_size=0.3)"). The model achieves the following results on the evaluation set:

• —Loss: 0.3148
• —Accuracy: 0.8776
• —F1: 0.8770

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 2e-05
• —trainbatchsize: 64
• —evalbatchsize: 64
• —seed: 42
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —num_epochs: 2

Training results

Framework versions

• —Transformers 4.25.1
• —Pytorch 1.13.1+cpu
• —Datasets 2.8.0
• —Tokenizers 0.13.2