kk08/CryptoBERT

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

CryptoBERT

This model is a fine-tuned version of ProsusAI/finbert on the Custom Crypto Market Sentiment dataset. It achieves the following results on the evaluation set:

• —Loss: 0.3823

from transformers import BertTokenizer, BertForSequenceClassification
from transformers import pipeline

tokenizer = BertTokenizer.from_pretrained("kk08/CryptoBERT")
model = BertForSequenceClassification.from_pretrained("kk08/CryptoBERT")

classifier = pipeline("sentiment-analysis", model=model, tokenizer=tokenizer)
text = "Bitcoin (BTC) touches $29k, Ethereum (ETH) Set To Explode, RenQ Finance (RENQ) Crosses Massive Milestone"
result = classifier(text)
print(result)

[{'label': 'LABEL_1', 'score': 0.9678454399108887}]

Model description

This model fine-tunes the ProsusAI/finbert, which is a pre-trained NLP model to analyze the sentiment of the financial text. CryptoBERT model fine-tunes this by training the model as a downstream task on Custom Crypto Sentiment data to predict whether the given text related to the Crypto market is Positive (LABEL1) or Negative (LABEL0).

Intended uses & limitations

The model can perform well on Crypto-related data. The main limitation is that the fine-tuning was done using only a small corpus of data

Training hyperparameters

The following hyperparameters were used during training:

• —learning_rate: 5e-05
• —trainbatchsize: 16
• —evalbatchsize: 8
• —seed: 42
• —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• —lrschedulertype: linear
• —num_epochs: 10

Training results

Framework versions

• —Transformers 4.28.0
• —Pytorch 2.0.0+cu118
• —Datasets 2.11.0
• —Tokenizers 0.13.3