CoolFace
Modelpublic

pszemraj/deberta-v3-small-sp500-edgar-10k

sourceHugging Facemitupdated 9mo agoView on Hugging Face
0likes18downloads
Model Card

<!-- This model card has been generated automatically according to the information the Trainer had access to. You should probably proofread and complete it, then remove this comment. -->

pszemraj/deberta-v3-small-sp500-edgar-10k

this predicts the ret column of the training dataset, given the text column.

<details>

<summary>Click to expand code example</summary>

py
import json
from transformers import pipeline
from huggingface_hub import hf_hub_download

model_repo_name = "pszemraj/deberta-v3-small-sp500-edgar-10k"
pipe = pipeline("text-classification", model=model_repo_name)
pipe.tokenizer.model_max_length = 1024

# Download the regression_config.json file
regression_config_path = hf_hub_download(
    repo_id=model_repo_name, filename="regression_config.json"
)
with open(regression_config_path, "r") as f:
    regression_config = json.load(f)

def inverse_scale(prediction, config):
    """apply inverse scaling to a prediction"""
    min_value, max_value = config["min_value"], config["max_value"]
    return prediction * (max_value - min_value) + min_value

def predict_with_pipeline(text, pipe, config, ndigits=5):
    result = pipe(text, truncation=True)[0] 
    scaled_score = inverse_scale(result['score'], config)
    return round(scaled_score, ndigits)

text = "This is an example text for regression prediction."

# Get predictions
predictions = predict_with_pipeline(text, pipe, regression_config)
print("Predicted Value:", predictions)

</details>

Model description

This model is a fine-tuned version of microsoft/deberta-v3-small on BEE-spoke-data/sp500-edgar-10k-markdown

It achieves the following results on the evaluation set:

  • —Loss: 0.0005
  • —Mse: 0.0005

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • —learning_rate: 2e-05
  • —trainbatchsize: 4
  • —evalbatchsize: 4
  • —seed: 30826
  • —gradientaccumulationsteps: 16
  • —totaltrainbatch_size: 64
  • —optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • —lrschedulertype: linear
  • —lrschedulerwarmup_ratio: 0.05
  • —num_epochs: 3.0
  • —mixedprecisiontraining: Native AMP

Training results

Training LossEpochStepValidation LossMse
0.00640.54500.00060.0006
0.00431.081000.00050.0005
0.00281.611500.00060.0006
0.00252.152000.00050.0005
0.00252.692500.00050.0005

Framework versions

  • —Transformers 4.38.0.dev0
  • —Pytorch 2.2.0+cu121
  • —Datasets 2.16.1
  • —Tokenizers 0.15.2