peyterho/finbert-macro-sentiment

FinBERT Macro Sentiment

A fine-tuned ProsusAI/finbert (109M params, BERT-base) for 3-class sentiment analysis on financial, macroeconomic, and climate/ESG text. Trained on 20K samples from 5 financial NLP datasets.

This model is the default FinBERT head and the backbone of the macro-sentiment-finbert ensemble pipeline. The topic router selects it for general financial news and social/tweet content. It can also be used standalone as a drop-in upgrade for the original ProsusAI/finbert.

Why Fine-Tune FinBERT?

The original ProsusAI/finbert was trained only on Financial PhraseBank (Araci, 2019) — ~4,800 sentences from English financial news. By fine-tuning on a broader mix of 5 datasets spanning news headlines, financial tweets, auditor reports, financial QA, and climate disclosures, we significantly improve recall on minority classes and cross-domain generalization:

The original model's weakness is its strong neutral bias — it achieves decent accuracy by defaulting to "neutral" but badly under-predicts both positive and negative classes. Our broader training mix and class weighting fix this.

Quick Start

Standalone Usage

from transformers import pipeline

pipe = pipeline("text-classification", model="peyterho/finbert-macro-sentiment", top_k=None)

pipe("Tesla shares surged 15% after beating earnings expectations.")
# [[{'label': 'positive', 'score': 0.99}, {'label': 'neutral', 'score': 0.01}, {'label': 'negative', 'score': 0.00}]]

pipe("Markets crashed amid recession fears and massive layoffs.")
# [[{'label': 'negative', 'score': 0.99}, {'label': 'neutral', 'score': 0.01}, {'label': 'positive', 'score': 0.00}]]

pipe("The Federal Reserve raised rates by 75bps citing persistent inflation.")
# [[{'label': 'neutral', 'score': 0.91}, {'label': 'negative', 'score': 0.07}, {'label': 'positive', 'score': 0.02}]]

As Part of the Full Pipeline

Within the macro-sentiment-finbert pipeline, this model is the default head — selected for general financial news, social media, and any text that doesn't trigger the policy or climate routing keywords:

from macro_sentiment import MacroSentimentPipeline

pipe = MacroSentimentPipeline(device="cpu")

result = pipe("$AAPL crushed earnings, revenue up 12% YoY. Bullish!")
print(result.summary())
# Sentiment: Positive (+0.687) | Policy: Neutral (+0.000) | Crisis: Normal (0.000) | Domain: financial_news
print(result.head_used)
# "finbert"

result = pipe("The latest jobs report showed unemployment rising to 4.2%, above consensus.")
print(result.summary())
# Sentiment: Negative (-0.312) | Policy: Neutral (+0.000) | Crisis: Normal (0.052) | Domain: financial_news

Label Mapping

This model preserves FinBERT's native label encoding:

Note: FinBERT's label ordering (positive=0, negative=1, neutral=2) differs from the standard convention (negative=0, neutral=1, positive=2). The ensemble pipeline handles this remapping internally. If using standalone, interpret labels by name, not by ID.

Evaluation Results

In-Domain (Combined Test Set — 4,333 samples)

All three fine-tuned heads compared on the same held-out test split:

Per-Class Breakdown

              precision    recall  f1-score   support

    negative     0.8051    0.8945    0.8474       711
     neutral     0.9518    0.8925    0.9212      2613
    positive     0.8417    0.9118    0.8754      1009

    accuracy                         0.8973      4333
   macro avg     0.8662    0.8996    0.8813      4333
weighted avg     0.9021    0.8973    0.8984      4333

Balanced recall across all three classes (89–91%) is the key improvement over the original FinBERT, which suffered from strong neutral bias (~90%+ neutral recall but <50% for positive/negative).

Out-of-Domain: Financial Phrasebank (785 samples)

Evaluated on Jean-Baptiste/financial_news_sentiment_mixte_with_phrasebank_75 — PhraseBank (75% agreement) + financial news articles. Not in the training mix.

All three heads generalize well, with >92% accuracy on this OOD financial text benchmark.

Out-of-Domain: Stock News Headlines (30,150 samples)

Evaluated on ic-fspml/stock_news_sentiment — 5-class labels mapped to 3-class. Not in the training mix.

FinBERT's financial pre-training gives it an edge over ClimateBERT on stock-specific headlines. The overall lower numbers reflect the 5→3 class mapping noise and significant domain shift (short, noisy headlines vs. the longer sentences in the training mix).

Comparison Against Baselines

Training Data

Fine-tuned on 20,034 training samples combined from 5 public financial/climate sentiment datasets:

Label distribution (train): 15.9% negative, 58.4% neutral, 25.7% positive — handled via class-weighted loss.

Training Details

Class Weighting Strategy

Following the approach in the original FinBERT paper (Araci, 2019, Section 4.4), sqrt-inverse-frequency class weights are applied to the cross-entropy loss to counteract the ~58% neutral class imbalance:

weight[c] = √(N_total / N_class_c) / mean(weights)

This upweights the minority negative class (1.30×) and downweights the majority neutral class (0.68×) while keeping the positive class near unity (1.02×).

Training Curve

Validation loss bottomed at epoch 3, but F1 macro continued improving through epoch 5. The divergence between val loss and F1 suggests the model becomes less calibrated (overconfident) on later epochs while still improving classification boundaries — a common pattern with class-weighted loss. The best checkpoint (epoch 5) trades slightly worse calibration for better discrimination.

Role in the Ensemble Pipeline

This model serves as the default financial news head in the macro-sentiment-finbert pipeline. It is selected when no specialized routing keyword is triggered, making it the workhorse for general financial content.

Input Text → Topic Router → policy keywords?     → RoBERTa-Large
                          → climate keywords?    → ClimateBERT
                          → non-English text?    → XLM-RoBERTa
                          → default (this model) → FinBERT ★

The pipeline fuses this model's prediction with four dictionary-based signals (Loughran-McDonald, Henry earnings tone, Sautner-style climate exposure, and macro policy/crisis dictionaries) using crisis-adaptive weighting — during crisis language, more weight shifts to the dictionary composite, since crisis keywords carry clearer signal than neural softmax outputs.

Ensemble Siblings

Custom Fine-Tuning

Fine-tune this model further on your own financial labelled data:

pip install transformers datasets accelerate evaluate torch

python -m macro_sentiment.finetune \
    --data my_labels.csv \
    --text-column headline \
    --label-column sentiment \
    --base-model peyterho/finbert-macro-sentiment \
    --output my-org/my-custom-model \
    --push-to-hub \
    --epochs 4 \
    --lr 1e-5 \
    --batch-size 32

The fine-tuning script (from the parent repo) accepts CSV, TSV, JSON, or JSONL. Labels can be strings ("positive", "negative", "neutral", "bullish", "bearish") or integers (0/1/2). Automatic label remapping handles FinBERT's non-standard label ordering (positive=0, negative=1, neutral=2). Class-weighted loss is applied by default.

Limitations

• —English only — FinBERT was pre-trained on English financial text (Reuters TRC2). Non-English text should use the multilingual head in the full pipeline.
• —128-token max length — fine-tuned with 128-token truncation. Longer documents (earnings calls, 10-K filings) should be chunked at paragraph or sentence level before scoring.
• —Neutral class dominance — despite class weighting, neutral precision (0.95) is noticeably higher than positive (0.84) or negative (0.81), reflecting the ~58% neutral training distribution. The model remains slightly conservative on polarity predictions.
• —Label noise — the FiQA dataset uses continuous sentiment scores thresholded at ±0.15, introducing boundary noise. The Twitter dataset contains informal language, $cashtags, and abbreviations that differ substantially from formal financial text.
• —No temporal or entity awareness — treats each text independently. Cannot reason about whether "rates rising" is positive (for banks) or negative (for growth stocks) without entity context.
• —FinBERT label ordering — label IDs are {0: positive, 1: negative, 2: neutral}, which differs from the standard convention. Use label names, not IDs, when interpreting results.
• —Parameter count trade-off — at 109M parameters, this model sits between ClimateBERT (82M) and RoBERTa-Large (355M). For maximum accuracy (+1.6 F1 points), use the RoBERTa-Large head; for minimum latency (−25% params), use ClimateBERT.

Citation

@article{araci2019finbert,
    title={FinBERT: Financial Sentiment Analysis with Pre-Trained Language Models},
    author={Araci, Dogu},
    journal={arXiv preprint arXiv:1908.10063},
    year={2019}
}

@article{loughran2011liability,
    title={When is a liability not a liability? Textual analysis, dictionaries, and 10-Ks},
    author={Loughran, Tim and McDonald, Bill},
    journal={The Journal of Finance},
    volume={66},
    number={1},
    pages={35--65},
    year={2011}
}

@article{henry2008earnings,
    title={Are investors influenced by how earnings press releases are written?},
    author={Henry, Elaine},
    journal={Journal of Business Communication},
    volume={45},
    number={4},
    pages={363--407},
    year={2008}
}

Framework Versions

• —Transformers 5.6.2
• —PyTorch 2.11.0+cu130
• —Datasets 4.8.4
• —Tokenizers 0.22.2

License

Apache 2.0