9mark9/finbert-minilm-sentiment

finbert-minilm-sentiment

A compact financial-sentiment classifier — 33.4M parameters, 95.3% real test accuracy.

finbert-minilm-sentiment classifies financial text into negative / neutral / positive sentiment. It is a compact MiniLM encoder (microsoft/MiniLM-L12-H384-uncased, 33.4M parameters) fine-tuned on the Financial PhraseBank benchmark. All metrics below are real, measured numbers on a held-out test set that was never seen during training — no illustrative or synthetic figures.

Companion project: this is the production variant. The from-scratch educational transformer (~2M params, no pretrained weights, fully annotated) lives at github.com/shaikn6/nano-finbert — built to teach the internals; this model is the fine-tuned version tuned for real accuracy.

Results (held-out test set)

Evaluated on a stratified 15% test split (340 sentences) of Financial PhraseBank Sentences_AllAgree — held out completely during training and validation.

Per-class performance:

Dataset

• —Source: Financial PhraseBank (Malo et al., 2014), config Sentences_AllAgree — 2,264 sentences where all annotators agreed on the label (the cleanest subset).
• —Labels: 3-class sentiment (negative, neutral, positive).
• —Split: stratified 70 / 15 / 15 train / validation / test (random_state=42).
• —train = 1,584 · validation = 340 · test = 340 (held out)

Training details

Model selection used best validation accuracy; the reported figures are then computed once on the untouched test set.

Usage

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "9mark9/finbert-minilm-sentiment"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)
model.eval()

texts = [
    "The company reported record quarterly profit, beating analyst expectations.",
    "Net sales decreased by 12% and the firm warned of further losses ahead.",
    "The board will meet next Tuesday to review the quarterly schedule.",
]

enc = tokenizer(texts, padding=True, truncation=True, max_length=96, return_tensors="pt")
with torch.no_grad():
    probs = model(**enc).logits.softmax(-1)

for text, p in zip(texts, probs):
    label = model.config.id2label[int(p.argmax())]
    print(f"[{label:8s} {p.max():.2f}] {text}")
# [positive 0.97] The company reported record quarterly profit, ...
# [negative 0.98] Net sales decreased by 12% and the firm warned ...
# [neutral  0.99] The board will meet next Tuesday to review ...

Intended use

• —Sentiment scoring of short, formal financial text: news headlines, earnings summaries, analyst notes, regulatory filings.
• —Research, prototyping, and educational use as a compact, CPU-friendly alternative to 110M-parameter FinBERT.

Limitations

• —Trained on the `AllAgree` subset, which skews neutral (1,391 of 2,264). The minority negative class has the fewest examples; expect slightly lower reliability there.
• —Domain is formal financial reporting language (Reuters-style). Performance on informal text (social media, retail-investor slang, emojis) is not characterised by this benchmark.
• —Not financial advice. Outputs are sentiment labels, not buy/sell signals.
• —Max sequence length is 96 tokens; longer documents are truncated.

Reproducibility

Metrics are reproducible: reloading the published model.safetensors and re-running inference on the same stratified test split (random_state=42) reproduces 95.29% accuracy exactly.

Citation

Dataset:

@article{malo2014good,
  title={Good debt or bad debt: Detecting semantic orientations in economic texts},
  author={Malo, Pekka and Sinha, Ankur and Korhonen, Pekka and Wallenius, Jyrki and Takala, Pyry},
  journal={Journal of the Association for Information Science and Technology},
  volume={65}, number={4}, pages={782--796}, year={2014}
}

Contact

Maintainer: nagizaazs@gmail.com

License

MIT