CoolFace
Modelpublic

sanju2007/phi2-earnings-summarizer-qlora

sourceHugging Faceapache-2.0updated 4mo agoView on Hugging Face
0likes6downloads
Model Card

Phi-2 QLoRA Fine-tuned — Earnings Call Summarizer

A QLoRA fine-tuned adapter on top of Microsoft's Phi-2 (2.7B) for summarizing financial earnings call excerpts into structured highlights.

Model Details

  • —Developed by: Sanjay R K
  • —Base model: microsoft/phi-2
  • —Model type: Causal LM with LoRA adapter
  • —Language: English
  • —License: Apache 2.0
  • —Fine-tuning method: QLoRA (NF4 4-bit quantization + LoRA)

Training Details

  • —Dataset: gbharti/finance-alpaca (1,000 samples)
  • —Epochs: 3
  • —Learning rate: 2e-4
  • —LoRA rank (r): 8
  • —LoRA alpha: 16
  • —Target modules: qproj, kproj, v_proj, dense
  • —Trainable parameters: 5,242,880 / 2,784,926,720 (0.19%)
  • —Hardware: NVIDIA Tesla T4 (Google Colab)
  • —Training time: ~29 minutes

Training Loss

EpochTraining LossValidation Loss
11.91231.7098
21.75301.6598
31.65551.6539

Evaluation Results

Evaluated on a held-out earnings call excerpt against a reference summary.

MetricBase Phi-2Fine-tuned
ROUGE-1—0.7368
ROUGE-2—0.5161
ROUGE-L—0.6316

How to Use

python
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb_config = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_use_double_quant=True,
    bnb_4bit_compute_dtype=torch.float16
)

tokenizer = AutoTokenizer.from_pretrained("sanju2007/phi2-earnings-summarizer-qlora")

base_model = AutoModelForCausalLM.from_pretrained(
    "microsoft/phi-2",
    quantization_config=bnb_config,
    device_map="auto",
    trust_remote_code=True
)

model = PeftModel.from_pretrained(base_model, "sanju2007/phi2-earnings-summarizer-qlora")
model.eval()

prompt = """### Instruction:
Summarize the key financial highlights from the following earnings call excerpt.

### Input:
Your earnings call text here.

### Response:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
with torch.no_grad():
    outputs = model.generate(**inputs, max_new_tokens=200,
                             temperature=0.7, do_sample=True,
                             pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Limitations

  • —Trained on only 1,000 samples — not production ready
  • —Performance may degrade on non-English earnings calls
  • —Based on Phi-2 which has a 2048 token context limit
  • —Not suitable for financial advice or decision making

Environmental Impact

  • —Hardware: NVIDIA Tesla T4
  • —Cloud provider: Google Colab
  • —Training time: ~29 minutes