Wezenite/gemma-2b-trader-fp16

Gemma-2-2B Fine-tuned for Trading Journal Summarization

Model Description

This model is a fine-tuned version of google/gemma-2-2b, specifically trained to generate structured bullet-point summaries of trading journal entries and financial documents.

Training Details

• —Base Model: google/gemma-2-2b
• —Fine-tuning Method: LoRA (Low-Rank Adaptation)
• —LoRA Rank: 16
• —LoRA Alpha: 32
• —Target Modules: qproj, kproj, vproj, oproj, gateproj, upproj, down_proj
• —Training Epochs: 3
• —Learning Rate: 2e-05
• —Batch Size: 2 (effective: 16)
• —Training Precision: fp16

Performance Metrics

• —Training Loss: 0.4682
• —Validation Loss: 0.4266
• —Perplexity: 1.53

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "./gemma-2b-trader-fp16",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("./gemma-2b-trader-fp16")

prompt = '''### Instruction:
Summarize the following trading journal entry using structured bullet points and precise trading terminology.

### Input:
[Your trading entry here]

### Summary:
'''

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.7)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Data

Fine-tuned on the EDGAR-CORPUS-Financial-Summarization dataset for financial text domain adaptation.

Limitations

• —Optimized for trading/financial context
• —Best results with English text
• —May require domain-specific prompting for optimal output

Citation

If you use this model, please cite the original Gemma model and this fine-tuned version.