automationkasey/projectx-economic-brain-dpo

ProjectX Economic Brain — Phase 2 (DPO)

Direct Preference Optimization fine-tuning of Qwen/Qwen2.5-7B-Instruct for financial decision-making.

Training Details

• —Base: automationkasey/projectx-economic-brain-sft (Phase 1 SFT)
• —Method: DPO with LoRA (r=16, alpha=32)
• —Dataset: ProjectX DPO dataset (738 train / 83 eval)
• —Epochs: 1
• —Learning rate: 1e-5 (cosine)
• —Beta (DPO temperature): 0.3
• —Batch: 1 (gradient accumulation 4 → effective 4)

Training Metrics

• —Final loss: ~0.0023
• —Reward margin (chosen - rejected): ~7.0
• —Accuracy: 1.0 (always correctly distinguished chosen/rejected)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base, "automationkasey/projectx-economic-brain-dpo")
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")