theprint/GeneralChat-Llama3.2-3B-DPO

[image]

GeneralChat-Llama3.2-3B-DPO

A DPO fine-tuned version of theprint/GeneralChat-Llama3.2-3B.

Description

GeneralChat-Llama3.2-3B, a general-purpose conversational fine-tune of Llama 3.2 3B.

This model was trained with Direct Preference Optimization (DPO) on the theprint/Tom-4.2k-alpaca dataset. Rejected responses were generated using a weak local model to create preference pairs, with chosen responses drawn from the original dataset.

Quick Start

from transformers import pipeline

generator = pipeline("text-generation", model="theprint/GeneralChat-Llama3.2-3B-DPO", device="cuda")
output = generator(
    [{"role": "user", "content": "Your prompt here"}],
    max_new_tokens=256,
    return_full_text=False
)[0]
print(output["generated_text"])

Training Details

Framework Versions

• —TRL: 0.29.0
• —Transformers: 5.3.0
• —PyTorch: 2.12.0.dev20260310+cu128
• —Datasets: 4.5.0
• —PEFT: 0.15.2