Modelpublic

Moonlight556/sokrates-qwen3-8b-prontoqa-oak-dpo-iter3

sourceHugging Faceotherupdated 10mo agoView on Hugging Face

0likes11downloads

Model Card

SOKRATES: Qwen3-8B PrOntoQA OaK-DPO Iteration 3

Best performing model from the SOKRATES OaK-DPO training loop, achieving 98.2% accuracy on PrOntoQA.

Model Details

Property	Value
Base Model	Qwen/Qwen3-8B
Training Data	PrOntoQA (14,346 examples)
Training Method	Options and Knowledge DPO (3 iterations)
Starting Point	`sokrates-qwen3-8b-prontoqa-oak-dpo-iter2`
Hardware	6× NVIDIA B200 (183GB VRAM each)

Performance

Stage	Accuracy	Step Validity
SFT (baseline)	93.3%	11.3%
DPO Iter 1	96.8%	44.7%
DPO Iter 2	98.1%	83.5%
DPO Iter 3	98.2%	91.8%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained(
    "Moonlight556/sokrates-qwen3-8b-prontoqa-oak-dpo-iter3",
    torch_dtype="bfloat16",
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(
    "Moonlight556/sokrates-qwen3-8b-prontoqa-oak-dpo-iter3"
)

Related Models

Model	Accuracy	Description
`sokrates-qwen3-8b-prontoqa-sft-optionized`	93.3%	SFT baseline
`sokrates-qwen3-8b-prontoqa-oak-dpo-iter1`	96.8%	First DPO iteration
`sokrates-qwen3-8b-prontoqa-oak-dpo-iter2`	98.1%	Second DPO iteration
`sokrates-qwen3-8b-prontoqa-oak-dpo-iter3`	98.2%	Best model