yujackein/onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-step323

OneReason-8B LoRA: Item32K / User75 / Rec50 / WorldClean1601 / LR2e-4 / Step323

This is an experimental LoRA adapter for `OpenOneRec/OneReason-8B-pretrain-competition`, trained for the OneReason recommendation competition.

This repository contains the intermediate checkpoint at step 323 from a 646-step, two-epoch-schedule run. It is the state after one packing pass, but it is not equivalent to a standalone one-epoch run because warmup and cosine decay were configured over all 646 steps.

Data recipe

All assistant loss weights were normalized to 1.0. The 32,768-token packing plan covers 99.1727% of rendered tokens in one safe pass.

Training

• —Base model: OneReason-8B pretrain competition release
• —Method: LoRA (r=32, alpha=32, dropout 0.05)
• —Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
• —Precision: BF16
• —Context length: 32,768
• —Global batch size: 16 on 4 x NVIDIA A800 80GB
• —Learning rate: 2e-4, cosine schedule, 3% warmup
• —Full schedule: 646 optimizer steps
• —This checkpoint: step 323
• —Seed: 42

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "OpenOneRec/OneReason-8B-pretrain-competition"
adapter_id = "yujackein/onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-step323"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

Status

Formal evaluation results have not yet been added. This checkpoint is kept separately from the final step-646 adapter for controlled comparison.