yujackein/onereason-8b-lora-r0-fact-dense-long-caption-step41-r64a64

OneReason-8B LoRA: Fact-Dense Long Caption Residual, Step 41

This is an experimental LoRA adapter for `OpenOneRec/OneReason-8B-pretrain-competition`, prepared for formal OneReason platform evaluation.

It exactly represents the sum of:

1. 1.the rank-32 RL10 parent whose recorded formal platform score is 1.2402;
2. 2.a rank-32 residual checkpoint after 41 of 164 planned optimizer updates on locally cleaned, fact-dense SID-to-caption supervision.

The two branches are losslessly concatenated into one rank/alpha 64/64 adapter with alpha/r = 1. This is not a merged base model.

Residual experiment

• —Source rows: 301,951 across video, product, advertisement, and livestream
• —Unique SIDs: 170,142
• —Official/local evaluation SID overlap: zero in the recorded audit
• —Cleaning: deterministic deletion of narration templates and duplicate sentences; no external model or API was used
• —Retained supervised tokens: 95.889%
• —Context length: 32,768
• —Residual LoRA: rank/alpha 32/32, dropout 0.05
• —Updated layers: 0--34 across all seven linear modules
• —Frozen branch: layer 35 remained exactly zero
• —Optimizer: AdamW, LR 1e-4, cosine schedule, 3% warmup
• —Hardware: 4 x NVIDIA A800 80GB
• —Checkpoint: step 41/164, approximately one quarter of the packed pass

The frozen parent remained bit-identical, the residual was nonzero only in the intended tensors, and the rank-concatenation slice verification passed.

Adapter SHA-256: 6151ed32612f2534039d5feab33696970d3712f43f085bdd53523e3540a7df7e

Local diagnostics

These are deterministic proxies, not official scores and not an LLM Judge. On 40 fixed raw-SID examples, this checkpoint had character-unigram F1 0.212361 versus 0.237865 for RL10, boilerplate presence 22.5% versus 72.5%, and one repetition loop that reached the 512-token cap. On the fixed 400-example local set, reference NLL was 1.472657 versus 1.398245 for RL10 (lower is better).

The local evidence therefore does not establish an improvement. This upload is intended to test whether the formal platform disagrees with those proxies.

Usage

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "OpenOneRec/OneReason-8B-pretrain-competition"
adapter_id = "yujackein/onereason-8b-lora-r0-fact-dense-long-caption-step41-r64a64"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

Evaluation

No formal platform score has been recorded for this checkpoint yet. The 1.2402 value belongs to its RL10 parent and must not be attributed to this adapter.