Modelpublic

theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness

sourceHugging Faceupdated 7d agoView on Hugging Face

0likes747downloads

Model Card

theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness

Reward model trained for the harmlessness preference on HH-RLHF harmlessness; original HH train only.

Model

—Type: oracle
—Base checkpoint: meta-llama/Llama-3.1-8B
—Label type: hard
—Architecture: LlamaForSequenceClassification
—Tokenizer: PreTrainedTokenizerFast
—Transformers compatibility: transformers>=4,<5

Evaluation

Validation metrics are computed on the temporary eval split created by the training sweep.

metric	value
Raw validation accuracy	n/a
Eval loss	n/a
Eval margin	n/a
Mean reward	n/a

For proxy models, the validation labels are produced by the corresponding oracle reward model.

Loading

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer

tokenizer = AutoTokenizer.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness")
model = AutoModelForSequenceClassification.from_pretrained("theSOL1/llama-3.1-8b-oracle-rm-hh-rlhf-harmlessness", num_labels=1)

This repository intentionally contains only model, tokenizer, and configuration files. Optimizer states, scheduler states, checkpoints, and trainer state files are not uploaded.