sheng22213/pku-safe-rlhf-masked-arm-round4-extreme-v2-training
PKU SafeRLHF Masked ARM Round4 Extreme V2 Training Bundle This bundle contains the training data and code for the masked_round4_extreme_v2 ARM training runs. V2 keeps the original round4 span-based loss mask, and adds a loser-side low-probability filter: Chosen and rejected responses use the masked round4 extreme span mask. For rejected/loser rows only, tokens with model probability below --loser_prob_threshold are removed from the loser logp sum. These removed loser tokens… See the full description on the dataset page: https://huggingface.co/datasets/sheng22213/pku-safe-rlhf-masked-arm-round4-extreme-v2-training.
Conversations for this repository live on Hugging Face.
CoolFace shows imported repositories read-only. Posting into someone else’s repository from here would need an authorised integration and the account holder’s consent, so the link goes to the source instead.
Open discussions on Hugging Face