redugo/Chem-RLVR

Chem-RLVR

Chem-RLVR is a benchmark for reasoning over experimental reaction records, with direct reaction-yield prediction and counterfactual yield-reasoning tasks.

Paper: "Chem-RLVR: Verifier-Based Training for Reaction Yield Prediction"

Dataset

Chem-RLVR contains 12,000 questions:

• —6,000 yield-prediction questions
• —6,000 counterfactual questions

Each task contains:

• —4,200 RL-training examples
• —1,800 frozen held-out evaluation examples

The benchmark covers three reaction families:

• —Buchwald-Hartwig C-N coupling
• —Suzuki-Miyaura coupling
• —amide coupling

The released eval split is the frozen held-out evaluation split used for the experiments reported in the paper. Its labels are public in this release and it should not be interpreted as a permanently hidden test set.

Structured answer keys

answer_keys/prediction/ contains the final structural-v3 prediction Layer-3 answer keys.

answer_keys/counterfactual/ contains the final structural-v1 counterfactual Layer-3 answer keys.

Code

Generation, evaluation, structural scoring, and RLVR training code are released separately in the Chem-RLVR GitHub repository.

Source data and licensing

Chem-RLVR is derived from reaction records distributed through the Open Reaction Database (ORD). The released benchmark dataset is distributed under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

See metadata/SOURCES.md for source-dataset provenance, attribution, modifications made in Chem-RLVR, and licensing details.

Citation

Citation information will be updated with the arXiv identifier and archival DOI after publication.