TuringEnterprises/Open-RL
Open-RL Dataset Summary This dataset contains self-contained, verifiable, and unambiguous STEM reasoning problems across Physics, Mathematics, Biology, and Chemistry. Each problem: Requires multi-step reasoning Involves symbolic manipulation and/or numerical computation Has a deterministic, objectively verifiable final answer The problems were evaluated against contemporary large language models. Observed pass rates indicate that the tasks are non-trivial yet… See the full description on the dataset page: https://huggingface.co/datasets/TuringEnterprises/Open-RL.
185168
1---2license: mit3language:4- en5tags:6- chemistry7- physics8- math9- biology10- science11pretty_name: open-rl12size_categories:13- n<1K14task_categories:15- question-answering16---17 18# Open-RL19 20[](https://opensource.org/licenses/MIT)21[](https://turing.com)22 23---24 25## Dataset Summary26 27This dataset contains **self-contained, verifiable, and unambiguous STEM reasoning problems** across Physics, Mathematics, Biology, and Chemistry.28 29Each problem:30 31* Requires multi-step reasoning32* Involves symbolic manipulation and/or numerical computation33* Has a deterministic, objectively verifiable final answer34 35The problems were evaluated against contemporary large language models. Observed pass rates indicate that the tasks are **non-trivial yet solvable**, placing them within reach of advanced models while still exposing meaningful reasoning gaps.36 37This makes the dataset particularly suitable for:38 39* Reinforcement learning (RL) fine-tuning40* Reward modeling41* Outcome-supervised training42* Verifiable reasoning benchmarks43 44---45 46## Dataset Structure47 48| Field | Type | Description |49| ----------------- | ------ | ----------------------------------------- |50| `conversation_id` | string | Unique identifier for each QA pair. |51| `domain` | string | Physics, Math, Chemistry, Biology. |52| `sub_domain` | string | Specific discipline. |53| `question` | string | STEM problem statement (LaTeX supported). |54| `answer` | string | Deterministic ground-truth solution. |55 56---57 58## Example59 60```json61{62 "conversation_id": "217998",63 "domain": "Physics",64 "sub_domain": "Astrophysics",65 "question": "Consider a Navarro–Frenk–White (NFW) dark matter halo profile where...",66 "answer": "\( \frac{4GM_{0}}{r_{0}} + \frac{16\pi Gk}{r_{0}}\left[ \ln\left(\frac{r_{0}}{r_{s}}\right) + 0.31 \right] \)"67}68```69 70---71 72## Verifiability and Automatic Grading73 74A core design principle of this dataset is **objective verifiability**.75 76Each problem is constructed such that:77 78* The final answer is deterministic79* Correctness can be evaluated programmatically80* No subjective interpretation is required81* There is a clear separation between reasoning steps and final outcome82 83### Answer Types84 85The dataset includes answers that are:86 87* Closed-form symbolic expressions88* Numerical scalars89* Algebraic identities90* Simplified analytic forms91* Canonical LaTeX representations92 93Because answers are deterministic, evaluation can be performed via:94 95* Exact string matching (after normalization)96* Symbolic equivalence checking (e.g., SymPy)97* Numerical tolerance comparison98* Unit consistency validation (where applicable)99 100---101 102## Data Quality Assurance Process103 104To ensure scientific validity of the answer, all tasks are prepared and reviewed twice by PhD experts.105 106Key quality rubrics include:107 108* Prompt and answer accuracy 109* Clarity of prompt and underlying reasoning 110* Expert-verified model breaking cases due to model’s incorrect reasoning process111* Google-proof originality validation.112 113---114 115## Reinforcement Learning and Outcome Supervision116 117This dataset is designed to support **outcome-based reinforcement learning** for reasoning models.118 119In contrast to preference-based RL (RLHF), which relies on subjective ranking signals, this dataset enables:120 121* Outcome-supervised reinforcement learning (OSRL)122* Deterministic reward assignment123* Binary or graded correctness rewards124* Scalable automated evaluation125 126### Example RL Setup127 128Given:129 130* Prompt: `question`131* Model output: predicted final answer132 133Reward can be computed as:134 135* `+1` if the final answer matches ground truth136* `0` or `-1` otherwise137* Optional partial credit via symbolic or numerical closeness138 139This allows:140 141* Policy gradient methods (e.g., PPO)142* Direct optimization against correctness signals143* Reward model bootstrapping144* Iterative self-improvement pipelines145 146### Calibration Regime147 148The problems were stress-tested against advanced language models and found to be:149 150* Not trivially solved151* Not universally failed152* Within the capability frontier of modern LLMs153 154This places them in a **learning-efficient regime**:155 156* Hard enough to produce gradient signal157* Solvable enough to avoid reward sparsity158* Suitable for curriculum-style training159 160---161 162 163## Future Directions: NuRL and Structured Nudging164 165We plan to extend this dataset with additional problem sets and a structured **"nudge" augmentation layer** inspired by the paper *["Nudging the Boundaries of LLM Reasoning"](https://arxiv.org/html/2509.25666v1)*.166 167### Motivation168 169Standard online RL algorithms (e.g., GRPO-style approaches) can only learn from problems where the model occasionally produces correct rollouts. For sufficiently difficult problems with a **0% pass rate**, no reward signal is generated, and therefore no gradient updates occur. As a result, such problems cannot contribute to expanding the model’s reasoning frontier.170 171### NuRL-Style Nudging172 173To address this limitation, future versions of this dataset will include:174 175* Abstract, high-level **hints ("nudges")**176* Hints generated conditionally using the gold answer177* Carefully designed cues that reduce problem difficulty without revealing the solution178 179Under a NuRL-style training pipeline:180 1811. Rollouts are first generated without hints.1822. If pass rate > 0%, standard RL proceeds.1833. If pass rate = 0%, a structured hint is injected.1844. A new batch of trajectories is generated with the hint.185 186This enables:187 188* Previously unsolvable samples to produce non-zero rewards189* Learning signal from frontier-level problems190* Expansion of the model’s upper reasoning bound191 192### Design Principles for Effective Nudges193 194Planned nudges will follow empirical findings from prior work:195 196* Hints should be **abstract and knowledge-oriented**, not answer-revealing197* Hints should preserve distributional alignment with base policy reasoning198* Hints should be injected only when necessary199* Nudges are most effective after base RL convergence200 201---202 203This evolution positions the dataset not only as a verifiable benchmark, but as a controlled testbed for **upper-bound expansion in reinforcement learning for reasoning models**.204 205---206 207## Citation208 209```bibtex210@dataset{turing_2026_open_rl,211 title = {Open-RL },212 author = {Saurabh Patil, Anshuman Lall, Marko Pavlovic , Chinmayee Shukla, Seetesh Pande, Tejass Mohan Ukarde , Amanda Gollo Bertollo, Mahesh Joshi, Kihwan Han},213 year = {2026},214 url = {https://huggingface.co/datasets/TuringEnterprises/Open-RL}215}216```