HuggingFaceH4/Qwen2.5-Math-7B-Instruct-PRM-0.2

1---2base_model: Qwen/Qwen2.5-Math-7B-Instruct3datasets: HuggingFaceH4/prm800k-trl-dedup4library_name: transformers5model_name: Qwen2.5-Math-7B-Instruct-PRM-0.26tags:7- generated_from_trainer8- trl9- prm10licence: license11---12 13# Model Card for Qwen2.5-Math-7B-Instruct-PRM-0.214 15This model is a fine-tuned version of [Qwen/Qwen2.5-Math-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Math-7B-Instruct) on the [HuggingFaceH4/prm800k-trl-dedup](https://huggingface.co/datasets/HuggingFaceH4/prm800k-trl-dedup) dataset.16It has been trained using [TRL](https://github.com/huggingface/trl).17 18## Quick start19 20How to use the model:21 22```python23from transformers import pipeline24 25pipe = pipeline("token-classification", model="HuggingFaceH4/Qwen2.5-Math-7B-Instruct-PRM-0.2", device="cuda")26 27example = {28    "prompt": "Let $a,$ $b,$ and $c$ be positive real numbers.  Find the set of all possible values of\n\\[\\frac{c}{a} + \\frac{a}{b + c} + \\frac{b}{c}.\\]",29    "completions": [30        "This problem involves finding the range of an expression involving three variables.",31        "One possible strategy is to try to eliminate some variables and write the expression in terms of one variable only.",32        "To do this, I might look for some common factors or symmetries in the expression.",33        "I notice that the first and last terms have $c$ in the denominator, so I can factor out $c$ from the whole expression and get\n\\[\\frac{1}{c}\\left(c + \\frac{a^2}{b + c} + b\\right).\\]"34    ],35    "labels": [True, True, True, False],36}37 38 39separator = "\n\n"  # It's important to use the same separator as the one used during training40 41for idx in range(1, len(example["completions"]) + 1):42    steps = example["completions"][0:idx]43    text = separator.join((example["prompt"], *steps)) + separator  # Add a separator between the prompt and each steps44    pred_entity = pipe(text)[-1]["entity"]45    pred = {"LABEL_0": False, "LABEL_1": True}[pred_entity]46    label = example["labels"][idx - 1]47    print(f"Step {idx}\tPredicted: {pred} \tLabel: {label}")48 49# Step 1  Predicted: True         Label: True50# Step 2  Predicted: True         Label: True51# Step 3  Predicted: True         Label: True52# Step 4  Predicted: False        Label: False53```54 55## Training procedure56 57[<img src="https://raw.githubusercontent.com/wandb/assets/main/wandb-github-badge-28.svg" alt="Visualize in Weights & Biases" width="150" height="24"/>](https://wandb.ai/plaguss/huggingface/runs/lkcjyfs2)58 59This model was trained with PRM.60 61### Framework versions62 63- TRL: 0.13.0.dev064- Transformers: 4.47.065- Pytorch: 2.4.166- Datasets: 3.0.167- Tokenizers: 0.21.068 69## Citations70 71Cite PRM as:72 73```bibtex74@article{uesato2022solving,75    title        = {Solving Math Word Problems With Process- and Outcome-Based Feedback},76    author       = {Uesato, Jonathan and Kushman, Nate and Kumar, Ramana and Song, Francis and Siegel, Noah and Wang, Lisa and Creswell, Antonia and Irving, Geoffrey and Higgins, Irina},77    year         = 2022,78    journal      = {arXiv preprint arXiv:2211.14275}79}80```81 82Cite TRL as:83    84```bibtex85@misc{vonwerra2022trl,86	title        = {{TRL: Transformer Reinforcement Learning}},87	author       = {Leandro von Werra and Younes Belkada and Lewis Tunstall and Edward Beeching and Tristan Thrush and Nathan Lambert and Shengyi Huang and Kashif Rasul and Quentin Gallouédec},88	year         = 2020,89	journal      = {GitHub repository},90	publisher    = {GitHub},91	howpublished = {\url{https://github.com/huggingface/trl}}92}93```