Modelpublic

Nagi-ovo/Qwen2.5-7B-Reasoning-Adapter

sourceHugging Faceapache-2.0updated 2y agoView on Hugging Face

0likes15downloads

Model Card

This model uses reinforcement learning to train on the GSM8K dataset, generating reasoning chains and formatted outputs despite the dataset lacking intermediate steps. A reward function guides the model, prioritizing answer correctness and XML format adherence.

Training Details:

—Dataset: GSM8K
—Algorithm: GRPO
—Hardware: Single NVIDIA GeForce RTX 3090 Ti
—Training Duration: 250 epochs, ~48 minutes

image/png Limitations:

The output length limit(200) restricts the model's ability to generate complex reasoning chains, hindering observation of output length growth during training.

Example:

Which one is bigger? 9.11 or 9.8? image/png

This qwen2.5 model was trained 2x faster with Unsloth and Huggingface's TRL library.