prithivMLmods/GRAM-Qwen3-4B-RewardModel-GGUF

GRAM-Qwen3-4B-RewardModel-GGUF

GRAM-Qwen3-4B-RewardModel is a generative reward model developed to address reward generalization for Large Language Models (LLMs), released by NiuTrans. Unlike traditional models that depend heavily on task-specific labeled data, this model leverages both labeled and unlabeled data—a novel approach that allows it to generalize better across various tasks. It introduces a generative reward model framework that pre-trains on large amounts of unlabeled data and is subsequently fine-tuned with supervised data. The methodology also employs label smoothing and a regularized ranking loss to further boost performance, effectively bridging the gap between generative and discriminative reward modeling techniques.

This model is built on the Qwen3-4B base and can be directly used or adapted for aligning LLMs without the need to train a reward model from scratch on extensive datasets. In evaluations on the JudgeBench benchmark—covering Chat, Code, Math, and Safety tasks—GRAM-Qwen3-4B-RewardModel achieves a competitive average score of 65.9, making it suitable for use as an open-source, plug-and-play reward model for a variety of LLM alignment scenarios. The repository provides usage instructions and demonstration code to facilitate immediate adoption for research and development purposes

Model Files

Quants Usage

(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)

Here is a handy graph by ikawrakow comparing some lower-quality quant types (lower is better):