huangrh9/Bagel-SpectraReward-AWM-Qwen3VL-30B-A3B

Bagel-SpectraReward-AWM-Qwen3VL-30B-A3B

Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation

LoRA adapter for BAGEL-7B-MoT, trained with SpectraReward (external reward MLLM = Qwen3-VL-30B-A3B-Instruct) and the AWM (Advantage-Weighted Model) reinforcement-learning algorithm for reasoning text-to-image generation.

• —Base model: ByteDance-Seed/BAGEL-7B-MoT
• —Method: SpectraReward (frozen MLLM scores generated images)
• —Reward model: Qwen/Qwen3-VL-30B-A3B-Instruct
• —RL algorithm: AWM
• —Adapter: LoRA (r = 32, α = 64) on the generation experts
• —Code: https://github.com/huangrh99/AlphaGRPO
• —License: Apache-2.0

What is SpectraReward?

SpectraReward evaluates a generated image by the mean image-conditioned prompt log-likelihood under a frozen pretrained MLLM: the image is given as visual context, the prompt is teacher-forced, and the per-token log-likelihoods are averaged into a scalar reward. A higher reward means the prompt is more predictable from the image. This checkpoint uses Qwen3-VL-30B-A3B-Instruct as the reward MLLM.

Usage

This is a PEFT LoRA adapter for BAGEL-7B-MoT (not a standalone model). Download it, then follow the Evaluation section of the AlphaGRPO repo with BAGEL_LORA_PATH pointing at the downloaded folder.

hf download huangrh9/Bagel-SpectraReward-AWM-Qwen3VL-30B-A3B --local-dir ./bagel-spectrareward-awm
export BAGEL_LORA_PATH=$(pwd)/bagel-spectrareward-awm

Note: the reward model (Qwen3-VL-30B-A3B-Instruct) is only used during training; inference with this adapter does not need it.

Reproduce

See the AlphaGRPO repo. Because the reward MLLM is large (30B-A3B), serve it on dedicated GPUs and point training to it via SPECTRAREWARD_URL (the same config switches to remote scoring):

# Reward-server node(s): serve the reward MLLM (8-way data-parallel here).
bash scripts/serve_spectrareward.sh Qwen/Qwen3-VL-30B-A3B-Instruct 0.0.0.0 18090 8

# Training side:
export SPECTRAREWARD_MODEL_ID=Qwen/Qwen3-VL-30B-A3B-Instruct
export SPECTRAREWARD_URL=http://<reward_server_ip>:18090
torchrun --nproc_per_node=8 alpha_grpo/train.py \
  --config config/bagel.py:spectrareward_t2i_awm

Citation

If you use this model, please cite SpectraReward (and AlphaGRPO, the underlying framework):

@article{huang2026readitback,
  title={Read It Back: Pretrained MLLMs Are Zero-Shot Reward Models for Text-to-Image Generation},
  author={Huang, Runhui and others},
  year={2026}
}

@inproceedings{huang2026alphagrpo,
  title={AlphaGRPO: Unlocking Self-Reflective Multimodal Generation in Unified Multimodal Models via Decompositional Verifiable Reward},
  author={Huang, Runhui and Wu, Jie and Yang, Rui and Liu, Zhe and Zhao, Hengshuang},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026}
}